Object detection device

Abstract

To properly calculate a lap rate of another vehicle entering the own lane.SOLUTION: An object detection device detects a pair of lane division lines and an another vehicle rear surface area which represents a rear surface of another vehicle that travels in front of a vehicle from each of multiple images showing an environment in front of the vehicle, generated at multiple times. The object detection device, when the pair of lane division lines is detected from the image, calculates a right lane division line rear surface lap rate and a left lane division line rear surface lap rate on the basis of each position of the lane division line and the other vehicle rear surface area detected from the image, and stores them in a storage unit. The object detection device, when one of the pair of lane division lines is not detected from the image, calculates a rear surface lap rate corresponding to the lane division line that is not detected on the basis of each position of the lane division line and the other vehicle rear surface area detected from the image, the right lane division line rear surface lap rate and the left lane division line rear surface lap rate calculated on the basis of the image generated at the past time.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to an object detection device that detects an object from an image.

Background Art

[0002] There is known a driving control device that automatically controls the driving of a vehicle based on a peripheral image generated by a camera mounted on the vehicle. The driving control device controls the driving of the vehicle based on a lap rate indicating the degree to which other vehicles enter from an adjacent lane into the lane in which the vehicle is traveling.

[0003] In Patent Document 1, when the lane dividing line changes from a state where it is detected to a state where it is not detected, a virtual lane dividing line is estimated based on the position of the lane dividing line detected in the past, and the driving of the host vehicle is controlled so as to be at a predetermined position with respect to the virtual lane dividing line. A driving control device is described.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In an in-vehicle camera image that captures a situation where another vehicle has entered halfway into the host vehicle's lane, the position of the lane dividing line estimated based on the position of the lane dividing line detected in the past may deviate from the position of the actual lane dividing line. When the position of the estimated lane dividing line deviates from the position of the actual lane dividing line, the lap rate cannot be appropriately calculated, and the driving of the vehicle cannot be appropriately controlled.

[0006] An object of the present disclosure is to provide an object detection device that can appropriately calculate the lap rate of other vehicles entering the host vehicle's lane.

Means for Solving the Problems

[0007] The gist of the present disclosure is as follows.

[0008] (1) A detection unit that detects, from each of a plurality of images representing the environment in front of a vehicle generated at a plurality of times, a pair of lane dividing lines and an other-vehicle rear region in which the rear of another vehicle traveling in front of the vehicle is represented; When both of the pair of lane dividing lines are detected from one of the plurality of images, a right lane-dividing line rear wrap rate and a left lane-dividing line rear wrap rate are calculated based on the positions of the lane dividing lines and the other-vehicle rear region detected from the one image, and are stored in a storage unit. When one of the pair of lane dividing lines is not detected from the one image, a rear wrap rate corresponding to the lane dividing line not detected is calculated based on the positions of the lane dividing line and the other-vehicle rear region detected from the one image and the right lane-dividing line rear wrap rate and the left lane-dividing line rear wrap rate calculated based on an image generated at a time past the time when the one image was generated among the plurality of images; a calculation unit; An object detection device comprising the above.

[0009] (2) A detection unit that detects, from each of a plurality of images representing the situation in front of a vehicle generated at a plurality of times, a pair of lane dividing lines that respectively demarcate the left end and the right end of the lane in which the vehicle travels and an other-vehicle rear region in which the rear of another vehicle traveling in front of the vehicle is represented; If both of the pair of lane markings are detected in the image generated at one of the multiple time points, the right lane marking overlap rate is calculated by subtracting the value of the lateral coordinate of the lower left end of the rear area of ​​the other vehicle from the value of the lateral coordinate of the position corresponding to the lower end of the rear area of ​​the other vehicle of the right lane marking, and dividing the result by the lateral length of the rear area of ​​the other vehicle. The left lane marking overlap rate is calculated by subtracting the value of the lateral coordinate of the position corresponding to the lower end of the rear area of ​​the other vehicle of the left lane marking, of the pair of lane markings, from the value of the lateral coordinate of the lower right end of the rear area of ​​the other vehicle, and dividing the result by the lateral length of the rear area of ​​the other vehicle. The calculated right lane marking overlap rate and the left lane marking overlap rate are stored in the storage unit. If one of the pair of lane markings is not detected in the image generated at one of the multiple time points, the image generated at a time earlier than one of the multiple time points is stored in the storage unit. A calculation unit calculates a rear overlap rate corresponding to one of the pair of lane markings that were not detected in the image generated at one time, by adding to the difference obtained by subtracting from An object detection device equipped with the following features.

[0010] According to the object detection device described herein, the overlap rate of other vehicles entering the same lane can be appropriately calculated. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of a vehicle equipped with an object detection device. [Figure 2] This is a schematic diagram of the ECU hardware. [Figure 3] This is a functional block diagram of the processor in the ECU. [Figure 4] This is a schematic diagram of an image representing the environment in front of the vehicle at the first time point. [Figure 5] This is a schematic diagram of an image representing the environment in front of the vehicle at the second time point. [Figure 6] This is a flowchart of the object detection process. [Modes for carrying out the invention]

[0012] The following describes in detail an object detection device that can appropriately calculate the overlap rate of other vehicles entering its own lane, with reference to the diagrams.

[0013] The object detection device detects, from multiple images representing the environment in front of the vehicle, generated at multiple times, a pair of lane markings that demarcate the left and right edges of the lane the vehicle is traveling in, and an area representing other vehicles traveling in front of the vehicle.

[0014] If both lane markings of a pair are detected in an image generated at one of several time points, the object detection device calculates the right lane marking rear overlap ratio by subtracting the value of the lateral coordinate of the lower left edge of the rear area of ​​the other vehicle from the value of the lateral coordinate of the position corresponding to the lower edge of the rear area of ​​the other vehicle of the right lane marking of the pair, and then dividing the result by the lateral length of the rear area of ​​the other vehicle.

[0015] Furthermore, the object detection device calculates the left lane marking rear overlap ratio by subtracting the value of the lateral coordinate of the position corresponding to the lower edge of the rear area of ​​the other vehicle on the left lane marking of a pair of lane markings from the value of the lateral coordinate of the lower right edge of the rear area of ​​the other vehicle, and then dividing the result by the lateral length of the rear area of ​​the other vehicle.

[0016] Furthermore, the object detection device stores the calculated right lane line overlap rate and left lane line overlap rate in its memory unit.

[0017] On the other hand, if one of a pair of lane markings is not detected in an image generated at a given time, the object detection device calculates a rear overlap rate corresponding to one of the pair of lane markings that is not detected in an image generated at a given time by adding the difference obtained by subtracting one of the left lane marking rear overlap rates and right lane marking rear overlap rates calculated using the lane marking that was not detected in the image generated at a given time from one of the left lane marking rear overlap rates and right lane marking rear overlap rates calculated using the lane marking that was detected in the image generated at a given time from one of the left lane marking rear overlap rates and right lane marking rear overlap rates calculated using the lane marking that was detected in the image generated at a given time from one of the left lane marking rear overlap rates and right lane marking rear overlap rates calculated using an image generated at a given time from an image generated at a given time.

[0018] Figure 1 is a schematic diagram of a vehicle equipped with an object detection device.

[0019] Vehicle 1 has a camera 2 and an ECU 3 (Electronic Control Unit). The ECU 3 is an example of an object detection device. Camera 2 and the ECU 3 are connected to communicate via an in-vehicle network compliant with standards such as a controller area network.

[0020] Camera 2 is an example of an imaging unit that generates an image representing the situation in front of the vehicle. Camera 2 has a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to infrared light, such as a CCD or C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. Camera 2 is positioned, for example, at the front upper part of the vehicle interior, facing forward, and photographs the situation in front of the vehicle 1 through the windshield at predetermined shooting cycles (e.g., 1 / 30 second to 1 / 10 second), and generates an image corresponding to the surrounding situation.

[0021] The ECU3 includes a communication interface, memory, and a processor. Based on images received from the camera 2 via the communication interface, the ECU3 detects other vehicles in front of vehicle 1 and calculates the overlap rate of other vehicles entering the lane in which vehicle 1 is traveling.

[0022] Figure 2 is a schematic hardware diagram of ECU3. ECU3 includes a communication interface 31, memory 32, and processor 33.

[0023] The communication interface 31 is an example of a communication unit and has a communication interface circuit for connecting the ECU 3 to the in-vehicle network. The communication interface 31 supplies the received data to the processor 33. The communication interface 31 also outputs the data supplied from the processor 33 to the outside.

[0024] Memory 32 is an example of a storage unit and includes volatile semiconductor memory and non-volatile semiconductor memory. Memory 32 stores various data used for processing by the processor 33, such as parameters of a classifier used to detect lane markings and other vehicle areas from images, and the right lane marking rear overlap rate and left lane marking rear overlap rate calculated based on images generated at past times. Memory 32 also stores various application programs, such as an object detection computer program that performs object detection processing.

[0025] The processor 33 is an example of a control unit and has one or more processors and their peripheral circuits. The processor 33 may further have other arithmetic circuits such as a logic unit, a numerical unit, or a graphics processing unit.

[0026] Figure 3 is a functional block diagram of the processor 33 in the ECU 3.

[0027] The processor 33 of the ECU3 has a detection unit 331 and a calculation unit 332 as functional blocks. Each of these parts of the processor 33 is a functional module implemented by a program executed on the processor 33. Alternatively, each of these parts of the processor 33 may be implemented in the ECU3 as an independent integrated circuit, microprocessor, or firmware.

[0028] The detection unit 331 receives multiple images from the camera 2 via a communication interface, which are generated at multiple times and represent the environment in front of the vehicle 1. From each of the received multiple images, the detection unit 331 detects a pair of lane markings that demarcate the left and right ends of the lane in which the vehicle 1 is traveling. Lane markings are lines displayed on the road to demarcate lanes. The detection unit 331 also detects from each of the received multiple images a region showing the rear of another vehicle traveling in front of the vehicle 1.

[0029] The detection unit 331 inputs the image received from the camera 2 into a classifier that has been pre-trained to detect lane markings and the rear area of ​​other vehicles, thereby identifying the lane markings that define the lane in which vehicle 1 is traveling and the rear area of ​​other vehicles that shows the rear of other vehicles ahead.

[0030] The classifier can be, for example, a convolutional neural network (CNN) having multiple convolutional layers connected in series from the input side to the output side. By inputting images containing lane markings and the rear areas of other vehicles as training data into the CNN and performing training, the CNN operates as a classifier that detects lane markings and the rear areas of other vehicles.

[0031] Figure 4 is a schematic diagram of an image representing the environment in front of the vehicle at the first time point. Figure 5 is a schematic diagram of an image representing the environment in front of the vehicle at the second time point. The first time point is earlier than the second time point.

[0032] At the first time point, vehicle 1, which is vehicle 1, is traveling in lane L1, which is demarcated by lane markings LL1 and LL2. Also at the first time point, another vehicle 10, which is different from vehicle 1, is traveling ahead of vehicle 1 in lane L2, which is demarcated by lane markings LL2 and LL3.

[0033] The detection unit 331 detects a pair of lane markings LL1 and LL2 from the image generated by the camera 2 at a first time step. Lane marking LL1 is a lane marking that demarcates the left side of lane L1 in which vehicle 1 is traveling. Lane marking LL2 is a lane marking that demarcates the right side of lane L1 in which vehicle 1 is traveling.

[0034] The detection unit 331 detects, from the image generated by the camera 2 at a first time point, the rear area 100 of another vehicle that is traveling in front of vehicle 1, which shows the rear of the other vehicle.

[0035] The detection unit 331 identifies the lateral coordinates corresponding to the lower end of the rear area 100 of the other vehicle, based on the lane markings and the rear area 100 of the other vehicle detected from the image. The lateral coordinates are expressed as relative positions in the image, with reference to a predetermined position on a straight line passing through the lower end of the rear area 100 of the other vehicle (for example, the position of the lane marking detected on the far left). In the example in Figure 4, the lateral coordinate of lane marking LL1 is H LL1 Therefore, the lateral coordinate of the lane marking LL2 is HLR1 Therefore, the horizontal coordinate of the lower left edge of the rear area 100 of the other vehicle is H VL1 Therefore, the horizontal coordinate of the lower right corner of the rear area 100 of the other vehicle is H VR1 That is the case.

[0036] At the second time point, vehicle 1, which is vehicle 1, is traveling in lane L1, which is demarcated by lane markings LL1 and LL2, just as at the first time point. At the second time point, other vehicle 10 is traveling from lane L2 ahead of vehicle 1, crossing lane marking LL2 and entering lane L1.

[0037] The detection unit 331 detects lane marking LL1 as the lane marking that demarcates the left edge of the lane in which vehicle 1 is traveling, from the image generated by camera 2 at the second time step. At the second time step, the detection unit 331 cannot detect the lane marking that demarcates the right edge from the image generated by camera 2 because lane marking LL2 is hidden by another vehicle 10.

[0038] The detection unit 331 detects, from the image generated by the camera 2 at a second time point, the rear area 200 of another vehicle 10 traveling in front of vehicle 1, which is shown on the rear of the other vehicle.

[0039] The detection unit 331 identifies the lateral coordinates corresponding to the lower end of the rear area 200 of the other vehicle, based on the lane markings and the rear area 200 of the other vehicle detected from the image. The lateral coordinates are expressed as relative positions in the image, with reference to a predetermined position on a straight line passing through the lower end of the rear area 200 of the other vehicle (for example, the position of the lane marking detected on the far left). In the example in Figure 5, the lateral coordinate of lane marking LL1 is H LL2 Therefore, the horizontal coordinates of the lower left edge of the rear area 200 of the other vehicle are H VL2 Therefore, the horizontal coordinate of the lower right corner of the rear area 200 of the other vehicle is H VR2 That is the case.

[0040] The calculation unit 332 calculates the right lane marking rear overlap ratio and the left lane marking rear overlap ratio based on the positions of the lane markings and the rear areas of other vehicles detected from the image.

[0041] In the image generated by the camera 2 at the first time, both of a pair of lane dividing lines that demarcate the lane L1 on which the vehicle 1 travels are detected. In this case, the calculation unit 332 calculates the coordinate H of the horizontal position of the lane dividing line LL2 on the right side of the lane L1 LR1 from the value of the coordinate H of the horizontal position of the left lower end of the rear area 100 of the other vehicle VL1 and divides the value obtained by subtracting the value of the coordinate H of the horizontal position of the left lower end of the rear area of the other vehicle (corresponding to the interval DR1 shown in FIG. 4) by the horizontal length of the rear area of the other vehicle (corresponding to the vehicle width WV1 shown in FIG. 4) to calculate the right dividing line rear wrap rate.

[0042] That is, the right dividing line rear wrap rate WR R1 at the first time can be expressed by the following formula (1). (1) WR R1 = (H LR1 -H VL1 ) / (H VR1 -H VL1 )

[0043] Further, the calculation unit 332 calculates the coordinate H of the horizontal position of the lane dividing line LL1 on the left side of the lane L1 from the value of the coordinate H of the horizontal position of the right lower end of the rear area 100 of the other vehicle VR1 and divides the value obtained by subtracting the value of the coordinate H of the horizontal position of the left side of the lane dividing line LL1 (corresponding to the interval DL1 shown in FIG. 4) by the horizontal length of the rear area of the other vehicle (corresponding to the vehicle width WV1 shown in FIG. 4) to calculate the left dividing line rear wrap rate. LL1

[0044] That is, the left dividing line rear wrap rate WR L1 at the first time can be expressed by the following formula (2). (2) WR L1 = (H VR1 -H LL1 ) / (H VR1 -H VL1 )

[0045] The calculation unit 332 stores the calculated right dividing line rear wrap rate WR R1 and the left dividing line rear wrap rate WR L1 in the memory 32 in association with the time when the image is generated.

[0046] In the second time step, the image generated by camera 2 does not detect the right lane marking LL2 of the pair of lane markings that demarcate the lane L1 in which vehicle 1 is traveling. In this case, the calculation unit 332 first calculates the left lane marking rear overlap ratio WR according to the above formula (2), based on the positions of the detected lane marking (lane marking LL1) and the rear area 200 of the other vehicle. L2 Calculate.

[0047] Here, the right lane line overlap rate WR at the first time step. R1 and left lane line rear overlap rate WR L1 The sum of these two can be expressed as shown in equation (3) below. (3) WR R1 +WR L1 = (H LR1 -H LL1 +H VR1 -H VL1 ) / (H VR1 -H VL1 )

[0048] H LR1 -H LL1 This corresponds to the distance between a pair of lane markings (lane width), H VR1 -H VL1 This corresponds to the vehicle width WV1. Since the lane width is constant for lane L1, and the vehicle width WV1 is constant for other vehicles 10, the value of equation (3) is constant regardless of the distance between vehicle 1 and other vehicles 10. Therefore, the right lane marking rear overlap rate WR corresponds to the first time. R1 and the left lane line back overlap rate WR L1 And the right lane line back overlap rate WR corresponding to the second time. R2 and the left lane line back overlap rate WR L2 The relationship can be expressed as shown in equation (4) below.

[0049] (4) WR R2 +WR L2 = WR R1 +WR L1 ∴ WR R2 = WR R1+WR L1 -WR L2

[0050] The calculation unit 332 calculates the rear overlap rate (right lane marking rear overlap rate WR) corresponding to one of the pair of lane markings that are not detected in the image, according to formula (4). R2 Calculate ).

[0051] In other words, the calculation unit 332 calculates the right lane marking rear overlap rate WR based on images from the memory 32 at a time earlier than the second time (for example, the first time) in which both lane markings of the pair were detected. R1 and the left lane line back overlap rate WR L1 Read it out.

[0052] The calculation unit 332 calculates the rear overlap rate (left lane line rear overlap rate WR) based on the lane line corresponding to the lane line detected from the image generated at the second time, at the first time. L1 ) and the rear overlap rate (left lane marking rear overlap rate WR) calculated based on the lane markings detected from the image generated at the second time point. L2 Find the difference after subtracting ( ).

[0053] The calculation unit 332 calculates the rear overlap rate (right lane marking rear overlap rate WR) based on the lane markings that are not detected in the image generated at the second time, at the first time. R1 By adding the above difference to ), the right lane line back overlap ratio WR R2 Calculate.

[0054] Figure 6 is a flowchart of the object detection process. The ECU 3 repeatedly performs the object detection process at predetermined time intervals (for example, every 1 / 10 second) while the vehicle 1 is in motion.

[0055] First, the detection unit 331 detects from each of the multiple images received from the camera 2 a pair of lane markings that demarcate the left and right ends of the lane in which vehicle 1 is traveling, and a rear view area of ​​another vehicle that represents the rear area of ​​another vehicle traveling in front of vehicle 1 (step S1).

[0056] The calculation unit 332 determines whether both lane markings of a pair are detected in the image generated at one time from among the multiple images (step S2).

[0057] If both lane markings of a pair are detected (step S2:Y), the calculation unit 332 uses the position of the detected right lane marking and the position of the rear area of ​​the other vehicle to calculate the rear overlap ratio of the right marking and the rear overlap ratio of the left marking (step S3).

[0058] The calculation unit 332 stores the calculated right partition line back overlap rate and left partition line back overlap rate in the memory 32 (step S4), and terminates the object detection process.

[0059] If one of the pair of lane markings is not detected (step S2:N), the calculation unit 332 calculates the right lane marking rear overlap rate and the left lane marking rear overlap rate based on the position of the detected lane marking, the right lane marking rear overlap rate and the left lane marking rear overlap rate calculated based on images generated at past times, and the positions of the lane marking and other vehicle rear areas detected from the image (step S5), and then terminates the object detection process.

[0060] By performing object detection processing in this manner, the ECU3 can appropriately calculate the overlap rate of other vehicles entering its own lane.

[0061] In a modified example, the ECU 3 may further perform driving control processing that controls the steering and acceleration / deceleration of vehicle 1 using the rear overlap rate calculated by the object detection process, for example, such that the larger the calculated rear overlap rate, the longer the distance to other vehicles.

[0062] Those skilled in the art will understand that various changes, substitutions, and modifications can be made to this disclosure without deviating from its spirit and scope. [Explanation of Symbols]

[0063] 1 vehicle 3 ECU 331 Detection unit 332 Calculation Unit

Claims

[Claim 1] A detection unit that detects a pair of lane markings and an area showing the rear of another vehicle traveling in front of the vehicle from each of multiple images representing the environment in front of the vehicle, generated at multiple times, If both of the pair of lane markings are detected in one of the multiple images, the calculation unit calculates the rear overlap rate of the right lane marking and the rear overlap rate of the left lane marking based on the respective positions of the lane marking and the rear area of ​​other vehicles detected in the one image and stores them in the storage unit. If one of the pair of lane markings is not detected in the one image, the calculation unit calculates the rear overlap rate corresponding to the undetected lane marking based on the respective positions of the lane marking and the rear area of ​​other vehicles detected in the one image, and the rear overlap rate of the right lane marking and the rear overlap rate of the left lane marking calculated based on images generated at a time earlier than the time the one image was generated. An object detection device equipped with the following features.

Images