Object detection device and object detection method

Abstract

To provide an object detection device that can determine the highly accurate position of an object to be detected.SOLUTION: The object detection device has an estimation circuit, an accumulation unit, and an extraction unit. The estimation circuit estimates a first position of an object a plurality of times using a first distance to the object measured by one or more distance measuring sensors and a direction of the object estimated from an image of the object acquired by an imaging sensor. The accumulation unit accumulates the first positions of the object estimated a plurality of times at each predetermined distance from the imaging sensor. The extraction unit extracts a second position of the object from the accumulation result.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an object detection device and an object detection method. [Background technology]

[0002] 2. Description of the Related Art An object detection device mounted on a vehicle detects objects in the vicinity of the vehicle, and the detection results are used for driving assistance processing such as avoiding collisions between the vehicle and the objects. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 079252 Summary of the Invention [Problem to be solved by the invention]

[0004] In an object detection device, an object may be detected by using both an imaging sensor that captures an image of the object and a distance measuring sensor that measures the distance to the object. In some cases, it may be difficult for the distance measuring sensor to detect the object using reflected waves, and therefore sensitivity adjustment is performed as in Patent Document 1. However, the sensitivity adjustment disclosed in Patent Document 1 may result in insufficient accuracy in the detection distance.

[0005] The present disclosure provides an object detection device and an object detection method that are capable of determining the position of an object to be detected with high accuracy. [Means for solving the problem]

[0006] The object detection device according to the present disclosure includes an estimation circuit, an accumulation unit, and an extraction unit. The estimation circuit estimates a first position of an object multiple times using a first distance to the object measured by one or more distance measuring sensors and an orientation of the object estimated from an image of the object acquired by an image sensor. The accumulation unit accumulates the first position of the object estimated multiple times for each predetermined distance from the image sensor. The extraction unit extracts a second position of the object from the accumulation result. Effect of the Invention

[0007] According to the object detection device and object detection method disclosed herein, the position of an object to be detected can be determined with high accuracy. [Brief description of the drawings]

[0008] [Figure 1] 1 is a side view showing a schematic configuration of a vehicle on which an object detection device according to an embodiment is mounted. [Diagram 2] 1 is a plan view showing a schematic configuration of a vehicle on which an object detection device according to an embodiment is mounted. [Diagram 3] 1 is a block diagram showing a hardware configuration of a vehicle equipped with an object detection device according to an embodiment. [Figure 4] 1 is a block diagram showing a functional configuration of a vehicle equipped with an object detection device according to an embodiment. [Diagram 5] FIG. 4 is a diagram showing an estimated orientation in the embodiment. [Figure 6] 11A and 11B are diagrams illustrating setting of a plurality of regions on a straight line in the embodiment. [Figure 7] FIG. 6 is a diagram showing the distribution of reflection intensity on a straight line in the embodiment. [Figure 8] 5A and 5B are diagrams illustrating differences in reflection characteristics between an object and a road surface in the embodiment. [Figure 9] 4 is a diagram showing reflection points of an object and a road surface in the embodiment. [Figure 10] FIG. 11 is a diagram showing the distribution of cumulative reflection intensity on a straight line in the embodiment. [Figure 11] 4 is a flowchart showing the operation of the object detection device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of an object detection device according to the present disclosure will be described with reference to the drawings.

[0010] (Embodiment) The object detection device according to the embodiment is mounted on a vehicle to detect objects in the vicinity of the vehicle, and the detection results are used for driving assistance processing such as collision avoidance with the object in the vehicle, with the device being devised to determine the position of the detected object with high accuracy.

[0011] For example, in an object detection device based on existing technology, the larger the reflective surface of an object, the greater (stronger) the reflection intensity of the object, and the object can be detected from a distance using a distance measurement sensor. However, the smaller the reflective surface of an object, the weaker the reflection intensity is, so that accurate detection may be difficult depending on the distance from the object to the distance measurement sensor. For this reason, the object detection device adjusts the sensitivity by lowering the reflection threshold of the distance measurement sensor, making it possible to detect objects with low reflection intensity and ensuring the detection range. However, as the threshold is lowered, the distance measurement sensor also detects undetectable objects such as the road surface.

[0012] Therefore, in Patent Document 1, a method is adopted in which the threshold is lowered only in the vicinity of the distance of the object detected by the imaging sensor. However, if the sensitivity adjustment is not set to the correct range using the detection result due to the influence of the type of imaging sensor and the background of the detection target, the non-detected object is erroneously detected. For this reason, the object detection device of the existing technology may detect the detection target at an inaccurate position. Therefore, in the present disclosure, the threshold is set low for the entire detection area, and the detection target is distinguished from the non-detected object such as the road surface, thereby improving the accuracy of position detection even for detection targets with low reflection intensity.

[0013] The object detection device 2 may be mounted on a vehicle 1 as shown in Figures 1 and 2. In the following, the traveling direction of the vehicle 1 is defined as the positive direction of the X axis, the direction from the right side to the left side across the width of the vehicle is defined as the positive direction of the Y axis, the direction perpendicular to the X and Y directions is defined as the Z direction, and the direction toward the roof of the vehicle 1 is defined as the positive direction of the Z axis.

[0014] The vehicle 1 includes a vehicle body 8, a plurality of wheels 9f, 9r, a plurality of image sensors 11_F, 11_R, a plurality of distance measuring sensors 12_F1 to 12_F4, 12_R1 to 12_R4, an ECU (Electronic Control Unit) 14, and an HMI (Human-Machine Interface) 7.

[0015] The wheels 9f, 9r are attached to the vehicle body 8 along the X-axis direction. The wheel 9f, which is a front wheel, is attached to the front part of the vehicle body 8, and the wheel 9r, which is a rear wheel, is attached to the rear part of the vehicle body 8. In addition, in the X-axis direction, the +X direction is also referred to as the front, and the -X direction is also referred to as the rear.

[0016] The vehicle body 8 has a substantially rectangular shape in the XY plane view, and has, in the X-axis direction, a front end portion 8f near the front bumper at the front portion and a rear end portion 8r near the rear bumper at the rear portion.

[0017] An imaging sensor 11_F and multiple distance measurement sensors 12_F1-12_F4 are attached to the front end 8f. The imaging sensor 11_F is disposed near the center of the vehicle width at the front end 8f. The multiple distance measurement sensors 12_F1-12_F4 are arranged in the Y-axis direction at the front end 8f. The distance measurement sensors 12_F1-12_F4 are spaced apart from each other in the Y-axis direction and are generally spaced apart from the imaging sensor 11_F in the Y-axis direction. Note that the imaging sensor 11_F and the multiple distance measurement sensors 12_F1-12_F4 do not have to be disposed at the same position on the X-axis.

[0018] Figures 1 and 2 show an example of a configuration in which four distance measuring sensors 12_F1 to 12_F4 are arranged at the front end 8f, but the number of distance measuring sensors 12 at the front end 8f may be three or less, or five or more, and it is desirable for them to be arranged symmetrically with respect to the X-axis passing through the center of the vehicle width.

[0019] The imaging sensor 11_F is connected to the ECU 14 via a communication line 41f. Any sensor capable of acquiring an image may be used as the imaging sensor 11_F. The imaging sensor 11_F may be, for example, a CMOS image sensor or a CCD image sensor. The imaging sensor 11_F is capable of acquiring an image of the periphery of the vehicle body 8 and is capable of supplying image data corresponding to the acquired image to the ECU 14.

[0020] The distance measuring sensors 12_F1 to 12_F4 are each connected to the ECU 14 via a communication line 42f. Any sensor capable of measuring distance may be used as the distance measuring sensors 12_F1 to 12_F4. The distance measuring sensors 12_F1 to 12_F4 may be a sonar (ultrasonic sensor), a radar, or a lidar. The distance measuring sensors 12_F1 to 12_F4 are capable of acquiring a signal indicating the distance to an object in the vicinity of the vehicle body 8.

[0021] The distance measuring sensors 12_F1 to 12_F4 transmit waves at a predetermined interval and receive reflected waves at a timing different from the timing of the transmission. The distance measuring sensors 12_F1 to 12_F4 may generate a signal indicating the distance L to an object according to the speed v of the measurement medium (e.g., sound waves in the case of sonar, radio waves in the case of radar, and laser light in the case of lidar) and the time (Time of Flight) T from the timing of transmission to the timing of reception.

[0022] In addition, each of the distance measuring sensors 12_F1 to 12_F4 may be configured to alternate between wave transmission timing and wave reception timing, for example. For example, the following operation may be periodically repeated: distance measuring sensor 12_F1 transmits a wave, distance measuring sensor 12_F4 receives a wave, distance measuring sensor 12_F2 transmits a wave, distance measuring sensor 12_F1 receives a wave, distance measuring sensor 12_F3 transmits a wave, distance measuring sensor 12_F2 receives a wave, distance measuring sensor 12_F4 transmits a wave, distance measuring sensor 12_F3 receives a wave, distance measuring sensor 12_F1 transmits a wave, distance measuring sensor 12_F4 receives a wave, distance measuring sensor 12_F1 transmits a wave, distance measuring sensor 12_F4 receives a wave.

[0023] The distance measuring sensors 12_F1 to 12_F4 are capable of supplying the acquired signals to the ECU .

[0024] The ECU 14 is connected to the HMI 7. The ECU 14 includes an object detection device 2. Using the object detection device 2, the ECU 14 can detect an object near the front of the vehicle body 8 based on an image acquired by the image sensor 11_F and a distance measured by the distance measuring sensors 12_F1 to 12_F4. The ECU 14 can control the vehicle 1 based on a detection result of the object detection device 2, and can execute driving assistance processing such as avoiding a collision with an object.

[0025] An imaging sensor 11_R and multiple distance measurement sensors 12_R1-12_R4 are attached to the rear end 8r. The imaging sensor 11_R is disposed near the center of the vehicle width at the rear end 8r. The multiple distance measurement sensors 12_R1-12_R4 are arranged in the Y-axis direction at the front end 8r. The distance measurement sensors 12_R1-12_R4 are spaced apart from each other in the Y-axis direction and are generally spaced apart from the imaging sensor 11_R in the Y-axis direction. Note that the imaging sensor 11_R and the multiple distance measurement sensors 12_R1-12_R4 do not have to be disposed at the same position on the X-axis.

[0026] 1 and 2 show an example of a configuration in which four distance measuring sensors 12_R1 to 12_R4 are arranged at the rear end 8r, but the number of distance measuring sensors 12 arranged at the rear end 8r may be three or less, or five or more. It is preferable that the multiple distance measuring sensors 12 are arranged symmetrically with respect to the X-axis passing through the center of the vehicle width.

[0027] The imaging sensor 11_R is connected to the ECU 14 via a communication line 41r. Any sensor capable of acquiring an image may be used as the imaging sensor 11_R. The imaging sensor 11_R may be, for example, a CMOS image sensor or a CCD image sensor. The imaging sensor 11_R is capable of acquiring an image of the periphery of the vehicle body 8 and is capable of supplying image data corresponding to the acquired image to the ECU 14.

[0028] The distance measuring sensors 12_R1 to 12_R4 are each connected to the ECU 14 via a communication line 42r. Any sensor capable of measuring distance may be used as the distance measuring sensors 12_R1 to 12_R4. The distance measuring sensors 12_R1 to 12_R4 may be sonar, radar, or LiDAR. The distance measuring sensors 12_R1 to 12_R4 are capable of acquiring a signal indicating the distance to an object in the vicinity of the vehicle body 8.

[0029] Distance measuring sensors 12_R1 to 12_R4 may generate a signal indicating the distance L to an object depending on the speed v of the measurement medium (e.g., sound waves in the case of sonar, radio waves in the case of radar, or laser light in the case of lidar) and the time of flight T from the time of transmission to the time of reception.

[0030] The distance measuring sensors 12_R1 to 12_R4 may be configured to alternate between wave transmission timing and wave reception timing, for example. For example, the following operation may be periodically repeated: the distance measuring sensor 12_R1 transmits a wave, the distance measuring sensor 12_R4 receives a wave, the distance measuring sensor 12_R2 transmits a wave, the distance measuring sensor 12_R1 receives a wave, the distance measuring sensor 12_R3 transmits a wave, the distance measuring sensor 12_R2 receives a wave, the distance measuring sensor 12_R4 transmits a wave, the distance measuring sensor 12_R3 receives a wave, the distance measuring sensor 12_R1 transmits a wave, and the distance measuring sensor 12_R4 receives a wave.

[0031] The distance measuring sensors 12_R1 to 12_R4 are capable of supplying the acquired signals to the ECU .

[0032] The ECU 14 includes an object detection device 2. Using the object detection device 2, the ECU 14 can detect an object near the rear of the vehicle body 8 based on the image acquired by the image sensor 11_R and the distance measured by the distance measuring sensors 12_R1 to 12_R4. The ECU 14 can control the vehicle 1 and execute a predetermined driving assistance process based on the detection result of the object detection device 2.

[0033] For simplicity, the following description will be mainly given using the image sensor 11_R and distance measurement sensors 12_R1-12_R4 attached to the rear end 8r. The image sensor 11_R, distance measurement sensors 12_R1-12_R4, and communication lines 41r, 42r will be simply referred to as the image sensor 11, distance measurement sensors 12_1-12_4, and communication lines 41, 42. The following description can be similarly applied to the image sensors 11_F and 12_F1-12_F4 attached to the front end 8f.

[0034] Next, the hardware configuration of the vehicle 1 will be described with reference to FIG.

[0035] 1 , the vehicle 1 has an interface (I / F) 13, a vehicle control device 6, and a bus 13. The interface 13, the ECU 14, the HMI 7, and the vehicle control device 6 are communicatively connected to each other via the bus 13. The ECU 14 includes, in addition to the object detection device 2, a volatile storage unit 15 and a non-volatile storage unit 16.

[0036] The interface 13 is connected to the imaging sensor 11 via a communication line 41, and is connected to the plurality of distance measuring sensors 12_1 to 12_4 via a communication line .

[0037] The volatile storage unit 15 may be any device capable of temporarily storing information, such as a dynamic random access memory (DRAM). The non-volatile storage unit 16 may be a device or apparatus capable of non-volatilely storing information, such as a flash memory or a hard disk. The non-volatile storage unit 16 may store a program PG for the driving assistance process.

[0038] In response to detection of an object by the object detection device 2 near the rear of the vehicle body 8 (negative direction of the X-axis), the ECU 14 controls the vehicle control device 6 to stop the vehicle 1 and / or causes the HMI 7 to issue warning information corresponding to the presence of the object.

[0039] The vehicle control device 6 drives and controls the wheels 9f, 9r. The vehicle control device 6 can rotate the wheels 9f, 9r, stop the rotation of the wheels 9f, 9r, and steer the wheels 9f, 9r. Although not shown, the vehicle control device 6 may include, for example, a vehicle speed sensor, an accelerator sensor, a brake sensor, a brake actuator, and an engine controller (or a motor controller).

[0040] The vehicle control device 6 may control the vehicle 1 to avoid the object detected by the object detection device 2 by controlling a brake actuator and / or an engine controller (or a motor controller) in response to a control signal from the ECU 14.

[0041] The HMI 7 may notify information indicating a warning by visual and / or auditory means in response to a control signal from the ECU 14. The HMI 7 may generate a display object indicating a warning and display it on a screen in the vehicle cabin, or generate an audio signal of a warning sound or warning message and output it from a speaker in the vehicle cabin.

[0042] Next, the functional configuration of the vehicle 1 will be described with reference to FIG.

[0043] The vehicle 1 includes an image sensor 11, a plurality of distance measuring sensors 12_1 to 12_4, an object detection device 2, a drive control unit 16, and a notification unit 17.

[0044] The drive control unit 16 may be realized by the vehicle control device 6 (see FIG. 3). The notification unit 17 may be realized by the HMI 7.

[0045] The object detection device 2 includes a plurality of functions. All of the functions included in the object detection device 2 may be realized in hardware (for example, as a circuit) in the ECU 14. All of the functions included in the object detection device 2 may be realized in software in the ECU 14. Some of the functions included in the object detection device 2 may be realized in hardware and the remaining part may be realized in software. The functions realized in software may be deployed as functional modules in the volatile storage unit 15 collectively at the time of compilation or sequentially as the processing progresses by the ECU 14 executing the program PG. This allows the functions to be equivalently considered to be realized in software in the ECU 14.

[0046] The object detection device 2 may include a detection processing unit 3, a determination unit 4, and a control unit 5, as shown in FIG.

[0047] The detection processing unit 3 is disposed between the image sensor 11, the multiple distance measuring sensors 12_1 to 12_4, and the determination unit 4. The determination unit 4 is disposed between the detection processing unit 3 and the control unit 5. The control unit 5 is disposed between the determination unit 4, the drive control unit 16, and the notification unit 17.

[0048] The detection processing unit 3 virtually sets a coordinate space on the road surface RD around the vehicle body 8. The detection processing unit 3 may set the coordinate space with the position of the imaging sensor 11 as the origin. As information obtained from the image acquired by the imaging sensor 11, the accuracy of the orientation of the subject is relatively high, but the accuracy of the distance to the subject is relatively low. As information obtained from the signals of the distance measuring sensors 12_1 to 12_4, the accuracy of the orientation of the measurement target is relatively low, but the accuracy of the distance to the measurement target is relatively high. The detection processing unit 3 uses a combination of the image acquired by the imaging sensor 11 and the signals of the distance measuring sensors 12_1 to 12_4. The detection processing unit 3 detects the presence or absence of an object and its coordinate position in the coordinate space on the road surface RD according to the image acquired by the imaging sensor 11 and the distance measured by the distance measuring sensors 12_1 to 12_4.

[0049] For example, the detection processing unit 3 may have a distance calculation unit 31, an object detection unit 32, an estimation unit 33, an accumulation unit 34, and an extraction unit 35. The distance calculation unit 31 is disposed between the multiple distance measurement sensors 12_1 to 12_4 and the estimation unit 33. The object detection unit 32 is disposed between the image sensor 11 and the estimation unit 33. The estimation unit 33 is disposed between the distance calculation unit 31, the object detection unit 32, and the accumulation unit 34. The accumulation unit 34 is disposed between the estimation unit 33 and the extraction unit 35.

[0050] The distance calculation unit 31 receives signals from the distance measurement sensors 12_1 to 12_4. The distance calculation unit 31 extracts the reflection intensity and the distance from the received signals. The distance calculation unit 31 generates distance measurement information in which the reflection intensity and the distance (Ds) are associated with each other. The distance calculation unit 31 supplies the distance measurement information to the estimation unit 33.

[0051] The object detection unit 32 receives an image captured by the imaging sensor 11. The object detection unit 32 detects an object contained in the image by extracting features from the image and comparing the feature extraction result with features learned in advance. The object detection unit 32 may identify the pixel position of the object in the image. The detection result by the object detection unit 32 includes the pixel position of the object in the image and the type of the object. The type of object includes, for example, a pedestrian, a bicycle, etc. The object detection unit 32 supplies the detection result to the estimation unit 33.

[0052] The object detection unit 32 may supply the detection result to the distance calculation unit 31, and the distance calculation unit 31 that has received the detection result may lower the threshold value of the reflection intensity and extract the reflection intensity and distance from the received signal when the detection result indicates the presence of a pedestrian, bicycle, etc., or may extract the reflection intensity and distance from the received signal without lowering the threshold value of the reflection intensity when the detection result indicates the absence of a pedestrian, bicycle, etc. This makes it possible to reduce the load of the extraction process in the distance calculation unit 31 depending on the detection result.

[0053] The estimation unit 33 receives distance measurement information from the distance calculation unit 31 and receives the detection result from the object detection unit 32. The estimation unit 33 estimates the orientation θc of the object using the image acquired by the imaging sensor 11. The estimation unit 33 extracts information on the pixel position of the object in the image from the detection result of the object detection unit 32. The estimation unit 33 may estimate the orientation θc of the object without the camera detection distance Dc depending on the pixel position of the object in the image. The estimation unit 33 may further estimate the position of the object (orientation θc, camera detection distance Dc) using the image.

[0054] For example, the estimation unit 33 calculates the object position from the road surface estimated from the mounting height and mounting angle of the imaging sensor 11 (camera), lens characteristics, etc., and the position where the object detected by the object detection unit 32 contacts the road surface.

[0055] The estimation unit 33 calculates an object orientation θc and a camera detection distance Dc based on the imaging sensor 11 from the position Pc of the object OB detected from the image shown in FIG. 5. The estimation unit 33 does not need to calculate the camera detection distance Dc from the object OB detected from the image. The estimation unit 33 also regards the sonar detection distance Ds measured by the distance calculation unit 31 as a reflection distance from the object OB, calculates a fusion distance Df on the orientation θc of the object OB obtained from the image, and supplies the estimation result to the accumulation unit 34. The estimation result includes information on the orientation of the object OB and information on the distance of the object OB. The measurement by the imaging sensor 11 and the distance measurement sensor 12 may be repeated periodically. The distance measurement sensor 12 may also measure multiple sonar detection distances Ds in one measurement. For example, when measurements are repeated k times (k is an integer equal to or greater than 1) and n distances are detected for each measurement, the information (θc(k)) relating to the azimuth of the object OB, the fusion distance Df(k), and the sonar detection distance (Ds(k, n)) of the object OB, which are the estimation results supplied to the accumulator 34, are k and k×n, respectively. Note that the sonar detection distance Ds(k, n) may have a different value of n for each measurement, and may be a different distance for each of k and n. Note that FIG. 5 shows that the distance measuring sensor 12_4 detects one sonar detection distance Ds(1, 1) in one measurement.

[0056] The accumulation unit 34 receives the estimation result from the estimation unit 33. The accumulation unit 34 sets a plurality of coordinate areas on the road surface RD according to a combination of the image acquired by the imaging sensor 11 and the distances measured by the distance measuring sensors 12_1 to 12_4. The accumulation unit 34 may set a plurality of coordinate areas on a straight line OR corresponding to the orientation estimated by the estimation unit 33. The accumulation unit 34 may set each of the plurality of coordinate areas on a straight line with a width according to the object.

[0057] The accumulation unit 34 refers to the information on the type of object included in the detection result by the object detection unit 32, and specifies (determines) a width according to the type of object. If the object is a pedestrian, it specifies the width W that is expected for the pedestrian. The accumulation unit 34 may set a coordinate area CR3 of width W including a position corresponding to a distance Dc from the imaging sensor 11 on a straight line OR (see FIG. 5) corresponding to the orientation, as shown in FIG. 6. The accumulation unit 34 may set coordinate areas CR2 and CR1 in units of width W in the coordinate area CR3 toward the imaging sensor 11, and may set coordinate areas CR4, CR5, and CR6 in units of width W away from the imaging sensor 11. The accumulation unit 34 may also set a coordinate area of ​​width W according to the distribution state of the fusion distance Df.

[0058] For example, if the reflection intensity indicated by the signal of the distance measuring sensor 12 used to calculate the fusion distance Df is plotted on the vertical axis representing the reflection intensity and the distance from the image sensor 11 on the line OR on the horizontal axis, a distribution of reflection intensity as shown in Fig. 7 is obtained. In Fig. 7, the position corresponding to the distance Dc to the object OB identified from the image of the image sensor 11 is indicated by a white triangle. In the distribution of reflection intensity shown in Fig. 7, the reflection intensity from the object OB and the reflection intensity from the road surface RD are mixed and have similar intensity levels. For this reason, in the state shown in Fig. 7, it is difficult for the object detection device to determine the object OB based on the reflection intensity.

[0059] The accumulator 34 can set the threshold value Sth1 at a level slightly higher than the average magnitude of the reflection intensity from the road surface RD. As a result, for an object with a relatively high reflection intensity such as a wall, the accumulator 34 can distinguish a reflection intensity (not shown) that exceeds the threshold value Sth1 from among the multiple reflection intensities as the reflection intensity of the object from the reflection intensity of the road surface RD.

[0060] However, as shown in FIG. 7, objects such as pedestrians, whose reflection intensity is relatively smaller than that of walls, are buried below the threshold value Sth1, and the reflection intensity of the road surface RD is also mixed in, making it difficult for the accumulation unit 34 to distinguish the reflection intensity of the object from the reflection intensity of the road surface RD.

[0061] Here, in (a) and (b) of FIG. 8, the identified direction θc of the pedestrian is indicated by a solid straight line OR as information obtained from an image captured by the image sensor 11.

[0062] As information obtained from the signals of the distance measuring sensors 12_1 to 12_4, in (a) and (b) of Fig. 8, the distance measured by the distance measuring sensor 12_3 to the pedestrian is indicated by a dashed circle, and the distance measured by the distance measuring sensor 12_4 is indicated by a dotted circle. In (a) and (b) of Fig. 8, the distance obtained by the distance measuring sensor 12 roughly coincides with the position of the pedestrian.

[0063] As shown in Figures 9(a) and (b), the reflection points of an object (e.g., a pedestrian) for the transmission wave of the distance measuring sensor 12 are somewhat densely located on the XY plane, although there is some variation in the Z-axis direction. Note that, as shown in Figures 8(a) and (b), the densely located positions on the XY plane correspond to the object and exist near the line OR. For this reason, the distances of the object (e.g., a pedestrian) measured by multiple distance measuring sensors 12 tend to be densely located in a specific area on the line OR, as shown by the intersections of the line OR with the dashed-dotted circle and the dotted circle in Figure 8(a).

[0064] On the other hand, the reflection points of the road surface RD vary in position on the XY plane, as shown by the white triangles in Fig. 8(b) and the black circles in Fig. 9(c). Unlike objects (e.g., pedestrians), the varying positions on the XY plane are not located near the line OR. Therefore, the distances to the road surface RD measured by the multiple distance measuring sensors 12 tend to vary on the line OR, as shown by the intersections of the line OR with the dashed and dotted circle and the dotted circle in Fig. 8(b).

[0065] Comparing Figure 8(a) and Figure 8(b), it is expected that by focusing on the difference in the degree of variation between the object and the road surface RD, for example, the variation in the calculated fusion distance, it will be possible to distinguish between the reflection intensity of the object and the reflection intensity of the road surface RD from multiple reflection intensities.

[0066] Therefore, the accumulation unit 34 lowers the threshold value for the reflection intensity so as to pick up the reflection intensities of the object and the road surface RD. The accumulation unit 34 accumulates the reflection intensities indicated by the signal of the distance measuring sensor 12 in each of a plurality of coordinate regions set on the straight line OR.

[0067] For example, as shown in Fig. 7, the accumulator 34 lowers the threshold level from Sth1 to Sth2. The threshold level Sth2 is a level that is slightly lower than the average magnitude of the reflection intensity from the road surface RD. In the case of Fig. 7, an example is shown in which, as reflection intensities exceeding the threshold level Sth2, one reflection intensity is distributed in the coordinate region CR2, four reflection intensities are distributed in the coordinate region CR4, and two reflection intensities are distributed in the coordinate region CR6.

[0068] The accumulator 34 accumulates the reflection intensities exceeding the threshold value Sth2 among the multiple reflection intensities in each of the multiple coordinate regions CR1 to CR6 on the straight line OR. In the case of FIG. 7, the accumulator 34 accumulates one reflection intensity in the coordinate region CR2, accumulates four reflection intensities in the coordinate region CR4, and accumulates two reflection intensities in the coordinate region CR6. In this way, the accumulator 34 can obtain an accumulation result as shown in FIG. 10. In FIG. 10, the distribution of the accumulated reflection intensities for each of the coordinate regions CR1 to CR6 is shown in a histogram. In the above description, the coordinate regions CR1 to CR6 are set based on the distance Dc obtained from the image sensor 11, but the coordinate region CR may be set in units of width W based on the fusion distance Df at which the reflection intensity on the straight line OR is the highest.

[0069] The accumulation unit 34 supplies the accumulation result to the extraction unit 35 .

[0070] The extraction unit 35 receives the accumulation result from the accumulation unit 34. The extraction unit 35 extracts the coordinate position of the object from the accumulation result of the accumulation unit 34. The extraction unit 35 extracts, as the coordinate position of the object, a coordinate position corresponding to a coordinate area in which the accumulated value of the reflection intensity indicated by the signal of the distance measuring sensor 12 exceeds a threshold value among a plurality of coordinate areas set on the straight line OR. At this time, the extraction unit 35 may set the threshold value for the accumulated reflection intensity at a level according to the object. Alternatively, the threshold value may be set to a value higher than the accumulated reflection intensity of an object such as a road surface that is not to be detected. In this way, the extraction unit 35 can extract the coordinate position of the object using a threshold value at an appropriate level according to the object to be detected.

[0071] For example, the extraction unit 35 refers to information on the type of object included in the detection result by the object detection unit 32 and specifies a level according to the type of object. If the object is a pedestrian, it specifies the cumulative reflection intensity expected for the pedestrian, and sets the level of the threshold CSth for the cumulative reflection intensity at a level lower than the specified level and higher than the expected cumulative reflection intensity of a non-detected object such as a road surface, as shown in FIG. 10. The extraction unit 35 compares the cumulative reflection intensity of each of the coordinate regions CR1 to CR6 with the threshold CSth. The extraction unit 35 can determine, as the coordinate position of the object OB, the coordinate position corresponding to the distance Dt of the coordinate region CR4, of the multiple coordinate regions CR1 to CR6, where the cumulative reflection intensity exceeds the threshold CSth.

[0072] The extraction unit 35 may check the reliability of the extracted coordinate position based on whether the extracted coordinate position falls within a predetermined position range including a position estimated from the image of the image sensor 11. If the extracted coordinate position falls within the predetermined position range, the extraction unit 35 may output the extracted coordinate position to the determination unit 4 as being "probable" as the coordinate position of an object, and if the extracted coordinate position does not fall within the predetermined position range, the extraction unit 35 may discard the extracted coordinate position as being "unprobable" as the coordinate position of an object.

[0073] For example, the extraction unit 35 sets a position range ΔP of a predetermined length (for example, a length of 1.5×W in both directions) before and after the coordinate position corresponding to the distance Dc shown by the dotted triangle in Fig. 10 as the predetermined position range. In the case of Fig. 10, the coordinate position corresponding to the extracted distance Dt falls within the predetermined position range ΔP. Therefore, the extraction unit 35 can output the extracted coordinate position to the determination unit 4 as being likely to be the coordinate position of the object OB.

[0074] In the above description, the distance Dc of the coordinate region CR has been used for explanation. However, in order to increase the distance resolution, one of the fusion distances Df constituting the cumulative reflection intensity of the coordinate region CR may be selected and processed.

[0075] 4 supplies the detection result to the determination unit 4. The detection result includes information on the presence or absence of an object and its coordinate position.

[0076] A position range in the coordinate space on the road surface where a collision is highly likely to occur is preset as a collision judgment range in the judgment unit 4. When the judgment unit 4 receives a detection result from the detection processing unit 3, it judges whether or not the coordinate position of the object is within the collision judgment range according to the detection result. The judgment unit 4 supplies the judgment result to the control unit 5.

[0077] The control unit 5 can execute driving assistance processing according to the judgment result. If the coordinate position of the object is outside the collision judgment range, the control unit 5 generates attention-calling information and supplies it to the notification unit 17. As a result, the notification unit 17 notifies the user of the attention-calling information.

[0078] If the coordinate position of the object is within the collision determination range, the control unit 5 generates control information including a request to activate the brakes and supplies it to the drive control unit 16. As a result, the drive control unit 16 activates the brakes to stop the vehicle 1. At the same time, the control unit 5 generates warning information and supplies it to the notification unit 17. As a result, the notification unit 17 notifies the user of the warning information.

[0079] Next, the operation of the object detection device 2 will be described with reference to FIG.

[0080] In the detection processing unit 3 of the object detection device 2, the distance calculation unit 31 acquires distance measurement information from the distance measurement sensors 12_1 to 12_4 (S1). The distance calculation unit 31 extracts the reflection intensity and the distance from the received signal. The distance calculation unit 31 generates distance measurement information in which the reflection intensity and the distance are associated with each other. The distance calculation unit 31 supplies the distance measurement information to the estimation unit 33.

[0081] The object detection unit 32 acquires image data from the imaging sensor 11 and executes a detection process from the acquired image (S2). The object detection unit 32 detects an object contained in the image by extracting features from the image and comparing the feature extraction results with features learned in advance. The object detection unit 32 may identify the pixel position of the object in the image. The detection result by the object detection unit 32 includes the pixel position of the object in the image and the type of object. The type of object includes, for example, a pedestrian, a bicycle, etc. The object detection unit 32 supplies the detection result to the estimation unit 33.

[0082] The estimation unit 33 determines whether or not the detection result of the object detection unit 32 is an object to be detected, such as a person (S3). The estimation unit 33 may refer to information on the type of object included in the detection result, and determine whether or not the detection result is an object to be detected, depending on the information on the type of object.

[0083] If the detection result is not an object to be detected (No in S3), the estimation unit 33 returns the process to S1. In this case, since the reflection intensity is sufficiently large, the position estimation by the imaging sensor 11 may be omitted, and the position estimation may be performed by the multiple distance measuring sensors 12.

[0084] If the detection result is an object to be detected (Yes in S3), the accumulation unit 34 lowers the threshold for reflection intensity to pick up the reflection intensity of the object and the road surface RD (S4). The accumulation unit 34 lowers the threshold level from Sth1 to Sth2 as shown in Fig. 7. The level of the threshold Sth2 is set to a level at which the reflection intensity of the object to be detected, including the reflection from the road surface RD, exceeds the threshold. Note that if the detection result by the estimation unit 33 is an object to be detected, the distance may be measured again after lowering the threshold of the distance measuring sensor 12.

[0085] The estimation unit 33 estimates the orientation θc of the object using the image acquired by the imaging sensor 11. The estimation unit 33 extracts information on the pixel position of the object in the image from the detection result of the object detection unit 32. The estimation unit 33 also regards the object distance Ds measured by the distance calculation unit 31 as the reflection distance from the object OB, and calculates a fusion distance Df on the orientation θc of the object OB obtained from the image.

[0086] The accumulation unit 34 receives the calculated estimation result from the estimation unit 33. The accumulation unit 34 sets a plurality of coordinate areas on the road surface RD according to a combination of the image acquired by the imaging sensor 11 and the distance Ds measured by the distance measuring sensors 12_1 to 12_4 (S5). The accumulation unit 34 may set a plurality of coordinate areas on a straight line OR corresponding to the direction estimated by the estimation unit 33. The accumulation unit 34 may set each of the plurality of coordinate areas on a straight line with a width according to the object.

[0087] The accumulator 34 accumulates the reflection intensities in each of the multiple coordinate regions to obtain a distance vs. accumulated reflection intensity distribution (S6). The accumulator 34 accumulates the reflection intensities that exceed a threshold Sth2 among the multiple reflection intensities in each of the multiple coordinate regions CR1 to CR6 on the straight line OR. The accumulator 34 supplies the accumulation result to the extractor 35.

[0088] The extraction unit 35 extracts the coordinate position corresponding to the distance where the accumulated reflection intensity exceeds the threshold as the coordinate position of the detection target (S7). The extraction unit 35 receives the accumulation result from the accumulation unit 34. The extraction unit 35 sets the threshold for the accumulated reflection intensity at a level according to the object. The extraction unit 35 extracts the coordinate position corresponding to the coordinate area where the accumulated value of the reflection intensity indicated by the signal from the distance measuring sensor 12 exceeds the threshold from among multiple coordinate areas set on the straight line OR as the coordinate position of the object.

[0089] The extraction unit 35 may compare the extracted detection target with the detection coordinates of the image sensor 11 to determine whether or not the detection target is within a valid range.

[0090] The extraction unit 35 checks the accuracy of the extracted coordinate position based on whether the extracted coordinate position falls within a predetermined position range including a position estimated from the image of the image sensor 11. If the extracted coordinate position falls within the predetermined position range, the extraction unit 35 outputs the extracted coordinate position to the determination unit 4 as a "likely" coordinate position of an object, and if the extracted coordinate position does not fall within the predetermined position range, the extraction unit 35 may discard the extracted coordinate position as "unlikely" as a coordinate position of an object. Note that the extraction unit 35 may use the detection distance of the distance measurement sensor 12 instead of the detection coordinate of the image sensor 11.

[0091] As described above, in the embodiment, in the object detection device 2, the signal of the distance measurement sensor 12 is accumulated in each of a plurality of coordinate regions set on the road surface RD according to a combination of the image acquired by the imaging sensor 11 and the distance measured by the distance measurement sensor 12, and the coordinate position of the object is extracted from the accumulated result. This makes it possible to distinguish and detect objects with relatively low reflection intensity from the road surface while lowering the threshold of the distance measurement sensor 12 to increase its sensitivity, thereby expanding the range of reflection intensity detectable by the combination of the imaging sensor 11 and the distance measurement sensor 12, and improving the accuracy of position detection even for detection objects with low reflection intensity.

[0092] In addition, in the object detection device of the above-mentioned embodiment, the notation "··· part" used for each component can be replaced with other notations such as "··· circuitry", "··· assembly", "··· device", "··· unit", or "··· module", as mentioned above.

[0093] In addition, in the object detection device of the above-described embodiment, the ECU 14 executes the program PG installed in the non-volatile storage unit 16, for example, to perform the object detection process.

[0094] However, the program executed by the object detection device may be provided by being recorded in a recording medium readable by the ECU 14, such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD), in the form of an installable or executable file. Alternatively, the program may be downloaded via a network and executed by the ECU 14.

[0095] Furthermore, at least a part of the functions of the object detection device may be realized by a dedicated hardware circuit that does not have a CPU.

[0096] In this way, the object detection device of the above-mentioned embodiment can be realized by software, hardware, or software linked to hardware. The object detection device of the above-mentioned embodiment may also be realized by a system, a device, a method, an integrated circuit, a computer program, or a recording medium, or may be realized by any combination of a system, a device, a method, an integrated circuit, a computer program, and a recording medium. The program product is a computer-readable medium on which a computer program is recorded.

[0097] Furthermore, each functional block of the object detection device of the above-mentioned embodiment may be partially or entirely realized as an LSI, which is an integrated circuit, and each process of the object detection device of the above-mentioned embodiment may be partially or entirely controlled by one LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of one chip so as to include some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, a system LSI, a super LSI, or an ultra LSI.

[0098] However, the method of integration is not limited to LSI, and may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, after LSI manufacture, a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI may be used. Each process of the object detection device of the above-mentioned embodiment may be realized as digital processing or analog processing.

[0099] Furthermore, if a new integrated circuit technology that can replace LSI appears due to the progress of semiconductor technology or a derivative technology, it is possible to integrate the functional blocks using that technology. The application of biotechnology is also a possibility.

[0100] Although some embodiments of the present disclosure have been described, these embodiments are presented as examples and are not intended to limit the scope of the present disclosure. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the scope of the present disclosure. These embodiments and their modifications are included in the scope and gist of the present disclosure, as well as in the scope of the present disclosure and its equivalents as described in the claims. [Explanation of symbols]

[0101] 1 vehicle 2. Object detection device 3. Detection processing section 4 Judgment section 5. Control section 31 Distance calculation unit 32 Object detection unit 33 Estimation part 34 Accumulation Section 35 Extraction part

Claims

1. an estimation circuit that estimates a first position of the object multiple times using a first distance to the object measured by one or more distance sensors and an orientation of the object estimated from images of the object acquired by an imaging sensor; an accumulator that accumulates the first position of the object that is estimated a plurality of times for each predetermined distance from the image sensor; an extraction unit that extracts a second position of the object from the accumulation result; An object detection device comprising:

2. an object detection circuit for detecting a type of the object using the image; the one or more distance measuring sensors are multiple, If the type of the object is not a predetermined object, the estimation circuit The object detection device according to claim 1 , further comprising: a second position of the object being extracted using a plurality of second distances measured by the plurality of distance measuring sensors.

3. an object detection circuit for detecting a type of the object using the image; The predetermined distance from the image sensor is set according to an interval depending on the type of the object. The object detection device according to claim 1 .

4. The extraction unit extracts the first position of the object as the second position of the object when a cumulative value of the first position of the object exceeds a threshold value. The object detection device according to claim 1 .

5. the estimation circuitry further estimates a fourth position of the object using the image; The extraction unit extracts a position within a predetermined range including the fourth position from the accumulation result as the second position of the object. The object detection device according to claim 1 .

6. estimating a first position of the object multiple times using a first distance to the object measured by one or more distance sensors and an orientation of the object estimated from images of the object acquired by an imaging sensor; accumulating the first position of the object estimated multiple times at each predetermined distance from the image sensor; extracting a second position of the object from the accumulation result; Object detection methods.

7. Further, detecting a type of the object using the image; If the type of the object is not a predetermined object, extracting a second position of the object using a plurality of second distances measured by the plurality of distance measuring sensors; The object detection method according to claim 6.

8. Further, detecting a type of the object using the image; the predetermined distance from the image sensor is set according to an interval depending on the type of the object; The object detection method according to claim 6.

9. If the accumulated value of the first position of the object exceeds a threshold, the object is extracted as a second position of the object. The object detection method according to claim 6.

10. further estimating a fourth position of the object using the image; extracting a position within a predetermined range including the fourth position from the accumulation result as the second position of the object; The object detection method according to claim 6.

Images