Obstacle detection device, obstacle detection method, and program

Abstract

To provide an obstacle detection device and method, in which the accuracy of detecting an obstacle is improved.SOLUTION: The obstacle detection device comprises: a plurality of ranging sensors for transmitting an ultrasonic wave and receiving a reflected wave of the ultrasonic wave, which are arranged in a vehicle by being spaced apart in a direction crossing the ultrasonic wave transmission / reception direction; a camera which is mounted to the vehicle and capable of acquiring an image corresponding to the detection region by the plurality of ranging sensors; and a control circuit that detects an obstacle in the surrounding of the vehicle on the basis of the reflected wave received by the plurality of ranging sensors and the image acquired by the camera. The control circuit detects an obstacle on the basis of the image acquired by the camera from the detection region by the other ranging sensors in a prescribed period from the time an ultrasonic wave is transmitted by one of the plurality of ranging sensors to the time a reflected wave is received.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an obstacle detection device, an obstacle detection method, and a program. [Background technology]

[0002] The obstacle detection device is mounted on a vehicle. The obstacle detection device detects objects such as a preceding vehicle, an obstacle, or a pedestrian. The obstacle detection device has a distance measurement sensor. There are known techniques for the obstacle detection device to perform various controls for improving the traveling safety of the vehicle, such as activating an automatic brake or notifying the driver, based on the object detection result by the distance measurement sensor.

[0003] An example of an obstacle detection device based on existing technology is described in Patent Document 1. The technology described in Patent Document 1 includes an ultrasonic sensor that receives reflected waves of a search wave transmitted from a vehicle to measure the distance to an object, and an on-board camera that captures images of the vehicle's surroundings, and when the vehicle is approaching an object, the detection sensitivity of the ultrasonic sensor is changed depending on the position of the vehicle relative to the object. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6703471 Summary of the Invention [Problem to be solved by the invention]

[0005] An obstacle detection device according to the existing technology includes an ultrasonic sensor and an on-board camera. However, for example, there is no disclosure of a method for the ultrasonic sensor to detect an object from the time when it transmits a search wave until it receives a reflected wave of the transmitted search wave.

[0006] Non-limiting embodiments of the present disclosure provide an obstacle detection device, an obstacle detection method, and a program that improve the accuracy of obstacle detection. [Means for solving the problem]

[0007] The obstacle detection device disclosed herein comprises a plurality of ranging sensors that transmit ultrasonic waves, receive reflected ultrasonic waves, and are arranged on a vehicle at intervals in a direction intersecting the transmission and reception direction of the ultrasonic waves, a camera that is arranged on the vehicle and can acquire images corresponding to detection areas detected by the plurality of ranging sensors, and a control circuit that detects obstacles around the vehicle based on the reflected waves received by the plurality of ranging sensors and the images acquired by the camera, and the control circuit detects obstacles in the detection areas detected by the other ranging sensors based on the images acquired by the camera during a predetermined period from when one of the plurality of ranging sensors transmits ultrasonic waves to when it receives the reflected waves. Effect of the Invention

[0008] According to the obstacle detection device, obstacle detection method, and program of the present disclosure, it is possible to improve the accuracy of obstacle detection. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a plan view showing a vehicle equipped with an obstacle detection device according to a first embodiment. [Diagram 2] FIG. 2 is a time chart showing the flow of detection processing by the obstacle detection device using a camera and sonar. [Diagram 3] FIG. 3 is a flowchart illustrating the obstacle detection method. [Figure 4] FIG. 4 is a plan view showing a vehicle equipped with an obstacle detection device according to the second embodiment. [Diagram 5] FIG. 5 is a time chart showing the operation of the camera and sonar by the obstacle detection device. [Figure 6] FIG. 6 is a block diagram showing a control system of the obstacle detection device. [Figure 7] FIG. 7 is a block diagram showing a modified example of the control system of the obstacle detection device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Preferred embodiments of the present disclosure will be described in detail below with reference to the drawings. Note that the present disclosure is not limited to these embodiments, and when there are multiple embodiments, the present disclosure also includes a configuration in which each embodiment is combined. In addition, the components in the embodiments include those that a person skilled in the art can easily imagine, those that are substantially the same, and those that are within the so-called equivalent range.

[0011] [First embodiment] <Obstacle detection device> 1, the obstacle detection device 10 is mounted on a vehicle 100. The obstacle detection device 10 has a plurality of sonars (distance measuring sensors) 11, 12, 13, and 14, a camera 15, and a control circuit 16.

[0012] The sonars 11, 12, 13, and 14 are disposed at intervals in the vehicle width direction at the rear end 101 of the vehicle 100. However, the sonars 11, 12, 13, and 14 may be provided at the front end 102 or the left and right side portions 103 and 104 of the vehicle 100 other than at the rear end 101. Also, in the first embodiment, four sonars 11, 12, 13, and 14 are provided, but the number is not limited to four and it is sufficient that two or more sonars are provided.

[0013] The sonars 11, 12, 13, and 14 each have a transmitting section and a receiving section. The transmitting section and the receiving section may be configured with one microphone, or may be configured with two or more microphones separated into a transmitting section and a receiving section. The transmitting section transmits ultrasonic waves. The receiving section receives the reflected waves of the ultrasonic waves transmitted by the transmitting section. The sonars 11, 12, 13, and 14 transmit ultrasonic waves rearward from the rear end portion 101 of the vehicle 100, and receive the reflected waves that are reflected when the ultrasonic waves collide with an object. Here, the objects and obstacles are other vehicles, obstacles, pedestrians, etc., and objects that do not impede the travel of the vehicle 100, such as unevenness in the road surface, are not included in the obstacles.

[0014] The sonars 11, 12, 13, and 14 are set with detection areas A1, A2, A3, and A4, respectively. The detection areas A1, A2, A3, and A4 are areas where ultrasonic waves are transmitted by the transmitting unit, and are also areas where the receiving unit receives. The detection areas A1, A2, A3, and A4 are elliptical, and some areas overlap. For example, the sonars 11, 12, 13, and 14 are arranged at the rear end 101 of the vehicle 100 with a gap therebetween in a direction intersecting the direction of transmission and reception of ultrasonic waves, for example, in the vehicle width direction, so that some areas in the detection areas A1, A2, A3, and A4 overlap.

[0015] The camera 15 is disposed at the center in the vehicle width direction at the rear end 101 of the vehicle 100. However, the camera 15 may be provided at the front end 102 or left and right side portions 103, 104 other than the rear end 101 of the vehicle 100. Furthermore, a plurality of cameras 15 may be provided at the rear end 101 of the vehicle 100 at intervals in the vehicle width direction.

[0016] The camera 15 is an imaging device and acquires an image of the rear of the vehicle 100. The camera 15 is set with an imaging area A5. The imaging area A5 has a semicircular shape. The camera 15 acquires an image of the imaging area A5 corresponding to the detection areas A1, A2, A3, A4 by the multiple sonars 11, 12, 13, 14. In this case, the detection areas A1, A2, A3, A4 by the sonars 11, 12, 13, 14 and the imaging area A5 by the camera 15 almost overlap, but do not overlap in part. The imaging area A5 by the camera 15 is an area close to the rear end 101 of the vehicle 100, and the detection areas A1, A2, A3, A4 by the sonars 11, 12, 13, 14 are areas slightly far from the rear end 101 of the vehicle 100.

[0017] The control circuit 16 is disposed in the vehicle 100. The control circuit 16 has a sonar control circuit 21, a camera control circuit 22, and a main control circuit 23. In the first embodiment, the sonar control circuit 21, the camera control circuit 22, and the main control circuit 23 are provided as the control circuit 16, but this is for the purpose of easily explaining the functions of each, and the sonar control circuit 21 and the camera control circuit 22 may be one control circuit, or the sonar control circuit 21, the camera control circuit 22, and the main control circuit 23 may be one control circuit.

[0018] The control circuit 16 detects obstacles behind the vehicle based on the detection results of the sonars 11, 12, 13, and 14 and the images captured by the camera 15. In this case, if sonars and cameras are arranged at the front end 102 and left and right side portions 103 and 104 of the vehicle 100 in addition to the rear end portion 101 of the vehicle 100, the control circuit 16 can detect obstacles all around the vehicle 100 based on the detection results of the sonars and the images captured by the cameras.

[0019] The sonar control circuit 21 is connected to the sonars 11, 12, 13, and 14. The transmitting units of the sonars 11, 12, 13, and 14 transmit ultrasonic waves and receive reflected waves reflected from objects around the vehicle 100. The sonar control circuit 21 outputs the timing of transmitting ultrasonic waves to the transmitting units of the sonars 11, 12, 13, and 14. The sonar control circuit 21 receives signals indicating that the sonars 11, 12, 13, and 14 have received reflected waves reflected from objects around the vehicle 100. The sonar control circuit 21 detects objects around the vehicle 100 by measuring the time from transmitting ultrasonic waves to receiving the reflected waves, and calculates the distance to the detected objects.

[0020] The camera 15 is connected to the camera control circuit 22. The camera 15 captures an image of the rear of the vehicle 100. The image data captured by the camera 15 is input to the camera control circuit 22. The image data input from the camera 15 to the camera control circuit 22 is a moving image or a still image. The camera control circuit 22 recognizes objects by image processing the image data input from the camera 15. The camera control circuit 22 has an ISP (image signal processor) function.

[0021] The main control circuit 23 is connected to the sonar control circuit 21 and the camera control circuit 22. The main control circuit 23 receives the detection results of objects in the detection areas A1, A2, A3, and A4 of the sonars 11, 12, 13, and 14 from the sonar control circuit 21. The main control circuit 23 receives image data after image processing in the imaging area A5 of the camera 15 from the camera control circuit 22. The main control circuit 23 detects obstacles behind the vehicle 100 based on the detection results of objects in the detection areas A1, A2, A3, and A4 and the image data in the imaging area A5.

[0022] Furthermore, the main control circuit 23 detects an obstacle in the detection area of ​​the other sonars based on an image acquired by the camera 15 during a predetermined period from when the transmitting unit of one of the sonars 11, 12, 13, and 14 transmits ultrasonic waves until the receiving unit receives the reflected waves. In this case, the main control circuit 23 detects an obstacle in the detection area of ​​one sonar after recognizing the image acquired by the camera 15. Furthermore, when the main control circuit 23 detects an obstacle based on the image acquired by the camera 15, it causes the sonar control circuit 21 to increase the sensitivity of obstacle detection by the sonars 11, 12, 13, and 14.

[0023] <Detection processing method using cameras and sonar> 1 and 2, sonars 11, 12, 13, and 14 partially overlap each other in their detection areas A1, A2, A3, and A4, and therefore transmit and receive ultrasonic waves in the order of sonar 12, sonar 11, sonar 14, and sonar 13. Meanwhile, camera 15 operates constantly and captures images of imaging area A5.

[0024] The detection time for obstacle detection using sonars 11, 12, 13, 14 and camera 15 is divided into a first detection time, a second detection time, a third detection time, and a fourth detection time corresponding to the four sonars 11, 12, 13, 14. Sonar 12 transmits ultrasonic waves at the first detection time, sonar 11 transmits ultrasonic waves at the second detection time, sonar 14 transmits ultrasonic waves at the third detection time, and sonar 13 transmits ultrasonic waves at the fourth detection time. Also, camera 15 captures images at all of the first detection time, second detection time, third detection time, and fourth detection time.

[0025] First, at the first detection time, the sonar 12 transmits ultrasonic waves, and the sonars 11 and 12 receive the reflected waves and output them to the sonar control circuit 21. Also, at the first detection time, the camera 15 acquires a first image when the sonar 12 transmits ultrasonic waves, and acquires a second image when the sonars 11 and 12 receive the reflected waves, and output them to the camera control circuit 22. Next, at the second detection time, the sonar 11 transmits ultrasonic waves, and the sonars 11 and 12 receive the reflected waves, and output them to the sonar control circuit 21. Also, at the second detection time, the camera 15 acquires a third image when the sonar 11 transmits ultrasonic waves, and acquires a fourth image when the sonars 11 and 12 receive the reflected waves, and output them to the camera control circuit 22.

[0026] The camera control circuit 22 performs image processing on the first image to recognize the object in the latter half of the first detection time, and performs image processing on the second image to recognize the object in the first half of the second detection time. The camera control circuit 22 also performs image processing on the third image to recognize the object in the latter half of the second detection time, and performs image processing on the fourth image to recognize the object in the first half of the third detection time.

[0027] Similarly, at the third detection time, the sonar 14 transmits ultrasonic waves, and the sonars 13 and 14 receive the reflected waves and output them to the sonar control circuit 21. Also, at the third detection time, the camera 15 acquires a fifth image when the sonar 14 transmits ultrasonic waves, and the camera 15 acquires a sixth image when the sonars 11 and 12 receive the reflected waves, and output them to the camera control circuit 22. Then, at the fourth detection time, the sonar 13 transmits ultrasonic waves, and the sonars 13 and 14 receive the reflected waves, and output them to the sonar control circuit 21. Also, at the fourth detection time, the camera 15 acquires a seventh image when the sonar 13 transmits ultrasonic waves, and the camera 15 acquires an eighth image when the sonars 13 and 14 receive the reflected waves, and output them to the camera control circuit 22.

[0028] The camera control circuit 22 performs image recognition on the object by image processing the fifth image in the latter half of the third detection time, and performs image recognition on the object by image processing the sixth image in the first half of the fourth detection time. Also, the camera control circuit 22 performs image recognition on the object by image processing the seventh image in the latter half of the fourth detection time, and performs image recognition on the object by image processing the eighth image in the first half of the next first detection time.

[0029] The main control circuit 23 detects an obstacle behind the vehicle 100 during a detection time consisting of a first detection time, a second detection time, a third detection time, and a fourth detection time, based on the object detection results in the detection areas A1, A2, A3, and A4 input from the sonar control circuit 21 and the image data in the imaging area A5 input from the camera control circuit 22.

[0030] At this time, objects can be detected in the detection areas A1 and A2 during the first detection time from when the transmitting section of sonar 12 transmits ultrasonic waves to when the receiving section receives the reflected waves, and during the second detection time from when the transmitting section of sonar 11 transmits ultrasonic waves to when the receiving section receives the reflected waves. However, during the first detection time and the second detection time, sonars 13 and 14 are not in operation, making it difficult to detect objects in the detection areas A3 and A4.

[0031] Therefore, during the first and second detection times when the sonars 13 and 14 are not operating, detection of objects in the detection areas A3 and A4 is performed based on images acquired by the camera 15. For example, during the first and second detection times, detection of objects in the detection areas A3 and A4 is performed based on the first, second, and third images acquired by the camera 15 and recognized by the camera control circuit 22.

[0032] Similarly, at the third and fourth detection times when the sonars 11 and 12 are not operating, object detection is performed in the detection areas A1 and A2 based on images acquired by the camera 15. For example, at the third and fourth detection times, object detection is performed in the detection areas A1 and A2 based on the fourth, fifth, sixth, and seventh images acquired by the camera 15 and recognized by the camera control circuit 22.

[0033] In addition, when the main control circuit 23 detects an obstacle in the detection areas A3, A4 based on the first, second, and third images acquired by the camera 15 at the first detection time and the second detection time, it instructs the sonar control circuit 21 to increase the sensitivity of obstacle detection by the sonars 13, 14 activated at the third detection time and the fourth detection time.

[0034] In the above explanation, during the detection time when any of the sonars 11, 12, 13, and 14 are not operating, object detection is performed using time-delayed images acquired by the camera 15 and recognized by the camera control circuit 22, but the time delay period of the images may be set appropriately according to the performance of the camera 15 and the camera control circuit 22. Furthermore, when detecting an object based on images acquired by the camera 15 during the detection time when the sonars 11, 12, 13, and 14 are not operating, it is preferable for the camera control circuit 22 to shorten the processing time without performing distortion correction processing.

[0035] <Obstacle detection method> 1 and 3, in the operation of the sonars 11, 12, 13, and 14 and the processing of the sonar control circuit 21, the sonar 12 transmits in step S11, and the sonars 11 and 12 receive in step S12. The sonar 11 transmits in step S13, and the sonars 11 and 12 receive in step S14. The sensitivity of the sonars 13 and 14 is adjusted in step S15. Then, the sonar 14 transmits in step S16, and the sonars 13 and 14 receive in step S17. The sonar 13 transmits in step S18, and the sonars 13 and 14 receive in step S19. The sensitivity of the sonars 11 and 12 is adjusted in step S20.

[0036] In the operation of camera 15 and the processing of camera control circuit 22, in step S31, a first image is acquired and then recognized. In step S32, a second image is acquired and then recognized. In step S33, a third image is acquired and then recognized. In step S34, a fourth image is acquired and then recognized. In step S35, a fifth image is acquired and then recognized. In step S36, a sixth image is acquired and then recognized. In step S37, a seventh image is acquired and then recognized. In step S38, an eighth image is acquired and then recognized.

[0037] In step S41, the main control circuit 23 detects an obstacle behind the vehicle 100 based on the object detection results in the detection areas A1, A2, A3, A4 input from the sonar control circuit 21 and the image data in the imaging area A5 input from the camera control circuit 22. At this time, the main control circuit 23 detects an obstacle based on the image acquired by the camera 15 in the detection areas of the other sonars during a predetermined period from when the transmitting unit of one of the sonars 11, 12, 13, 14 transmits ultrasonic waves to when the receiving unit receives the reflected waves.

[0038] In addition, when the main control circuit 23 detects an obstacle in the detection areas A3, A4 based on the first, second, and third images acquired by the camera 15, it instructs the sonar control circuit 21 to increase the sensitivity of obstacle detection by the sonars 13, 14 activated at the third detection time and the fourth detection time.

[0039] [Second embodiment] 4, the obstacle detection device 10A is mounted on a vehicle 100. The obstacle detection device 10 has multiple sonars 11, 12, 13, 14, a camera 15, and a control circuit 16A. The control circuit 16A has a sonar control circuit 21, a camera control circuit 22, and a main control circuit 23A.

[0040] The main control circuit 23 detects an obstacle based on an image of a blind spot area A6, which is a blind spot of the sonars 11, 12, 13, 14 around the vehicle 100, among images acquired by the camera 15. The blind spot area A6 may be an area outside the detection areas A1, A2, A3, A4 of the sonars 11, 12, 13, 14, or an area inside the detection areas A1, A2, A3, A4 that is excluded from the detection target of the sonars 11, 12, 13, 14 when there is an attachment (for example, a trailer hitch) on the vehicle. In addition, when there is an attachment (for example, a trailer hitch installed on the rear of the vehicle) on the vehicle in the area inside the detection areas A1, A2, A3, A4, the control circuit 16A may not perform sonar detection for a certain period of time. For example, when a trailer is connected to the trailer hitch, if there is an input from another control circuit of the vehicle, the control circuit 16A may perform control not to perform sonar detection.

[0041] The sonars 11, 12, 13, 14 are set with detection areas A1, A2, A3, A4, and some areas of the detection areas A1, A2, A3, A4 overlap. The camera 15 is set with an imaging area A5. The detection areas A1, A2, A3, A4 by the sonars 11, 12, 13, 14 and the imaging area A5 by the camera 15 almost overlap, but do not overlap in part. The imaging area A5 by the camera 15 is an area close to the rear end 101 of the vehicle 100, and the detection areas A1, A2, A3, A4 by the sonars 11, 12, 13, 14 are areas slightly far from the rear end 101 of the vehicle 100. Therefore, the multiple sonars 11, 12, 13, 14 generate a blind spot area A6 in an area close to the rear end 101 of the vehicle 100.

[0042] Therefore, in the blind spot area A6 near the rear end 101 of the vehicle 100, which is difficult to detect using multiple sonars 11, 12, 13, and 14, the main control circuit 23 detects obstacles based on images acquired by the camera 15 and recognized by the camera control circuit 22.

[0043] In addition, since the multiple sonars 11, 12, 13, 14 are disposed exposed at the rear end 101 of the vehicle 100, there is a risk that foreign matter will adhere to the surface, making it difficult to transmit ultrasonic waves or receive reflected waves. Therefore, when signals receiving reflected waves from the sonars 11, 12, 13, 14 are not input, the sonar control circuit 21 detects obstacles in the detection areas A1, A2, A3, A4 of the multiple sonars 11, 12, 13, 14 that did not receive reflected waves, based on images acquired by the camera 15 and recognized by the camera control circuit 22.

[0044] 5, the main control circuit 23 detects an obstacle at a first detection time based on a first image recognized by the camera control circuit 22. The main control circuit 23 detects an obstacle at a second detection time based on a second image recognized by the camera control circuit 22. Similarly, the main control circuit 23 detects an obstacle at a third detection time based on a fourth image and a fifth image recognized by the camera control circuit 22. The main control circuit 23 detects an obstacle at a fourth detection time based on a sixth image and a seventh image recognized by the camera control circuit 22.

[0045] [Control system] As shown in Fig. 6, the control circuit 16 has a sonar control circuit 21, a camera control circuit 22, and a main control circuit 23. The sonar control circuit 21 has a power supply unit 21a and an inch-face (IF) 21b. In the sonar control circuit 21, sonars 11, 12, 13, 14, sonars 11A, 12A, 13A, 14A, and sonars 11B, 12B, 13B, 14B are connected to the inch-face 21b. The sonars 11, 12, 13, 14 are forward sonars, sonars 11A, 12A, 13A, 14A are rearward sonars, and sonars 11B, 12B, 13B, 14B are side sonars.

[0046] The camera control circuit 22 has a power supply unit 22a, an image processing unit 22b, a memory 22c, and an inch-interface (IF) 22d. The image processing unit 22b includes an ISP (image signal processor). In the camera control circuit 22, cameras 15, 15A, 15B, and 15C are connected to the inch-interface 22d. The camera 15 is a rear camera, the camera 15A is a front camera, and the cameras 15B and 15C are left and right side cameras.

[0047] The main control circuit 23 has a power supply unit 23a, a control circuit 23b, and an external inch-interface (IF) 23c. The control circuit 23b is configured as a SOC (System on Chip) having an MPU (Micro Processing Unit). However, some functions of the sonar control circuit 21 and the camera control circuit 22 may be integrated, the image processing unit 22b may be an SOC, and the control circuit 23b may be an MPU.

[0048] [Modification of control system] As shown in FIG. 7, the control circuit 16 has a sonar control circuit 21 and a camera control circuit 22. The sonars 11, 12, 13, and 14 are connected to the sonar control circuit 21, and the camera control circuit 22 is connected to the camera 15. The camera 15 is also connected to the sonar control circuit 21. Here, the control circuit 23b is configured with an SOC, the camera control circuit 22 is configured with an MCU (Micro Controller Unit), and the sonar control circuit 21 is configured with an MCU. However, the sonar control circuit 21 has an MCU for simple image processing. Therefore, for example, when a part of the sonars 11, 12, 13, and 14 fails, the sonar control circuit 21 detects an object in the detection area of ​​the failed sonar based on the image acquired by the camera 15.

[0049] [Effects of this embodiment] The obstacle detection device of the first embodiment comprises a plurality of sonars (ranging sensors) 11, 12, 13, 14 that transmit ultrasonic waves, receive reflected ultrasonic waves, and are arranged on the vehicle 100 at intervals in a direction intersecting the transmission and reception direction of the ultrasonic waves, a camera 15 that is arranged on the vehicle 100 and can acquire images corresponding to detection areas A1, A2, A3, A4 by the plurality of sonars 11, 12, 13, 14, and control circuits 16, 16A that detect obstacles around the vehicle 100 based on the reflected waves received by the plurality of ranging sensors and the image acquired by the camera 15, and the control circuit 16, 16A detects obstacles in the detection areas by the other sonars based on the image acquired by the camera 15 during a predetermined period from when one of the plurality of sonars 11, 12, 13, 14 transmits ultrasonic waves to when it receives the reflected waves.

[0050] According to the obstacle detection device of the first aspect, for example, during a predetermined period from when the sonars 11 and 12 transmit ultrasonic waves until when they receive the reflected waves, the sonars 13 and 14 do not transmit ultrasonic waves, and therefore during this predetermined period, an obstacle is detected based on an image acquired by the camera 15 in the detection areas A3 and A4 of the sonars 13 and 14. Therefore, it is possible to constantly detect an obstacle in the detection areas A1, A2, A3, and A4 of the multiple sonars 11, 12, 13, and 14, thereby improving the accuracy of obstacle detection.

[0051] The obstacle detection device according to the second embodiment detects obstacles in the detection areas of the other sonars 11, 12, 13, and 14 after recognizing the image acquired by the camera 15. Therefore, it is possible to improve the accuracy of detecting obstacles using the image acquired by the camera 15.

[0052] When an obstacle detection device according to the third aspect detects an obstacle based on an image acquired by the camera 15, the obstacle detection sensitivity of the sonars 11, 12, 13, and 14 is increased. Therefore, the accuracy of the obstacle detection by the sonars 11, 12, 13, and 14 can be improved.

[0053] The obstacle detection device according to the fourth aspect detects an obstacle based on an image of a blind spot area A6, which is a detection blind spot of the multiple sonars 11, 12, 13, and 14 around the vehicle 100, among images acquired by the camera 15. Therefore, since an obstacle is detected based on an image acquired by the camera 15 for the blind spot area A6 of the sonars 11, 12, 13, and 14, an obstacle in the vicinity of the vehicle 100 can be appropriately detected.

[0054] The obstacle detection device according to the fifth aspect detects an obstacle based on an image acquired by the camera 15 in a detection area of ​​the sonars 11, 12, 13, 14 that did not receive a reflected wave, among the multiple sonars 11, 12, 13, 14. Therefore, even if a failure occurs in some of the sonars 11, 12, 13, 14, the obstacle detection device can properly detect an obstacle in the detection areas A1, A2, A3, A4.

[0055] In the obstacle detection device according to the sixth aspect, multiple sonars 11, 12, 13, 14 and a camera 15 are installed at the rear of a vehicle 100, and when the control circuits 16, 16A determine that another vehicle is connected to the rear of the vehicle 100, the control circuits 16, 16A stop detecting obstacles based on images of blind spot areas that are blind spots for the multiple sonars 11, 12, 13, 14 around the vehicle 100, among images acquired by the camera 15. This makes it possible to improve the accuracy of obstacle detection by the sonars 11, 12, 13, 14.

[0056] In the above embodiment, the four sonars 11, 12, 13, and 14 are each configured as a unit, and when one of the sonars 11, 12 and 13, 14 is not in operation, an obstacle is detected using an image acquired by the camera 15 for the detection areas A1, A2, A3, and A4 of the other unit. However, this is not limited to the configuration. For example, when one sonar 11 of the four sonars 11, 12, 13, and 14 is in operation, an obstacle may be detected using an image acquired by the camera 15 for the detection areas A2, A3, and A4 of the remaining three sonars 12, 13, and 14.

[0057] In addition, the notation "... part" in the above-mentioned embodiments may be replaced with other notations such as "... circuitry", "... assembly", "... device", "... unit", or "... module".

[0058] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or amended examples within the scope of the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the disclosure.

[0059] Although specific examples of the present disclosure have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above. [Explanation of symbols]

[0060] 10,10A Obstacle detection device 11,12,13,14 Sonar 15 Camera 16,16A Control circuit 21 Sonar control circuit 22 Camera control circuit 23, 23A Main control circuit 100 vehicles A1, A2, A3, A4 detection area A5 Imaging area A6 Blind spot area

Claims

1. a plurality of distance measuring sensors that transmit ultrasonic waves and receive reflected waves of the ultrasonic waves and are arranged on the vehicle at intervals in a direction intersecting a transmission and reception direction of the ultrasonic waves; a camera disposed on the vehicle for acquiring an image corresponding to a detection area detected by the plurality of distance measuring sensors; a control circuit that detects obstacles around the vehicle based on the reflected waves received by the plurality of distance measuring sensors and the images acquired by the camera; Equipped with the control circuit detects an obstacle based on the image acquired by the camera within a predetermined period from when a first distance measurement sensor of the plurality of distance measurement sensors transmits an ultrasonic wave until when a first reflected wave is received, within a detection area of ​​the other distance measurement sensors of the plurality of distance measurement sensors; Obstacle detection device.

2. the control circuit detects an obstacle in the detection area of ​​the other distance measuring sensor after recognizing the image acquired by the camera; The obstacle detection device according to claim 1 .

3. When an obstacle is detected based on the image acquired by the camera, the control circuit increases the sensitivity of the obstacle detection by the distance measuring sensor.

3. An obstacle detection device according to claim 1 or 2.

4. the control circuit detects an obstacle based on an image of a blind spot area around the vehicle that is a blind spot for detection by the plurality of distance measuring sensors, among images acquired by the camera; The obstacle detection device according to claim 1 .

5. the control circuit detects an obstacle in a detection area of ​​the second distance measuring sensor based on an image acquired by the camera when the second distance measuring sensor does not receive a second reflected wave of the ultrasonic wave transmitted by the second distance measuring sensor among the plurality of distance measuring sensors; The obstacle detection device according to claim 1 .

6. the plurality of distance measuring sensors and the camera are installed at a rear of the vehicle; When the control circuit determines that another vehicle is connected to the rear of the vehicle, it stops detecting obstacles based on images of blind spot areas around the vehicle that are blind spots for the plurality of distance measuring sensors, among images acquired by the camera. The obstacle detection device according to claim 1 .

7. A method for detecting an obstacle by a vehicle, comprising: receiving a first reflected wave of an ultrasonic wave transmitted by a first ranging sensor among a plurality of ranging sensors included in the obstacle detection device disposed in the vehicle; a camera included in the obstacle detection device acquires images including detection areas of other distance measurement sensors among the plurality of distance measurement sensors during a predetermined period from when the first distance measurement sensor transmits the ultrasonic wave until when the first reflected wave is received; a control circuit included in the obstacle detection device detects an obstacle around the vehicle based on the first reflected wave and the image acquired by the camera; each of the plurality of distance measuring sensors sequentially transmits the ultrasonic waves and sequentially receives reflected waves of the ultrasonic waves, and is disposed on the vehicle at intervals in a direction intersecting a direction of transmission and reception of the ultrasonic waves; Obstacle detection method.

8. After recognizing the image acquired by the camera, an obstacle is detected in the detection area of ​​the other distance measuring sensor. The obstacle detection method according to claim 7.

9. When an obstacle is detected based on the image acquired by the camera, the sensitivity of obstacle detection by the plurality of distance measuring sensors is increased. The obstacle detection method according to claim 7.

10. An obstacle is detected based on an image of a blind spot area around the vehicle that is a blind spot for the plurality of distance measuring sensors among the images acquired by the camera. The obstacle detection method according to claim 7.

11. When a second ranging sensor among the plurality of ranging sensors does not receive a second reflected wave of the ultrasonic waves transmitted by the second ranging sensor, an obstacle is detected in the detection area of ​​the second ranging sensor based on the image acquired by the camera. The obstacle detection method according to claim 7.

12. When the plurality of distance measuring sensors and the camera are installed at the rear of the vehicle and another vehicle is connected to the rear of the vehicle, stopping obstacle detection based on images of blind spot areas around the vehicle that are blind spots for detection by the plurality of distance measuring sensors, among images acquired by the camera; The obstacle detection method according to claim 7.

13. A program to be executed by an obstacle detection device, comprising: receiving a first reflected wave of an ultrasonic wave transmitted by a first distance measuring sensor among a plurality of distance measuring sensors included in the obstacle detection device disposed on a vehicle; a camera included in the obstacle detection device acquires images including detection areas of other distance measurement sensors among the plurality of distance measurement sensors during a predetermined period from when the first distance measurement sensor transmits the ultrasonic wave until when the first reflected wave is received; a control circuit included in the obstacle detection device detects an obstacle around the vehicle based on the first reflected wave and the image acquired by the camera; Each of the plurality of distance measuring sensors sequentially transmits the ultrasonic waves and sequentially receives the reflected waves of the ultrasonic waves, and is disposed on the vehicle at intervals in a direction intersecting the direction of transmission and reception of the ultrasonic waves.

Images