Object detection device

Abstract

The object detection device includes a calculation device. The arithmetic device detects a predetermined object from each of the plurality of stereoscopic images, generates a parallax image from each of the plurality of stereoscopic images, synthesizes the plurality of parallax images for a predetermined range including the object, thereby generating a single parallax image, and outputs the single parallax image. And calculating the distance to the object and the height of the object from the single parallax image.

Description

Technical Field

[0001] This invention relates to an object detection device. Background Technology

[0002] A road surface condition detection device is known to exist, which uses a so-called stereo camera, which configures two cameras to generate parallax in the horizontal direction, to detect road surface conditions such as height and the presence or absence of obstacles in front of a vehicle. For example, Patent Document 1 describes a road surface condition detection device that calculates the parallax information between corresponding points in a stereo image captured by a stereo camera, calculates the distance from the stereo camera to the detection target point on the road surface based on the calculated parallax information and the parameter information of the stereo camera, and detects road surface conditions such as height and the presence or absence of obstacles based on the calculated distance.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-199178 Summary of the Invention

[0006] The problem the invention aims to solve

[0007] In the road surface condition detection device described in Patent Document 1, when the amount of light is insufficient, such as at dusk or night, the parallax calculated from the stereo image becomes less, making it difficult to detect obstacles such as bumps, potholes, or fallen objects on the road surface based on the parallax.

[0008] The purpose of this invention is to provide an object detection device that can detect obstacles with high accuracy even when the amount of lighting is insufficient, such as at dusk or at night.

[0009] Technical means to solve the problem

[0010] According to one aspect of the present invention, an object detection apparatus includes a computing unit that detects a specified object based on each of a plurality of stereoscopic images, generates a disparity image based on each of the plurality of stereoscopic images, and synthesizes the plurality of disparity images for a specified range containing the object to generate a single disparity image, and calculates the distance to the object and the height of the object based on the single disparity image.

[0011] The effects of the invention

[0012] According to the present invention, obstacles can be detected with high accuracy even when the amount of lighting is insufficient, such as at dusk or at night. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of a vehicle equipped with the object detection device according to the first embodiment.

[0014] Figure 2 This is a schematic diagram showing the hardware configuration of the object detection device according to the first embodiment.

[0015] Figure 3 This is a schematic diagram illustrating the functional configuration of the object detection device according to the first embodiment.

[0016] Figure 4 This is a flowchart illustrating an example of object detection processing performed by the object detection apparatus of the first embodiment.

[0017] Figure 5 This is a schematic diagram illustrating an example of positional correction between parallax images.

[0018] Figure 6 This is a schematic diagram of a vehicle equipped with the object detection device according to the second embodiment.

[0019] Figure 7 This is a flowchart illustrating an example of object detection processing performed by the object detection apparatus of the second embodiment.

[0020] Figure 8 This is a schematic diagram of a vehicle equipped with the object detection device according to the third embodiment.

[0021] Figure 9 This is a flowchart illustrating an example of object detection processing performed by the object detection apparatus of the third embodiment.

[0022] Figure 10 This is a schematic diagram of a vehicle equipped with the object detection device of Modified Example 1. Detailed Implementation

[0023] (First Implementation)

[0024] Reference Figures 1-5 The object detection apparatus according to an embodiment of the present invention will be described.

[0025] Figure 1This is a schematic diagram of a vehicle equipped with the object detection device according to the first embodiment. A camera device 2 and an object detection device 3 are mounted on the vehicle 1. The camera device 2 is a so-called stereo camera, having a pair of camera units. The camera device 2 captures images of the front of the vehicle 1 and outputs a pair of captured images (stereo images) as video signals to the object detection device 3. The optical axis 4a of one camera unit is located slightly to the left of the center of the vehicle 1 in the horizontal direction. The optical axis 4b of the other camera unit is located slightly to the right of the center of the vehicle 1 in the horizontal direction. In the following description, the distance between the two optical axes in the horizontal direction is referred to as the baseline length. One camera unit captures images of a range 5a slightly to the left of the front of the vehicle 1. The other camera unit captures images of a range 5b slightly to the right of the front of the vehicle 1.

[0026] Figure 2 This is a schematic diagram showing the hardware configuration of the object detection device according to the first embodiment. The imaging device 2 includes a first imaging unit 20a and a second imaging unit 20b. In the following description, the first imaging unit 20a and the second imaging unit 20b will be collectively referred to as imaging unit 20. Imaging unit 20 includes an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor and an imaging optical system. The first imaging unit 20a and the second imaging unit 20b respectively capture images of the front of the vehicle 1 and output a pair of imaging signals (stereoscopic image signals) to the object detection device 3. The first imaging unit 20a and the second imaging unit 20b are arranged such that the positions of the optical axis centers in the vertical direction are approximately equal. As described above, since the first imaging unit 20a and the second imaging unit 20b are arranged such that only the baseline length is separated in the horizontal direction, parallax corresponding to the distance from the imaging unit 20 to the subject and the baseline length is generated in the captured images that are the respective imaging results.

[0027] The object detection device 3 is composed of a computer equipped with a CPU (Central Processing Unit), MPU (Microprocessor), DSP (Digital Signal Processor) and other computing devices 11, ROM (Read Only Memory), flash memory, hard disk drive and other non-volatile memory 12, RAM (Random Access Memory) and other volatile memory 13, input / output interface 14, and other peripheral circuits. This hardware works in conjunction with software to achieve various functions. Furthermore, the object detection device 3 can be composed of a single computer or multiple computers. Additionally, the computing device 11 can be an ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or similar components.

[0028] The non-volatile memory 12 stores programs capable of performing various calculations. That is, the non-volatile memory 12 is a storage medium (storage device) capable of reading programs that implement the functions of this embodiment. The volatile memory 13 is a storage medium (storage device) that temporarily stores the calculation results of the arithmetic unit 11 and signals input from the input / output interface 14. The arithmetic unit 11 is a device that expands the program stored in the non-volatile memory 12 in the volatile memory 13 and performs calculations, performing prescribed arithmetic processing on data obtained from the input / output interface 14, the non-volatile memory 12, and the volatile memory 13 according to the program.

[0029] The input / output interface 14 is connected to the vehicle control device 6 and the camera unit 20 of the camera device 2. The input unit of the input / output interface 14 converts signals input from various devices (such as the camera unit 20) into data that can be processed by the arithmetic unit 11. Furthermore, the output unit of the input / output interface 14 generates an output signal corresponding to the processing result of the arithmetic unit 11 and outputs this signal to various devices (such as the vehicle control device 6). The vehicle control device 6, for example, controls the accelerator, brakes, suspension, etc. of the vehicle 1.

[0030] Figure 3 This is a schematic diagram illustrating the functional configuration of the object detection device according to the first embodiment. The object detection device 3 includes a parallax image generation unit 31, a road surface estimation unit 32, an object detection unit 33, a parallax image synthesis unit 34, an object determination unit 35, and a vehicle control unit 36.

[0031] The parallax image generation unit 31 generates a parallax image based on the stereoscopic image output by the imaging device 2 using a known method. For example, the parallax image generation unit 31 performs parallelization processing on the stereoscopic image. Parallax processing is a so-called stereo parallelization process. That is, it is a process of converting the same location in the world coordinate system into images arranged at the same height in both sides of two photographic images. Parallax processing is achieved by projecting a predetermined projection plane, such as a perspective projection plane, from a state that is geometrically deformed by the imaging optical system. The parallax image generation unit 31 performs parallax calculation processing to calculate the parallax between the parallelized photographic images. For each pixel of one photographic image, the parallax image generation unit 31 searches for pixels at the same location in the other photographic image, calculates the difference in horizontal coordinates (i.e., parallax) of the same location in the other photographic image based on the one photographic image, and thus generates a parallax image. The parallax image generation unit 31 uses methods such as SSD (Squared Difference) or SAD (Absolute Difference) to search for the same point.

[0032] The road surface estimation unit 32 estimates the area occupied by the road surface (road surface region) from the stereo image and the parallax image. The road surface is considered to be an area with a certain degree of flatness and whose normal vector is nearly vertically upward. Therefore, the road surface estimation unit 32, for example, converts the points contained in the parallax image into a group of points in three-dimensional space, and determines the road surface region by performing plane fitting on the group of points.

[0033] The object detection unit 33 detects specified objects from the road surface region determined by the road surface estimation unit 32 within the entire stereoscopic image using known methods such as semantic segmentation based on machine learning. The image used for object detection can be an image within the stereoscopic image. Specified objects refer to, for example, protrusions, cracks, potholes, or other uneven parts, which are structures existing on the road surface where vehicle 1 travels and can become obstacles to vehicle 1's movement. Additionally, objects that may fall onto the road from preceding vehicles and potentially impede vehicle 1's movement, such as tires or cardboard, may also be included among the objects being detected. Furthermore, the object detection method is not limited to these; for example, techniques such as template matching may be used.

[0034] The parallax image synthesis unit 34 synthesizes multiple parallax images generated by the parallax image generation unit 31 from multiple stereoscopic images captured by the imaging device 2 at different times, within the range of objects detected by the object detection unit 33, thereby generating a single parallax image. Since these multiple parallax images are based on multiple stereoscopic images captured at different times, the relative position of the object of interest differs in each parallax image. The parallax image synthesis unit 34 aligns the detected object's position within the multiple parallax images that form the synthesis object, generating a single parallax image.

[0035] The object determination unit 35 calculates the distance to the object detected by the object detection unit 33 based on a single parallax image synthesized by the parallax image synthesis unit 34. Based on the calculated distance to the object and the object's height, the object determination unit 35 determines whether the object is a predefined type of obstacle existing on the vehicle 1's travel path and potentially obstructing the vehicle 1's movement. Predefined types of obstacles refer to structures such as protrusions, cracks, and potholes that exist on the road surface where the vehicle 1 travels and can obstruct its movement. Additionally, objects such as tires and cardboard that have fallen onto the road from preceding vehicles and may impede the vehicle 1's movement can also be included in the predefined types of obstacles described here.

[0036] When an object detected by the object detection unit 33 is determined by the object determination unit 35 to be an obstacle of a specified type, the vehicle control unit 36 ​​outputs a specified control signal to the vehicle control device 6, causing the vehicle control device 6 to perform specified vehicle control on the vehicle 1. Specified vehicle control may include, for example, actuating the brakes of the vehicle 1 to reduce its speed, or increasing the damping force of the vehicle 1's suspension to improve its shock absorption. Furthermore, if the object is similar to the aforementioned falling object, control may include warning the driver of the presence of the falling object via sound or visual display, or automatically steering the vehicle 1 to avoid the falling object.

[0037] Figure 4 This is a flowchart illustrating an example of object detection processing performed by the object detection apparatus of the first embodiment. Figure 4 The object detection process shown is repeatedly executed by the computing unit 11 of the object detection device 3, for example, during the movement of the vehicle 1. In step S100, the parallax image generation unit 31 causes the imaging device 2 to capture a stereoscopic image and inputs the stereoscopic image to the object detection device 3. In step S110, the parallax image generation unit 31 generates a parallax image based on the stereoscopic image input in step S100.

[0038] In step S120, the road surface estimation unit 32 estimates the road surface area based on the stereoscopic image input in step S100 and the parallax image generated in step S110. In step S130, the object detection unit 33 detects a specified object from the road surface area based on one of the stereoscopic images input in step S100.

[0039] In step S140, the object detection unit 33 determines whether the detection of the object specified in step S130 was successful. If the detection of the object specified in step S130 is successful, that is, if one of the images in the stereoscopic image contains an object that is presumed to be the specified object, the process proceeds to step S150. On the other hand, if the detection of the object specified in step S130 fails, that is, if no object presumed to be the specified object is found in one of the images in the stereoscopic image, the process ends. Figure 4 The processing shown.

[0040] In step S150, the object detection unit 33 determines whether the magnitude of the disparity in the disparity image generated in step S110 is above a certain level. Specifically, it sums all disparity values ​​contained in the road surface area of ​​the entire disparity image and determines whether the sum of the disparity values ​​is above a predetermined threshold. For example, in dark environments with insufficient lighting, such as at dusk or night, the disparity values ​​contained in the disparity image are smaller, so the probability that the sum of the disparity values ​​here is less than the predetermined threshold is higher. If the sum of the disparity values ​​is less than the predetermined threshold in step S150, the process proceeds to step S160. On the other hand, if the sum of the disparity values ​​is above the predetermined threshold in step S150, the process proceeds to step S230.

[0041] The processing in steps S160 to S190 is the same as that in steps S100 to S130. Specifically, in step S160, the parallax image generation unit 31 causes the imaging device 2 to capture a stereoscopic image and inputs the stereoscopic image to the object detection device 3. In step S170, the parallax image generation unit 31 generates a parallax image based on the stereoscopic image input in step S160. In step S180, the road surface estimation unit 32 estimates the road surface area based on the stereoscopic image input in step S160 and the parallax image generated in step S170. In step S190, the object detection unit 33 detects a specified object from the road surface area based on one of the images in the stereoscopic image input in step S160.

[0042] In step S200, the disparity image synthesis unit 34 performs position correction on the disparity images generated in step S110 and step S180. Position correction here refers to the process of shifting the disparity images in the longitudinal and lateral directions so that the same object detected in each disparity image exactly overlaps. The shooting times (shooting timings) of the stereoscopic images on which the disparity images generated in step S110 and step S180 are based are different. Therefore, if the position of vehicle 1 changes during shooting, the relative position of the same object in the captured image (disparity image) is likely to change as well. Therefore, the disparity image synthesis unit 34 shifts the disparity images to perform position correction so that the disparity values ​​at the same location are added together during the synthesis of the disparity images.

[0043] In step S210, the disparity image synthesis unit 34 synthesizes the disparity image generated in step S110 and the disparity image generated in step S180 to generate a single disparity image. Disparity image synthesis is a process of adding disparity values ​​at the same location to generate a new disparity image. Furthermore, it is sufficient to synthesize a disparity image only over a defined area containing at least the object (e.g., a certain size area centered on the object's location, or a road surface area that repeats between disparity images); it is not necessary to synthesize the entire disparity image.

[0044] In step S220, the object detection unit 33 determines whether the magnitude of the disparity in the single disparity image generated in step S210 is above a certain level. Specifically, it sums all the disparity values ​​of the single disparity image generated in step S210 and determines whether the sum of the disparity values ​​is above a predetermined threshold. If the sum of the disparity values ​​is less than the threshold, the process proceeds to step S160, where a new stereoscopic image is captured, a new disparity image is generated, and a single disparity image including the new disparity image is synthesized. That is, the computing device 11 of this embodiment repeatedly generates a single disparity image while increasing the number of synthesized disparity images until the disparity value in the generated single disparity image reaches or exceeds the threshold. In step S220, if the sum of the disparity values ​​is above the threshold, the process proceeds to step S230.

[0045] In step S230, the object determination unit 35 calculates the distance to the detected object and the object's height based on the single disparity image synthesized in step S210. Furthermore, if step S210 is not executed (when proceeding from step S150 to step S230), the disparity image generated in step S110 is processed as a single disparity image. The same applies to subsequent processing.

[0046] In step S240, the object determination unit 35 determines, based on the distance to the object and the object's height calculated in step S230, whether the object is a defined type of obstacle existing on the vehicle 1's travel path and potentially obstructing the vehicle 1's movement. If it is determined to be a defined type of obstacle, the process proceeds to step S250. In step S250, the vehicle control unit 36 ​​outputs a defined control signal to the vehicle control device 6, causing the vehicle control device 6 to perform defined vehicle control on the vehicle 1, and the process ends. Figure 4 The processing. On the other hand, if in step S240 it is determined that the obstacle is not of the specified type, Figure 4 The processing is now complete.

[0047] Figure 5 This is a schematic diagram illustrating an example of position correction between parallax images. The parallax image synthesis unit 34 calculates the distance and direction of movement of vehicle 1 from the previous capture to the current capture based on vehicle 1's speed, steering angle, and yaw rate. The parallax image synthesis unit 34 estimates the estimated position 45 of the object in the currently generated parallax image 42 by applying the calculated distance and direction of movement of vehicle 1 to the position 43 of the object detected in the previously generated parallax image 41. When the position 44 of the object detected in the currently generated parallax image 42 and the estimated position 45 are sufficiently close, the parallax image synthesis unit 34 determines that they are the same object and performs position correction in a manner that makes positions 43 and 44 overlap.

[0048] According to the first embodiment described above, the following effects are achieved.

[0049] (1) The computing device 11 detects a specified object based on each of the multiple stereo images, generates a disparity image based on each of the multiple stereo images, synthesizes the multiple disparity images over a specified range containing the detected object to generate a single disparity image, and calculates the distance to the object and the height of the object based on the single disparity image. In this way, even in situations where the amount of lighting is insufficient, such as at dusk or night, the distance to the object or the height of the object can be calculated with high precision, thus enabling the high-precision detection of obstacles based on this high-precision information.

[0050] (2) The arithmetic unit 11 determines whether the object is a specified type of obstacle that exists on the driving path of the vehicle 1 and may become an obstacle to the driving of the vehicle 1, based on the distance to the object and the height of the object. In this way, obstacles can be detected with high accuracy even when the amount of lighting is insufficient, such as at dusk or at night.

[0051] (3) When the arithmetic unit 11 determines that an object is an obstacle, it performs prescribed vehicle control on the vehicle 1, such as actuating the brakes of the vehicle 1 to reduce the speed of the vehicle 1, or increasing the damping force of the suspension of the vehicle 1 to improve the degree of shock absorption of the suspension. In this way, the ride comfort of the vehicle when passing over bumps and potholes is improved. In addition, the behavior of the vehicle when passing over bumps and potholes can be stabilized, and unexpected movements of the vehicle can be suppressed.

[0052] (4) The processing unit 11 increases the number of synthesized multiple disparity images while repeatedly generating a single disparity image until the disparity value in the single disparity image reaches or exceeds a threshold. Therefore, disparity images with appropriate disparity values ​​can be generated regardless of the amount of illumination light.

[0053] (5) The computing device 11 generates multiple parallax images based on each of the multiple stereo images captured by the same camera device 2 at different times, and synthesizes these multiple parallax images to generate a single parallax image. In this way, it is not necessary to increase the number of camera devices 2 in order to prepare multiple parallax images, and the accuracy of obstacle detection can be improved at low cost.

[0054] (Second Implementation)

[0055] Reference Figure 6 , Figure 7 The object detection apparatus of the second embodiment of the present invention will be described. Furthermore, for components that are identical or equivalent to those described in the first embodiment, the main differences will be explained.

[0056] Figure 6 Is with Figure 1 The same figure is a schematic diagram of a vehicle equipped with the object detection device 103 of the second embodiment. In addition to the camera device 2 and the object detection device 103, the vehicle 101 is also equipped with headlights 107 and 108. Headlights 107 and 108 are illumination devices capable of illuminating light of a predetermined intensity at a predetermined angle in front of the vehicle 101. Headlights 107 and 108 are configured to change the illumination angle and intensity of the illumination light. The object detection device 103 can freely adjust the illumination angle and intensity of the illumination light from headlights 107 and 108.

[0057] Figure 7 Is with Figure 4 The same figure is a flowchart illustrating an example of object detection processing performed by the object detection apparatus 103 of the second embodiment. Figure 7 In the flowchart, Figure 4The flowchart adds a step S300 before step S100, and adds a step S310 before step S160.

[0058] In step S300, the computing device 11 randomly selects and sets the illumination angle and intensity of the headlights 107 and 108. Then, in step S100, the camera device 2 takes a picture of the subject illuminated by the headlights 107 and 108, and inputs the stereoscopic image to the object detection device 103.

[0059] In step S310, the computing device 11 randomly selects and sets the illumination angle and illumination intensity of the headlights 107 and 108. Furthermore, the range of the randomly selected illumination angle and illumination intensity is adjusted to cover the range of illumination light illuminating the object detected in step S130. Then, in step S160, the imaging device 2 captures an image of the subject illuminated by the headlights 107 and 108 and inputs the stereoscopic image to the object detection device 103.

[0060] Through the above processing, in step S210, the parallax image synthesis unit 34 synthesizes multiple parallax images generated from each of multiple stereoscopic images captured by the same camera device 2 at different times and under different lighting conditions, thereby generating a single parallax image.

[0061] According to the second embodiment described above, the following effects are achieved.

[0062] (1) The processing unit 11 generates multiple parallax images based on each of multiple stereo images captured by the same camera device 2 at different times and under different lighting conditions, and combines these multiple parallax images to generate a single parallax image. In this way, obstacles existing in locations where ambient light does not illuminate or is difficult to illuminate can be detected with high precision.

[0063] (2) The computing device 11 uses headlights 107 and 108 to change the lighting state. Since headlights 107 and 108 are considered to be standardly installed on general vehicles, the accuracy of obstacle detection can be improved without additional cost.

[0064] (Third implementation method)

[0065] Reference Figure 8 , Figure 9 The object detection apparatus of the third embodiment of the present invention will be described. Furthermore, for components that are identical or equivalent to those described in the first embodiment, the main differences will be explained.

[0066] Figure 8 Is with Figure 1 The same figure is a schematic diagram of a vehicle equipped with the object detection device 203 of the third embodiment. The vehicle 201 is equipped with camera device 2a, camera device 2b, and object detection device 203. Camera device 2a and camera device 2b have the same configuration as camera device 2 of the first embodiment.

[0067] Figure 9 Is with Figure 4 The same figure is a flowchart illustrating an example of object detection processing performed by the object detection apparatus 203 of the third embodiment. Figure 9 In the flowchart, instead Figure 4 The flowchart process in step S100 is replaced by the process in step S400, and the processes in steps S200 to S210 are replaced by the processes in steps S410 to S420. Additionally, the process in step S410 is deleted. Figure 4 The process of step S160 in the flowchart is executed immediately after the determination in step S220, followed by the process of step S430, instead of proceeding to step S160.

[0068] In step S400, the parallax image generation unit 31 causes the imaging devices 2a and 2b to capture stereoscopic images, and inputs the two stereoscopic images to the object detection device 203. Steps S110 to S130 are performed on the stereoscopic image captured by the imaging device 2a. Steps S170 to S190 are performed on the stereoscopic image captured by the imaging device 2b.

[0069] In step S410, the parallax image synthesis unit 34 performs position correction on the parallax image generated in step S110 (corresponding to the camera device 2a) and the parallax image generated in step S180 (corresponding to the camera device 2b). In this embodiment, the shooting time is the same, and since the mounting positions of the camera devices 2a and 2b are determined in advance, position correction can be performed based on the mounting positions of the camera devices 2a and 2b.

[0070] In step S420, the disparity image synthesis unit 34 synthesizes the disparity image generated in step S110 and the disparity image generated in step S180 to generate a single disparity image. That is, the computing device 11 of this embodiment generates multiple disparity images based on each of multiple stereoscopic images captured by multiple imaging devices 2a and 2b at approximately the same time, and synthesizes them to generate a single disparity image.

[0071] Subsequently, if the total disparity value is determined to be less than the threshold in step S220, the process proceeds to step S430. In step S430, the disparity image generation unit 31 causes the imaging devices 2a and 2b to capture stereoscopic images respectively, and inputs the two stereoscopic images to the object detection device 203. Then, steps S170 to S190, S410, and S420 are sequentially performed on one of the two stereoscopic images. If the disparity value is still insufficient, the same process is sequentially performed on the other of the remaining two stereoscopic images. Furthermore, if there are unprocessed stereoscopic images remaining, it is not necessary to capture them in step S430. In addition, when processing stereoscopic images captured at different times, the position correction performed in step S410 can be the same as in the first embodiment.

[0072] According to the third embodiment described above, the following effects are achieved.

[0073] (1) The processing unit 11 generates multiple parallax images based on each of the multiple stereoscopic images captured by the multiple imaging devices 2a and 2b respectively. In this way, it is easy to align the positions between the parallax images, and the amount of computation required for position correction can be reduced. In addition, it is expected to improve the accuracy of position correction.

[0074] The following variations are also within the scope of the present invention. The configurations shown in the variations can be combined with the configurations described in the above embodiments, or the configurations described in the different embodiments can be combined with each other. The configurations described in the following variations can also be combined with each other.

[0075] <Variation Example 1>

[0076] Figure 10 Is with Figure 1 The same figure is a schematic diagram of a vehicle equipped with the object detection device 303 of Modified Example 1. The vehicle 301 is equipped with single-lens imaging devices 2c, 2d, and 2e, which are not stereo cameras, as well as the object detection device 303.

[0077] In Modification 1, the processing unit 11 selects multiple pairs of images from multiple images captured by multiple imaging devices 2c, 2d, and 2e in different combinations. Then, these multiple pairs of images are treated as multiple stereo images, multiple parallax images are generated, and a single parallax image is synthesized. For example, the images captured by imaging device 2c and imaging device 2d are processed as a first stereo image. Similarly, the images captured by imaging device 2c and imaging device 2e are processed as a second stereo image. Likewise, the images captured by imaging device 2d and imaging device 2e are processed as a third stereo image.

[0078] Furthermore, more (four or more) single-lens camera devices can be mounted on vehicle 301. In this case, the combination of images captured can be considered as stereoscopic images.

[0079] According to the above variation example 1, the following effects are achieved.

[0080] (1) The processing unit 11 selects multiple pairs of images from multiple images captured by multiple camera devices 2c, 2d, and 2e in different combinations, treats the multiple pairs of images as multiple stereo images, and generates multiple parallax images. In this way, a low-cost object detection device that does not require a stereo camera can be provided.

[0081] <Variation Example 2>

[0082] In the third embodiment, three or more camera devices may be prepared. In this case, firstly, parallax images are generated sequentially based on the stereoscopic images captured by each camera device, and attempts are made to synthesize them until the total parallax value reaches or exceeds a threshold. If the total parallax value does not reach or exceed the threshold even after synthesizing all parallax images, parallax images are captured again, and parallax images are generated sequentially and attempts are made to synthesize them until the total parallax value reaches or exceeds the threshold.

[0083] <Variation Example 3>

[0084] The second and third embodiments can also be combined. That is, the lighting conditions can be changed when taking pictures at different times.

[0085] The embodiments of the present invention have been described above. However, the above embodiments only represent a part of the application examples of the present invention and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

[0086] Symbol Explanation

[0087] 1, 101, 201, 301… Vehicles; 2, 2a, 2b, 2c, 2d, 2e… Camera devices; 3, 103, 203, 303… Object detection devices; 6… Vehicle control devices; 11… Computing devices; 12… Non-volatile memory; 13… Volatile memory; 14… Input / output interface; 20… Camera unit; 20a… First camera unit; 20b… Second camera unit; 31… Parallax image generation unit; 32… Road surface estimation unit; 33… Object detection unit; 34… Parallax image synthesis unit; 35… Object determination unit; 36… Vehicle control unit.

Claims

1. An object detection device comprising a computing unit, characterized in that, The computing device detects a specified object based on each of the multiple stereo images. Generate a disparity image based on each of the plurality of stereo images. For a defined area containing the object, multiple disparity images are combined to generate a single disparity image. The distance to the object and the height of the object are calculated based on the single parallax image.

2. The object detection device according to claim 1, characterized in that, The computing device determines, based on the distance to the object and the height of the object, whether the object is a defined type of obstacle that exists on the vehicle's driving path and may become an obstacle to the vehicle's driving.

3. The object detection device according to claim 2, characterized in that, When the computing device determines that the object is an obstacle, it performs prescribed vehicle control on the vehicle.

4. The object detection device according to claim 1, characterized in that, The computing device increases the number of synthesized disparity images while repeatedly generating the single disparity image until the disparity value in the single disparity image reaches or exceeds a threshold.

5. The object detection device according to claim 1, characterized in that, The computing device generates multiple parallax images based on each of the multiple stereoscopic images captured by the same camera device at different times.

6. The object detection device according to claim 1, characterized in that, The computing device generates multiple parallax images based on each of the multiple stereoscopic images captured by the same camera device at different times and under different lighting conditions.

7. The object detection device according to claim 1, characterized in that, The computing device generates multiple parallax images based on each of the multiple stereoscopic images captured by multiple camera devices.

8. The object detection device according to claim 1, characterized in that, The computing device selects multiple pairs of images from multiple images captured by multiple camera devices in different combinations, and treats the multiple pairs of images as the multiple stereo images, thereby generating multiple parallax images.

Images