Object detection device

Abstract

Provided is an object detection device comprising an arithmetic device. The arithmetic device detects a prescribed object from each of a plurality of stereo images, generates a parallax image from each of the plurality of stereo images, combines the plurality of parallax images for a prescribed range including the object to generate a single parallax image, and calculates a distance to the object and the height of the object on the basis of the single parallax image.

Description

Object detection device

[0001] The present invention relates to an object detection device.

[0002] There is known a road surface condition detection device that uses a so-called stereo camera in which two cameras are arranged so as to generate parallax in the horizontal direction to detect road surface conditions such as the height of the road surface ahead of a vehicle and the presence or absence of obstacles. For example, Patent Document 1 describes a road surface condition detection device that calculates parallax information between corresponding points in stereo images captured by the stereo cameras, calculates the distance from the stereo cameras to a detection target point on the road surface based on the calculated parallax information and parameter information of the stereo cameras, and detects road surface conditions such as the height of the road surface and the presence or absence of obstacles on the road surface based on the calculated distance.

[0003] Japanese Patent Application Laid-Open No. 2017-199178

[0004] The road surface condition detection device described in Patent Document 1 has a problem in that in situations where the amount of illumination light is insufficient, such as in the evening or at night, the parallax calculated from the stereo images becomes small, making it difficult to detect obstacles such as bumps, holes, and fallen objects on the road surface from the parallax.

[0005] An object of the present invention is to provide an object detection device that can detect obstacles with high accuracy even under conditions where the amount of illumination light is insufficient, such as in the evening or at night.

[0006] An object detection device according to one aspect of the present invention includes a computing device that detects a predetermined object from each of a plurality of stereo images, generates a disparity image from each of the plurality of stereo images, synthesizes the plurality of disparity images for a predetermined range including the object to generate a single disparity image, and calculates the distance to the object and the height of the object based on the single disparity image.

[0007] According to the present invention, obstacles can be detected with high accuracy even under conditions where the amount of illumination light is insufficient, such as in the evening or at night.

[0008] FIG. 1 is a schematic diagram of a vehicle equipped with an object detection device according to a first embodiment. FIG. 2 is a schematic diagram showing the hardware configuration of the object detection device according to the first embodiment. FIG. 3 is a schematic diagram showing the functional configuration of the object detection device according to the first embodiment. FIG. 4 is a flowchart showing an example of object detection processing executed by the object detection device according to the first embodiment. FIG. 5 is a schematic diagram showing an example of position correction between parallax images. FIG. 6 is a schematic diagram of a vehicle equipped with an object detection device according to a second embodiment. FIG. 7 is a flowchart showing an example of object detection processing executed by the object detection device according to the second embodiment. FIG. 8 is a schematic diagram of a vehicle equipped with an object detection device according to a third embodiment. FIG. 9 is a flowchart showing an example of object detection processing executed by the object detection device according to the third embodiment. FIG. 10 is a schematic diagram of a vehicle equipped with an object detection device according to a first modification.

[0009] First Embodiment An object detection device according to an embodiment of the present invention will be described with reference to FIGS.

[0010] FIG. 1 is a schematic diagram of a vehicle equipped with an object detection device according to the first embodiment. The vehicle 1 is equipped with an imaging device 2 and an object detection device 3. The imaging device 2 is a so-called stereo camera and has a pair of imaging units. The imaging device 2 captures an image of the area in front of the vehicle 1 and outputs a pair of captured images (stereo images) captured by the pair of imaging units to the object detection device 3 in the form of imaging signals. The optical axis 4a of one of the imaging units is located slightly to the left of the center of the vehicle 1 in the horizontal direction. The optical axis 4b of the other imaging unit is located slightly to the right of the center of the vehicle 1 in the horizontal direction. In the following description, the distance between the two optical axes in the horizontal direction is referred to as the baseline length. One imaging unit captures an area 5a in front of the vehicle 1, slightly to the left. The other imaging unit captures an area 5b in front of the vehicle 1, slightly to the right.

[0011] FIG. 2 is a schematic diagram showing the hardware configuration of the object detection device according to the first embodiment. The imaging device 2 includes a first imaging unit 20a and a second imaging unit 20b. In the following description, the first imaging unit 20a and the second imaging unit 20b are collectively referred to as the imaging unit 20. The imaging unit 20 includes an image sensor, such as a complementary metal-oxide-semiconductor (CMOS) image sensor, and an imaging optical system. The first imaging unit 20a and the second imaging unit 20b each capture images of the area ahead of the vehicle 1 and output a pair of imaging signals (stereo image signals) to the object detection device 3. The first imaging unit 20a and the second imaging unit 20b are installed so that the positions of the optical axis centers in the vertical direction (perpendicular direction) are approximately equal. As described above, the first imaging unit 20a and the second imaging unit 20b are positioned horizontally apart by the base line length. Therefore, parallax occurs in the captured images resulting from each imaging operation, depending on the distance from the imaging unit 20 to the subject and the base line length.

[0012] The object detection device 3 is composed of a computer including an arithmetic unit 11 such as a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP), a non-volatile memory 12 such as a read-only memory (ROM), a flash memory, or a hard disk drive, a volatile memory 13 called a random access memory (RAM), an input / output interface 14, and other peripheral circuits. These hardware components work together to run software and realize multiple functions. The object detection device 3 may be composed of a single computer or multiple computers. The arithmetic unit 11 may be an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like.

[0013] The nonvolatile memory 12 stores programs capable of executing various calculations. In other words, the nonvolatile memory 12 is a storage medium (storage device) from which the programs that realize the functions of this embodiment can be read. The volatile memory 13 is a storage medium (storage device) that temporarily stores the results of calculations performed by the calculation device 11 and signals input from the input / output interface 14. The calculation device 11 is a device that loads the programs stored in the nonvolatile memory 12 into the volatile memory 13 and executes calculations, and performs predetermined calculations on data taken in from the input / output interface 14, the nonvolatile memory 12, and the volatile memory 13 in accordance with the programs.

[0014] The input / output interface 14 is connected to the vehicle control device 6 and the imaging unit 20 of the imaging device 2. The input unit of the input / output interface 14 converts signals input from various devices (such as the imaging unit 20) into data that can be calculated by the arithmetic unit 11. The output unit of the input / output interface 14 generates an output signal according to the calculation result of the arithmetic unit 11 and outputs the signal to various devices (such as the vehicle control device 6). The vehicle control device 6 controls, for example, the accelerator, brakes, suspension, etc. of the vehicle 1.

[0015] 3 is a schematic diagram showing the functional configuration of the object detection device according to the first embodiment. The object detection device 3 includes a parallax image generation unit 31, a road surface estimation unit 32, an object detection unit 33, a parallax image synthesis unit 34, an object determination unit 35, and a vehicle control unit 36.

[0016] The parallax image generator 31 generates parallax images from the stereo images output by the imaging device 2 using a known method. For example, the parallax image generator 31 performs rectification processing on the stereo images. The rectification processing is a so-called stereo rectification processing. That is, it is a processing for converting the stereo images into images in which the same point in the world coordinate system is aligned at the same height in both captured images. The rectification processing is realized by performing a projective transformation from a state in which geometric distortion occurs due to the imaging optical system onto a predetermined projection plane, such as a perspective projection plane. The parallax image generator 31 executes a parallax calculation processing to calculate the parallax between the rectified captured images. For each pixel in one captured image, the parallax image generator 31 searches for a pixel at the same point in the other captured image, calculates the difference in horizontal coordinates (i.e., parallax) of the same point in the other captured image based on one captured image, and generates a parallax image. The parallax image generating unit 31 uses a method such as SSD (Sum of Squared Difference) or SAD (Sum of Absolute Difference) to search for the same point.

[0017] The road surface estimation unit 32 estimates the area occupied by the road surface (road surface area) from the stereo images and the parallax images. The road surface is considered to be a relatively flat area with a normal vector pointing nearly vertically upward. Therefore, the road surface estimation unit 32, for example, converts each point included in the parallax images into a point cloud in three-dimensional space and identifies the road surface area by plane fitting the point cloud.

[0018] The object detection unit 33 detects a predetermined object from the road surface area identified by the road surface estimation unit 32 within the entire stereo image by a well-known method such as semantic segmentation using machine learning. The image used for object detection may be one of the stereo images. The predetermined object is, for example, an uneven portion such as a bump, crack, or pothole, which is a structure that exists on the road surface on which the vehicle 1 is traveling and may hinder the traveling of the vehicle 1. Furthermore, objects to be detected may include fallen objects, such as tires or cardboard boxes, that have been dropped onto the road by a preceding vehicle and may hinder the traveling of the vehicle 1. Note that the object detection method is not limited to this, and a technique such as template matching may also be used, for example.

[0019] The parallax image synthesis unit 34 synthesizes a plurality of parallax images generated by the parallax image generation unit 31 from a plurality of stereo images captured by the imaging device 2 at different times, for an area including an object detected by the object detection unit 33, to generate a single parallax image. Since the plurality of parallax images are based on a plurality of stereo images captured at different times, the relative position of the object of interest differs in each parallax image. The parallax image synthesis unit 34 aligns the position of the detected object in the plurality of parallax images to be synthesized, and generates a single parallax image.

[0020] The object determination unit 35 calculates the distance to the object detected by the object detection unit 33 and the height of the object from the single parallax image synthesized by the parallax image synthesis unit 34. Based on the calculated distance to the object and the height of the object, the object determination unit 35 determines whether the object is a predetermined type of obstacle that exists on the road on which the vehicle 1 is traveling and that may impede the traveling of the vehicle 1. The predetermined type of obstacle is, for example, a structure that exists on the road surface on which the vehicle 1 is traveling and may impede the traveling of the vehicle 1, such as a bump, a crack, or a pothole. In addition, objects that have fallen onto the road by a preceding vehicle and that may impede the traveling of the vehicle 1, such as a tire or cardboard box, may also be included in the predetermined type of obstacle.

[0021] When the object detection unit 33 determines that the object detected is a predetermined type of obstacle by the object determination unit 35, the vehicle control unit 36 ​​outputs a predetermined control signal to the vehicle control device 6, thereby causing the vehicle control device 6 to execute predetermined vehicle control for the vehicle 1. The predetermined vehicle control is, for example, control to operate the brakes of the vehicle 1 to reduce the vehicle speed of the vehicle 1, or control to increase the damping force of the suspension of the vehicle 1 to increase the degree to which the suspension absorbs shock. Note that, when the object is an object similar to the above-mentioned falling object, the predetermined vehicle control may be, for example, control to warn the driver of the presence of the falling object by voice or on-screen display, or control to automatically steer the vehicle 1 to avoid the falling object.

[0022] Fig. 4 is a flowchart showing an example of object detection processing executed by the object detection device according to the first embodiment. The object detection processing shown in Fig. 4 is repeatedly executed by the arithmetic unit 11 of the object detection device 3, for example, while the vehicle 1 is traveling. In step S100, the parallax image generation unit 31 causes the imaging device 2 to capture stereo images and inputs the stereo images to the object detection device 3. In step S110, the parallax image generation unit 31 generates parallax images from the stereo images input in step S100.

[0023] In step S120, the road surface estimation unit 32 estimates a road surface area based on the stereo images input in step S100 and the parallax image generated in step S110. In step S130, the object detection unit 33 detects a predetermined object from the road surface area in one of the stereo images input in step S100.

[0024] In step S140, the object detection unit 33 determines whether or not the detection of the predetermined object in step S130 was successful. If the detection of the predetermined object in step S130 was successful, i.e., if one of the stereo images contains an object that is presumed to be the predetermined object, the process proceeds to step S150. On the other hand, if the detection of the predetermined object in step S130 was unsuccessful, i.e., if no object that is presumed to be the predetermined object was found in one of the stereo images, the process shown in FIG. 4 ends.

[0025] In step S150, the object detection unit 33 determines whether the magnitude of the parallax in the parallax image generated in step S110 is equal to or greater than a certain value. Specifically, all parallax values ​​included in the road surface area of ​​the entire parallax image are summed up, and it is determined whether the sum of the parallax values ​​is equal to or greater than a predetermined threshold. For example, in a dark environment with insufficient illumination, such as in the evening or at night, the parallax values ​​included in the parallax image are small, so the sum of the parallax values ​​here is likely to be less than the predetermined threshold. If the sum of the parallax values ​​is less than the predetermined threshold in step S150, the process proceeds to step S160. On the other hand, if the sum of the parallax values ​​is equal to or greater than the predetermined threshold in step S150, the process proceeds to step S230.

[0026] The processing of steps S160 to S190 is the same as the processing of steps S100 to S130. That is, in step S160, the parallax image generation unit 31 causes the imaging device 2 to capture stereo images and inputs the stereo images to the object detection device 3. In step S170, the parallax image generation unit 31 generates parallax images from the stereo images input in step S160. In step S180, the road surface estimation unit 32 estimates a road surface area based on the stereo images input in step S160 and the parallax image generated in step S170. In step S190, the object detection unit 33 detects a predetermined object from the road surface area in one of the stereo images input in step S160.

[0027] In step S200, the parallax image synthesis unit 34 performs position correction on the parallax images generated in step S110 and the parallax images generated in step S180. Position correction here refers to a process of offsetting the parallax images vertically and horizontally so that identical objects detected in each parallax image are exactly overlapped. The parallax images generated in step S110 and the parallax images generated in step S180 were captured at different times (capture timings) of the stereo images on which they are based. Therefore, if the position of the vehicle 1 has changed between captures, it is highly likely that the relative positions of identical objects in the captured images (parallax images) have also changed. Therefore, the parallax image synthesis unit 34 performs position correction by offsetting the parallax images so that the parallax values ​​of the same point are added together when the parallax images are synthesized.

[0028] In step S210, the parallax image synthesis unit 34 synthesizes the parallax image generated in step S110 and the parallax image generated in step S180 to generate a single parallax image. Synthesizing parallax images is a process of adding parallax values ​​at the same point to generate a new parallax image. Note that in this case, it is sufficient to synthesize parallax images for at least a predetermined range that includes the object (for example, a range of a certain size centered on the position of the object or a range of a road surface area that overlaps between parallax images); it is not necessary to synthesize the entire parallax image.

[0029] In step S220, the object detection unit 33 determines whether the magnitude of the parallax in the single disparity image generated in step S210 is equal to or greater than a certain value. Specifically, the object detection unit 33 sums up all the parallax values ​​in the single disparity image generated in step S210 and determines whether the sum is equal to or greater than a predetermined threshold. If the sum is less than the threshold, the process proceeds to step S160, where new stereo images are captured, new parallax images are generated, and a single disparity image including the new parallax images is synthesized. That is, the computing device 11 of this embodiment repeatedly generates a single disparity image by increasing the number of synthesized parallax images until the disparity in the generated single disparity image is equal to or greater than the threshold. If the sum is equal to or greater than the threshold in step S220, the process proceeds to step S230.

[0030] In step S230, the object determination unit 35 calculates the distance to the detected object and the height of the object based on the single parallax image synthesized in step S210. Note that if step S210 is not executed (if the process proceeds from step S150 to step S230), the parallax images generated in step S110 are treated as a single parallax image. The same applies to subsequent processes.

[0031] In step S240, the object determination unit 35 determines whether the object is a predetermined type of obstacle that is present on the travel path of the vehicle 1 and that may hinder the travel of the vehicle 1, based on the distance to the object and the height of the object calculated in step S230. If it is determined that the object is a predetermined type of obstacle, the process proceeds to step S250. In step S250, the vehicle control unit 36 ​​outputs a predetermined control signal to the vehicle control device 6, causing the vehicle control device 6 to execute predetermined vehicle control for the vehicle 1, and the process of FIG. 4 ends. On the other hand, if it is determined in step S240 that the object is not a predetermined type of obstacle, the process of FIG. 4 ends.

[0032] 5 is a schematic diagram showing an example of position correction between parallax images. The parallax image synthesis unit 34 calculates the distance and direction of movement of the vehicle 1 between the previous capture and the current capture based on the vehicle speed, steering angle, and yaw rate of the vehicle 1. The parallax image synthesis unit 34 estimates an estimated position 45 of the object in the currently generated parallax image 42 by applying the calculated distance and direction of movement of the vehicle 1 to a position 43 of the object detected in the previously generated parallax image 41. If the position 44 and the estimated position 45 of the object detected in the currently generated parallax image 42 are sufficiently close to each other, the parallax image synthesis unit 34 determines that they are the same object and performs position correction so that the positions 43 and 44 overlap.

[0033] According to the above-described first embodiment, the following advantageous effects are achieved.

[0034] (1) The computing device 11 detects a predetermined object from each of a plurality of stereo images, generates a disparity image from each of the plurality of stereo images, synthesizes the plurality of disparity images for a predetermined range including the detected object to generate a single disparity image, and calculates the distance to the object and the height of the object based on the single disparity image. Since this configuration makes it possible to calculate the distance to the object and the height of the object with high accuracy even in a situation where the amount of illumination light is insufficient, such as in the evening or at night, it becomes possible to detect an obstacle with high accuracy based on this highly accurate information.

[0035] (2) Based on the distance to the object and the height of the object, the calculation device 11 determines whether the object is a predetermined type of obstacle that exists on the travel path of the vehicle 1 and that may impede the travel of the vehicle 1. In this way, it becomes possible to detect obstacles with high accuracy even in situations where the amount of illumination light is insufficient, such as in the evening or at night.

[0036] (3) When the computing device 11 determines that the object is an obstacle, it executes predetermined vehicle control on the vehicle 1, such as applying the brakes of the vehicle 1 to reduce the vehicle speed, or increasing the damping force of the suspension of the vehicle 1 to increase the degree to which the suspension absorbs shock. This improves the ride comfort of the vehicle when passing over bumps or potholes. Furthermore, it is possible to stabilize the behavior of the vehicle when passing over bumps or potholes, thereby preventing unexpected vehicle movements.

[0037] (4) The computing device 11 repeatedly generates a single disparity image while increasing the number of disparity images to be synthesized until the disparity value of the single disparity image becomes equal to or greater than a threshold value. This allows the generation of a disparity image with an appropriate disparity value, regardless of the amount of illumination light.

[0038] (5) The computing device 11 generates a plurality of parallax images from a plurality of stereo images captured at different times by the same imaging device 2, and synthesizes the plurality of parallax images to generate a single parallax image. This eliminates the need to increase the number of imaging devices 2 in order to prepare a plurality of parallax images, and improves the accuracy of obstacle detection at low cost.

[0039] Second Embodiment An object detection device according to a second embodiment of the present invention will be described with reference to Figures 6 and 7. Note that the same reference symbols are used to designate components that are the same as or equivalent to those described in the first embodiment, and differences will be mainly described.

[0040] FIG. 6 is a diagram similar to FIG. 1 and is a schematic diagram of a vehicle equipped with an object detection device 103 according to the second embodiment. In addition to the imaging device 2 and the object detection device 103, the vehicle 101 is equipped with headlights 107 and 108. The headlights 107 and 108 are lighting devices capable of emitting illumination light of a predetermined intensity toward a predetermined angle ahead of the vehicle 101. The headlights 107 and 108 are configured to be able to change the irradiation angle and irradiation intensity of the illumination light. The object detection device 103 can freely adjust the irradiation angle and irradiation intensity of the illumination light emitted by the headlights 107 and 108.

[0041] Fig. 7 is a flowchart similar to Fig. 4, showing an example of object detection processing executed by the object detection device 103 according to the second embodiment. In the flowchart of Fig. 7, processing of step S300 is added immediately before processing of step S100 in the flowchart of Fig. 4, and processing of step S310 is added immediately before processing of step S160.

[0042] In step S300, the arithmetic unit 11 randomly selects and sets the illumination angle and illumination intensity of the headlights 107 and 108. Thereafter, in step S100, the imaging device 2 captures an image of a subject illuminated by the illumination light from the headlights 107 and 108, and inputs the stereo image to the object detection device 103.

[0043] In step S310, the arithmetic unit 11 randomly selects and sets the illumination angle and illumination intensity of the headlights 107 and 108. The range of the randomly selected illumination angle and illumination intensity is adjusted so that the object detected in step S130 is illuminated with illumination light. Thereafter, in step S160, the imaging device 2 captures an image of the subject illuminated with illumination light from the headlights 107 and 108, and inputs the stereo image to the object detection device 103.

[0044] Through the above processing, in step S210, the parallax image synthesis unit 34 synthesizes multiple parallax images generated from multiple stereo images captured by the same imaging device 2 at different times under different lighting conditions, thereby generating a single parallax image.

[0045] According to the second embodiment described above, the following advantageous effects are achieved.

[0046] (1) The computing device 11 generates a plurality of parallax images from a plurality of stereo images captured by the same imaging device 2 at different times under different lighting conditions, and synthesizes the plurality of parallax images to generate a single parallax image. This makes it possible to accurately detect obstacles that are not exposed to ambient light or that are difficult to expose to ambient light.

[0047] (2) The computing device 11 changes the lighting state using the headlights 107 and 108. The headlights 107 and 108 are considered to be standard features of a typical vehicle, and therefore the accuracy of obstacle detection can be improved without any additional cost.

[0048] Third Embodiment An object detection device according to a third embodiment of the present invention will be described with reference to Figures 8 and 9. Note that the same reference symbols are used to designate components that are the same as or equivalent to those described in the first embodiment, and differences will be mainly described.

[0049] Fig. 8 is a diagram similar to Fig. 1 and is a schematic diagram of a vehicle equipped with an object detection device 203 according to the third embodiment. The vehicle 201 is equipped with an image capture device 2a, an image capture device 2b, and the object detection device 203. The image capture device 2a and the image capture device 2b have the same configuration as the image capture device 2 according to the first embodiment.

[0050] Fig. 9 is a flowchart similar to Fig. 4 showing an example of object detection processing executed by the object detection device 203 according to the third embodiment. In the flowchart of Fig. 9, step S400 is executed instead of step S100 in the flowchart of Fig. 4, and steps S410 to S420 are executed instead of steps S200 to S210. In addition, step S160 in the flowchart of Fig. 4 is deleted, and immediately after the determination in step S220, step S430 is executed instead of proceeding to step S160.

[0051] In step S400, the parallax image generation unit 31 causes the image capture device 2a and the image capture device 2b to capture stereo images, respectively, and inputs the two stereo images to the object detection device 203. The processes of steps S110 to S130 are performed on the stereo image captured by the image capture device 2a, out of the two stereo images. Also, the processes of steps S170 to S190 are performed on the stereo image captured by the image capture device 2b, out of the two stereo images.

[0052] In step S410, the parallax image composition unit 34 performs position correction between the parallax image (corresponding to the image capture device 2a) generated in step S110 and the parallax image (corresponding to the image capture device 2b) generated in step S180. In this embodiment, the image capture timing is the same, and the mounting positions of the image capture devices 2a and 2b are known in advance, so that the position correction can be performed based on the mounting positions of the image capture devices 2a and 2b.

[0053] In step S420, the parallax image synthesis unit 34 synthesizes the parallax image generated in step S110 and the parallax image generated in step S180 to generate a single parallax image. That is, the arithmetic unit 11 of the present embodiment generates a plurality of parallax images from a plurality of stereo images captured by the plurality of imaging devices 2 a, 2 b at approximately the same time, and synthesizes the plurality of parallax images to generate a single parallax image.

[0054] If it is determined in step S220 that the sum of the disparity values ​​is less than the threshold, the process proceeds to step S430. In step S430, the disparity image generator 31 causes the image capture device 2a and the image capture device 2b to capture stereo images, respectively, and inputs the two stereo images to the object detection device 203. Then, steps S170 to S190, S410, and S420 are sequentially performed on one of the two stereo images. If the disparity value is still insufficient, similar processing is sequentially performed on the other of the remaining two stereo images. Note that if unprocessed stereo images remain, there is no need to capture images in step S430. Furthermore, when handling stereo images captured at different times, the position correction performed in step S410 may be the same as in the first embodiment.

[0055] According to the above-described third embodiment, the following advantageous effects are achieved.

[0056] (1) The computing device 11 generates a plurality of parallax images from a plurality of stereo images captured by the plurality of imaging devices 2 a and 2 b. This facilitates alignment of the parallax images, reducing the amount of calculation required for position correction. Furthermore, it is expected that the accuracy of position correction will also be improved.

[0057] The following modified examples are also within the scope of the present invention, and it is possible to combine the configuration shown in the modified example with the configuration described in the above embodiment, to combine the configurations described in the different embodiments above, or to combine the configurations described in the different modified examples below.

[0058] <Modification 1> Fig. 10 is a diagram similar to Fig. 1 and is a schematic diagram of a vehicle equipped with an object detection device 303 according to Modification 1. The vehicle 301 is equipped with monocular image capture devices 2c, 2d, and 2e that are not stereo cameras, and the object detection device 303.

[0059] The arithmetic device 11 according to the first modification selects multiple pairs of images in different combinations from multiple images captured by the multiple imaging devices 2c, 2d, and 2e. Then, the multiple pairs of images are treated as multiple stereo images, multiple parallax images are generated, and a single parallax image is synthesized. For example, the image captured by the imaging device 2c and the image captured by the imaging device 2d are treated as a first stereo image. Similarly, the image captured by the imaging device 2c and the image captured by the imaging device 2e are treated as a second stereo image. Similarly, the image captured by the imaging device 2d and the image captured by the imaging device 2e are treated as a third stereo image.

[0060] Note that more (four or more) monocular imaging devices may be mounted on the vehicle 301. In this case, the number of combinations of captured images that can be regarded as stereo images increases.

[0061] The above-described first modification provides the following advantageous effects.

[0062] (1) The computing device 11 selects a plurality of pairs of images in different combinations from a plurality of images captured by the plurality of imaging devices 2c, 2d, and 2e, and generates a plurality of parallax images by regarding the plurality of pairs of images as a plurality of stereo images. In this way, it is possible to provide a low-cost object detection device that does not require a stereo camera.

[0063] <Variation 2> In the third embodiment, three or more imaging devices may be provided. In this case, first, parallax images are generated sequentially from stereo images captured by each imaging device, and synthesis is attempted until the total parallax value becomes equal to or greater than a threshold. If synthesis of all the parallax images does not become equal to or greater than the threshold, imaging is performed again, and parallax images are generated sequentially and synthesis is attempted until the total parallax value becomes equal to or greater than the threshold.

[0064] <Modification 3> It is also possible to combine the second embodiment with the third embodiment. That is, the lighting conditions during photography may be changed each time photography is performed at a different time.

[0065] Although the embodiments of the present invention have been described above, the above embodiments merely illustrate some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above embodiments.

[0066] DESCRIPTION OF SYMBOLS 1, 101, 201, 301...vehicle, 2, 2a, 2b, 2c, 2d, 2e...imaging device, 3, 103, 203, 303...object detection device, 6...vehicle control device, 11...arithmetic unit, 12...non-volatile memory, 13...volatile memory, 14...input / output interface, 20...imaging unit, 20a...first imaging unit, 20b...second imaging unit, 31...parallax image generation unit, 32...road surface estimation unit, 33...object detection unit, 34...parallax image synthesis unit, 35...object determination unit, 36...vehicle control unit

Claims

1. An object detection device including a computing device, wherein the computing device: - detects a predetermined object from each of a plurality of stereo images; - generates a disparity image from each of the plurality of stereo images; - synthesizes the plurality of disparity images for a predetermined range including the object to generate a single disparity image; and - calculates a distance to the object and a height of the object based on the single disparity image.

2. The object detection device according to claim 1, wherein the computing device determines whether the object is a predetermined type of obstacle that may exist on a travel path of the vehicle and may obstruct the travel of the vehicle based on the distance to the object and the height of the object.

3. The object detection device according to claim 2, wherein when the computing device determines that the object is the obstacle, the computing device executes predetermined vehicle control on the vehicle.

4. The object detection device according to claim 1, wherein the computing device repeatedly generates the single disparity image while increasing the number of the plurality of disparity images to be synthesized until a disparity value in the single disparity image becomes equal to or greater than a threshold value.

5. The object detection device according to claim 1, wherein the computing device generates the plurality of disparity images from each of the plurality of stereo images captured at different times by the same imaging device.

6. The object detection device according to claim 1, wherein the computing device generates the plurality of disparity images from each of the plurality of stereo images captured at different times under different lighting conditions by the same imaging device.

7. The object detection device according to claim 1, wherein the computing device generates the plurality of disparity images from each of the plurality of stereo images captured by a plurality of imaging devices.

8. In the object detection device according to claim 1, the arithmetic device selects a plurality of pairs of images from a plurality of images respectively captured by a plurality of imaging devices in different combinations, and generates the plurality of disparity images by regarding the plurality of pairs of images as the plurality of stereo images. An object detection device.

Images