Object Distance Detection Device

Abstract

An object distance detecting device according to the present invention includes: three image-capturing units 101-1, 101-2, and 101-3, which perform image-capturing of a same subject; a subject region identifying unit 102 that identifies a region in which a subject is present, on the basis of an image acquired from at least one image-capturing unit; an image selecting unit 103 that selects one baseline direction from a plurality of baseline directions defined by an optional two image-capturing units of the image-capturing units, on the basis of an image of the region identified by the subject region identifying unit 102, and selects an image of the region acquired from each of the two image-capturing units defining the selected baseline direction; and a distance detecting unit 104 that detects a distance to the subject present in the region, on the basis of the images selected by the image selecting unit.

Description

[Technical field]

[0001] The present invention relates to an object distance detection device. [Background technology]

[0002] One technology for measuring the distance to an object in three-dimensional space is stereo image processing, which uses the principles of triangulation to measure distance from images taken by two imaging devices.

[0003] For example, Patent Document 1 is an example of a conventional technology for stereo image processing. The publication states that "distance measurement may be temporarily disabled while the camera is being washed or the wiper is being driven. The present technology has been proposed in consideration of such a situation, and aims to suppress a decrease in the reliability of distance measurement." As a solution to this problem, the publication discloses a technology in which "an information processing device according to one aspect of the present technology is an information processing device that includes a selection unit that selects two imaging units that perform imaging for generating distance information from three or more imaging units that constitute an imaging unit, and a distance detection unit that detects a distance to an observation point based on images captured by the two selected imaging units," and "the selection unit can select the imaging units based on the operation of a configuration that affects the captured images." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-32986 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned stereo image processing, the baseline, which is one side of the triangle that serves as the basis for triangulation, is the line that connects the two imaging units. If any point of a target object on an image captured by one of the two imaging units and the same point of the same object on an image captured by the other imaging unit are parallel to this baseline, the object will only have lines parallel to the baseline. This means that the object will have almost no spatial frequency components parallel to the baseline.

[0006] In this way, when an object has almost no spatial frequency components parallel to the baseline and many spatial frequency components perpendicular to it, it becomes difficult to accurately find corresponding points of the object on the two images, and it becomes impossible to measure the correct 3D position. In other words, an object that has many spatial frequency components perpendicular to the baseline has few corresponding points from which the distance can be measured, and the distance cannot be measured with high accuracy.

[0007] In Patent Document 1, the imaging unit for measuring distance is selected based on the operation of components that affect the captured image, such as when the camera is being washed or the wiper is being driven, and it is not possible to accurately measure the distance to an object that has many spatial frequency components in the image perpendicular to the baseline. [Means for solving the problem]

[0008] In order to achieve the above object, the present invention employs the configuration described in the claims. For example, an object distance detection device of the present invention is an object distance detection device that detects a distance to an object around a vehicle, and is characterized by having at least three imaging units that capture an image of the same object, an object area identification unit that identifies an area in which the object exists based on an image acquired from at least one of the imaging units, an image selection unit that selects one of a plurality of baseline directions defined by any two of the three or more imaging units based on an image of the area identified by the object area identification unit, and selects an image of the area acquired from each of the two imaging units that define the selected baseline direction, and a distance detection unit that detects a distance to an object existing in the area based on the image selected by the image selection unit. Effect of the Invention

[0009] According to the present invention, it is possible to measure the distance with high accuracy regardless of the spatial frequency components of the image of the object. Further features related to the present invention will become apparent from the description of the present specification and the attached drawings. In addition, the problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a basic configuration of an object distance detection device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing the configuration of an object region identifying unit according to the first embodiment. [Diagram 3] FIG. 2 is a diagram showing the configuration of an image selection unit according to the first embodiment. [Figure 4] A diagram showing the principle of a stereo camera. [Figure 5A] FIG. 11 is a diagram showing a basic configuration of an object distance detection device according to a second embodiment. [Figure 5B] FIG. 11 is a diagram showing a basic configuration of an object distance detection device according to a second embodiment. [Figure 6] FIG. 11 is a diagram showing a basic configuration of an object distance detection device according to a third embodiment. [Figure 7] 3 is a diagram showing the positional relationship of an imaging unit used in the object distance detection device according to the present invention; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] <Example 1> Fig. 1 is a diagram showing the basic configuration of an object distance detection device according to Example 1. In Fig. 1, 101-1, 101-2, and 101-3 are image capture units, 102 is an object region specification unit, 103 is an image selection unit, and 104 is a distance detection unit.

[0013] In the object distance detection device shown in FIG. 1, the imaging unit 101-1, the imaging unit 101-2, and the imaging unit 101-3 are configured by appropriately using a lens group such as a focus lens, an iris, a shutter, an imaging element such as a CCD or CMOS, a CDS, an AGC, an AD converter, and the like. Then, the optical image received by the imaging element is photoelectrically converted based on exposure conditions such as the opening degree of the iris, the accumulation time of the sensor, the shutter speed, and the gain amount of the AGC. The obtained image signal is subjected to various camera image processes such as digital gain processing, demosaicing processing, luminance signal and color signal generation processing, and noise correction processing, and is output as a video signal. The video signal from one imaging unit 101-1 is transmitted to the object region identification unit 102, and the video signals from all the imaging units 101-1, 101-2, and 101-3 are transmitted to the image selection unit 103. In this embodiment, there are three imaging units, but a configuration of four or more imaging units may be used.

[0014] An example of the positional relationship between the imaging units 101-1, 101-2, and 101-3 used in the object distance detection device according to this embodiment is shown in Fig. 7. As shown in Fig. 7, the imaging units 101-1, 101-2, and 101-3 according to this embodiment are arranged such that the imaging unit 101-1 is disposed near the center of the front of the vehicle, the imaging unit 101-2 is in a horizontal relationship with the imaging unit 101-1, and the imaging unit 101-3 is in a vertical relationship with the imaging unit 101-1.

[0015] This is one example of the number and positional relationship of the imaging units, and if the number of imaging units is, for example, four, they can be arranged in any positional relationship, such as being located at the vertices of a square. Also, while Fig. 7 shows a case in which the imaging units are arranged on the front of the vehicle, this is not limiting, and it is of course also possible to arrange them on the side or rear of the vehicle, and the same applies to other embodiments.

[0016] The configuration and operation of the object region identification unit 102 that receives a video signal from one imaging unit 101-1 will be described with reference to FIG. 2. In FIG. 2, 201 is an object region determination unit, and 202 is a spatial frequency component calculation unit. The object region determination unit 201 receives a video signal from the imaging unit 101-1, determines in which region of the video signal a desired object, for example, an object such as a vehicle, is located, and outputs the region as data. When there are multiple desired objects such as vehicles in the screen, each of the multiple regions corresponding to each of the objects is output as data. The spatial frequency component calculation unit 202 calculates spatial frequency components for the data of the region output from the object region determination unit 201 using filter processing such as a band pass filter, and outputs the region, the horizontal spatial frequency components of the region, and the vertical spatial frequency components of the region to the image selection unit 103.

[0017] As described above, the object region identifying unit 102 identifies the region where the object exists and calculates the spatial frequency components for each identified region, but does not measure the distance to the object. Therefore, it is not necessary to input a plurality of video signals to the object region identifying unit 102. However, a configuration may be adopted in which a plurality of video signals are input from a plurality of imaging units. In that case, it is possible to expect an effect such as the blind spot region of a certain imaging unit being able to be supplemented by another imaging unit. In this embodiment, as shown in FIG. 7, the imaging unit 101-1 defines the horizontal / vertical directions with the other imaging units 101-2 and 101-3, and therefore the image captured by this imaging unit 101-1 is input to the object region identifying unit 102 as a reference.

[0018] 3, the configuration and operation of image selection unit 103 that receives the video signals from all of imaging units 101-1, 101-2, 101-3 and the spatial frequency components calculated by object region identification unit 102 will be described. In Fig. 3, reference numeral 301 denotes a baseline direction selection unit, and 302 denotes a video signal selection unit. Baseline direction selection unit 301 receives the spatial frequency components for each region calculated by object region identification unit 102, and outputs to video signal selection unit 302 the horizontal direction if there are more horizontal spatial frequency components, or the vertical direction if there are more vertical spatial frequency components, as the baseline direction to be selected for each region.

[0019] If the baseline direction received from the baseline direction selection unit 301 is a horizontal direction, the video signal selection unit 302 selects an imaging unit that defines a baseline direction horizontal to the imaging unit 101-1, and if the received baseline direction is a vertical direction, the video signal selection unit 302 selects an imaging unit that defines a baseline direction vertical to the imaging unit 101-1, for each region identified by the object region identification unit 102. In this embodiment, the imaging device has three components, but even if it has four or more components, if the baseline direction received from the baseline direction selection unit 301 is a horizontal direction, the video signal of any imaging unit that defines a baseline direction horizontal to the imaging unit 101-1 is selected for each region, and if the received baseline direction is vertical, the video signal of any imaging unit that defines a baseline direction vertical to the imaging unit 101-1 is selected for each region.

[0020] In the object distance detection device shown in FIG. 1, the distance detection unit 104 calculates the distance to an object in the video signal from two video signals for each area selected by the image selection unit 103.

[0021] The operation of distance detection unit 104 will be described with reference to Fig. 4. Fig. 4 is a diagram showing the general principle of a stereo camera used to measure the distance to an object in a three-dimensional space. In Fig. 4, 401 is a measurement point, 402 is a lens, 403 is an imaging plane, δ is parallax, Z is a measurement distance (distance from lens 402 to measurement point 401), f is a focal length (distance from imaging plane 403 to lens 402), and b is a base length (length between two imaging elements). The measurement distance Z is calculated by the following formula 1.

[0022]

number

[0023] According to this embodiment, for each region where an object to be recognized exists, the horizontal spatial frequency components and vertical spatial frequency components of the image of the region are calculated. Then, for each region, the image signals of the imaging units of a combination that defines the baseline direction in the direction with the most spatial frequency components are selected. That is, for a region where an object with many horizontal spatial frequency components such as a person or a car exists, the imaging units of a combination that defines the horizontal direction are selected from among the multiple imaging units, and for a region where an object with many vertical spatial frequency components such as a fallen object or a depression on the road exists, the imaging units of a combination that defines the vertical direction are selected from among the multiple imaging units. Therefore, for the object, the imaging units that define the baseline direction that can acquire more corresponding points required for three-dimensional distance measurement are selected, and the distance to the desired object can be measured with high accuracy.

[0024] <Example 2> Next, an object distance detection device according to a second embodiment of the present invention will be described with reference to Fig. 5A and Fig. 5B. Note that the same reference numerals are used to designate parts that have already been described in the first embodiment, and duplicated descriptions will be omitted. The same applies to the third embodiment.

[0025] 5A, the parallax image generating unit 501 receives video signals generated by the three imaging units 101-1, 101-2, and 101-3. Then, the parallax image generating unit 501 calculates the parallax δ represented by the number 1 in the first embodiment as a parallax image generated from the video signals output from the imaging units 101-1 and 101-2, and a parallax image generated from the video signals output from the imaging units 101-1 and 101-3, and outputs the parallax image to the image selecting unit 103.

[0026] In the image selection unit 103, the parallax image output from the parallax image generation unit 501 is input to the video signal selection unit 302, and if the baseline direction received from the baseline direction selection unit 301 is a horizontal direction, the image selection unit 302 selects, for each region, a parallax image generated between the imaging unit that specifies a horizontal baseline direction with respect to the imaging unit 101-1, and if the baseline direction received is a vertical direction, the image selection unit 103 selects, for each region, a parallax image generated between the imaging unit that specifies a vertical baseline direction with respect to the imaging unit 101-1.

[0027] Then, the distance detection unit 104 calculates the distances to the multiple objects in the video signal from the parallax images for each region selected by the image selection unit 103, using the above-mentioned formula 1.

[0028] 5A, the parallax image generating unit 501 is disposed between the three imaging units 101-1, 101-2, and 101-3 and the image selecting unit 103, and generates parallax images based on video signals output from the three imaging units. However, as shown in FIG. 5B, the parallax image generating unit 501 may be disposed between the image selecting unit 103 and the distance detecting unit 104.

[0029] In this case, the parallax image generating unit 501 generates parallax images based on the video signal output from the image selecting unit 103 through the process described in the embodiment 1. Then, the distance detecting unit 104 detects the distance to the object based on the generated parallax images.

[0030] According to this embodiment, for each region of an object to be recognized, a parallax image is selected from the imaging units that defines a baseline direction in a direction with a large number of spatial frequency components according to the horizontal and vertical spatial frequency components of the image of the region. Therefore, since a large number of corresponding points for measuring the distance to the object can be obtained, the obtained parallax image becomes clearer, and the distance to the desired object can be measured with high accuracy.

[0031] <Example 3> Next, an object distance detection device according to a third embodiment of the present invention will be described with reference to Fig. 6. The object distance detection unit according to the third embodiment differs from that of the first embodiment in that it includes a vehicle information acquisition unit 601. The vehicle information acquisition unit 601 includes various sensors such as a vehicle speed sensor that detects the speed of the host vehicle, a steering angle sensor that detects the steering angle of the steering wheel, and a GPS that acquires position information of the host vehicle. The various pieces of information acquired by the vehicle information acquisition unit 601 are output to the object region identification unit 102.

[0032] In this embodiment, the spatial frequency component calculation unit 202 in the object region identification unit 102 weights the values ​​of the horizontal and vertical spatial frequency components calculated for the image according to the vehicle speed, steering angle, etc. For example, when the vehicle is traveling at a low speed and the steering angle is turned by a certain angle or more, it can be determined that the vehicle is traveling through an intersection on a general road. In such a case, the object to be recognized is an object with many spatial frequency components in the horizontal image, such as a pedestrian or another vehicle. For this reason, the spatial frequency components in the horizontal image are multiplied by a certain numerical value so that the spatial frequency components in the vertical image are at a minimum value or the spatial frequency components in the horizontal image are at a maximum value.

[0033] Also, for example, if the vehicle is traveling at high speed and the steering angle is less than a certain angle, it can be determined that the vehicle is traveling on an expressway. In such a case, the object to be recognized is an object with many spatial frequency components in the vertical image, such as a fallen object or a pothole in the road. For this reason, a certain value is multiplied by the spatial frequency components of the vertical image so that the spatial frequency components of the horizontal image are at a minimum value or the spatial frequency components of the vertical image are at a maximum value.

[0034] In this embodiment, the vehicle speed, steering angle, and position information are given as examples of vehicle information, but road information such as intersections and expressways may be included from map information and vehicle position information.

[0035] According to this embodiment, for each area in which an object to be recognized exists, an appropriate video signal from the imaging unit can be selected according to the vehicle control and driving conditions, based on the horizontal and vertical spatial frequency components of the image in that area, as well as vehicle information such as vehicle speed and steering angle, thereby making it possible to accurately measure the distance to the desired object.

[0036] According to the embodiment of the present invention described above, the following advantageous effects are obtained. (1) The object distance detection device is characterized by having at least three imaging units 101-1, 101-2, 101-3 that capture images of the same object, an object area identification unit 102 that identifies the area in which the object exists based on an image acquired from at least one of the imaging units, an image selection unit 103 that selects one of a plurality of baseline directions defined by any two of the imaging units based on the image of the area identified by the object area identification unit 102, and selects an image of the area acquired from each of the two imaging units that define the selected baseline direction, and a distance detection unit 104 that detects the distance to an object existing in the area based on the image selected by the image selection unit 103.

[0037] This allows the combination of imaging units to be optimized based on the properties of the image-capturing object so as to obtain as many corresponding points as possible for three-dimensional distance measurement, thereby achieving distance detection with excellent accuracy.

[0038] (2) The object region identification unit 102 obtains the vertical and horizontal spatial frequency components of the image of the region, and the image selection unit 103 selects the baseline direction based on the obtained vertical and horizontal spatial frequency components. Therefore, for each region in which an object having many horizontal / vertical spatial frequency components exists, it is possible to select a combination of imaging units that can obtain many corresponding points required for three-dimensional distance measurement, thereby achieving distance detection with superior accuracy.

[0039] (3) The apparatus further includes a parallax image generating unit 501 that generates a parallax image of a region from an image selected by the image selecting unit 103, and the distance detecting unit 104 detects the distance to the object based on the parallax image. Alternatively, the apparatus further includes a parallax image generating unit 501 that generates a plurality of parallax images from images acquired from three or more imaging units, and the image selecting unit 103 selects a parallax image generated from images acquired from two imaging units that define the selected baseline direction, and the distance detecting unit 104 detects the distance to the object based on the parallax image selected by the image selecting unit 103. This makes it possible to provide a method for detecting distance from various angles, since distance detection is performed using a known parallax image.

[0040] (4) The vehicle information acquisition unit further includes a vehicle information acquisition unit that acquires at least one of vehicle information, such as vehicle motion state information or position information, and the vehicle information acquisition unit obtains vertical and horizontal spatial frequency components of the image data of the region by weighting them based on the vehicle information, and the image selection unit selects the baseline direction based on the vertical and horizontal spatial frequency components obtained by weighting. Therefore, for example, if it is determined that the vehicle is traveling through an intersection on a general road, adjustments can be made to make it easier to detect the distance to a person or another vehicle, and if it is determined that the vehicle is traveling on a highway, adjustments can be made to make it easier to detect the distance to a fallen object or a pothole on the road, making it possible to perform appropriate distance measurement according to the situation.

[0041] (5) When there are multiple regions in which objects exist, the image selection unit 103 selects a baseline direction for each region. This makes it possible to distinguish between objects and perform appropriate distance measurement even when there are multiple objects to be captured.

[0042] (6) The image selection unit selects the vertical direction as the baseline direction if the vertical spatial frequency components found by the object region identification unit are numerous, and selects the horizontal direction as the baseline direction if the horizontal spatial frequency components are numerous. That is, for an area where an object having many horizontal spatial frequency components exists, the image selection unit selects a combination of imaging units that defines a horizontal baseline direction from among the multiple imaging units, and for an area where an object having many vertical spatial frequency components exists, the image selection unit selects a combination of imaging units that defines a vertical baseline direction from among the multiple imaging units. Therefore, since it is possible to select imaging units that define a baseline direction that can obtain more corresponding points required for three-dimensional distance measurement for an object, it is possible to realize distance detection with higher accuracy.

[0043] The present invention is not limited to the above-mentioned embodiments, and various design changes can be made without departing from the spirit of the present invention described in the claims. For example, the above-mentioned embodiments have been described in detail to aid in understanding the present invention, and the present invention is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0044] 101-1, 101-2, 101-3: imaging units, 102: object region identification unit, 103: image selection unit, 104: distance detection unit, 501: parallax image generation unit, 601: vehicle information acquisition unit

Claims

1. An object distance detection device that detects a distance to an object around a vehicle, At least three imaging units for imaging the same object; an object region identifying unit that identifies a region in which the object exists based on an image acquired from at least one of the imaging units; an image selection unit that selects one of a plurality of baseline directions defined by any two of the three or more imaging units based on an image of the area identified by the object area identification unit, and selects images of the area acquired from each of the two imaging units that define the selected baseline direction; a distance detection unit that detects a distance to the object present in the area based on the image selected by the image selection unit, the object region identification unit determines vertical and horizontal spatial frequency components of the image of the region; the image selection unit, based on the vertical spatial frequency components and the horizontal spatial frequency components determined by the object region identification unit, selects the vertical direction as a baseline direction if there are more vertical spatial frequency components determined by the object region identification unit, and selects the horizontal direction as the baseline direction if there are more horizontal spatial frequency components determined by the object region identification unit; An object distance detection device comprising:

2. An object distance detection device that detects a distance to an object around a vehicle, At least three imaging units for imaging the same object; an object region identifying unit that identifies a region in which the object exists based on an image acquired from at least one of the imaging units; an image selection unit that selects one of a plurality of baseline directions defined by any two of the three or more imaging units based on an image of the area identified by the object area identification unit, and selects images of the area acquired from each of the two imaging units that define the selected baseline direction; a distance detection unit that detects a distance to the object present in the area based on the image selected by the image selection unit; a vehicle information acquisition unit that acquires at least one of vehicle information about the motion state of the vehicle and position information of the vehicle, the object region identification unit determines vertical spatial frequency components and horizontal spatial frequency components of the image of the region by weighting them based on the vehicle information; the image selection unit selects the baseline direction based on the vertical spatial frequency component and the horizontal spatial frequency component obtained by the object region identification unit through weighting based on the vehicle information; An object distance detection device comprising:

3. 3. The object distance detection device according to claim 1, a parallax image generating unit that generates a plurality of parallax images from the images acquired from each of the three or more imaging units, the image selection unit selects the parallax image generated from images acquired from two imaging units that define the selected baseline direction, the distance detection unit detects a distance to the object based on the parallax image selected by the image selection unit; An object distance detection device comprising:

4. 3. The object distance detection device according to claim 1, a parallax image generating unit that generates a parallax image of the area from the image selected by the image selecting unit, the distance detection unit detects a distance to the object based on the parallax image; An object distance detection device comprising:

5. 3. The object distance detection device according to claim 1, When there are a plurality of regions in which an object exists, the image selection unit selects the baseline direction for each of the regions; An object distance detection device comprising:

6. 3. An object distance detection device according to claim 2, the image selection unit selects the vertical direction as the baseline direction if the spatial frequency components in the vertical direction obtained by the object region identification unit are numerous, and selects the horizontal direction as the baseline direction if the spatial frequency components in the horizontal direction are numerous; An object distance detection device comprising:

Images