OBJECT POSITION DETECTION DEVICE, DRIVE CONTROL SYSTEM AND DRIVE CONTROL METHOD

Abstract

Object position detection device comprising the following: a first distance detection unit that detects the distance of an object from images derived from a first image acquisition unit and a second image acquisition unit; a first position detection unit that detects a position of the object based on the distance detected by the first distance detection unit; a processing area setting unit that sets a processing area containing the object detected by the first distance detection unit in one of the images taken by multiple image acquisition units, including the first and second image acquisition units; a second position detection unit that detects the object's position in the processing area; a second distance detection unit that detects a distance of the object based on the object's position detected by the second position detection unit; an orientation detection unit that detects the orientation of the object based on the object's position detected by the second position detection unit; and a determination unit that determines the distance and position of the object in accordance with information from detections performed by the first distance detection unit, the second distance detection unit, the first position detection unit, the second position detection unit, and the orientation detection unit, respectively.

Description

Technical field

[0001] The present invention relates to a device that detects the position of an object. Technical background

[0002] As a technical background to the relevant technical field, patent literature 1 proposes the attempt to limit an increase in the computational effort, which includes operations of the stereo camera to track an object.

[0003] Specifically, patent literature 1 discloses the operation for tracking the object by selecting the object recognition process between 3D recognition and 2D recognition in accordance with the distance derived from the 3D recognition and the size of the object in the image.

[0004] Furthermore, patent literature 2 describes a signal processing device for estimating a positional relationship between a first and a second object, which calculates a three-dimensional target position in a first and a second coordinate system, wherein the first position is determined from stereo images of a stereo camera and the second position from sensor signals of a sensor with lateral, longitudinal and depth information.

[0005] Patent literature 3 also discloses an object distance detection device that uses stereo-based distance determination to improve detection accuracy. Specifically, a stereo distance detection unit uses parallax to detect an object and its distance from two reference images, based on corresponding image elements. Subsequently, another distance detection unit determines the object distance in a previously defined processing region using additional image information.

[0006] An object recognition unit described in patent literature 4 also aims to provide increased sensor accuracy by using stereo processing to perform distance-based evaluation of selected image areas and subsequent object recognition based on grouped image elements. A second object recognition unit simultaneously determines object distances based on previously selected image areas.

[0007] Furthermore, patent literature 5 describes an image acquisition system for object distance measurement using different frame rates. This system comprises several image acquisition units with different frame rates, with at least one unit operating at a lower frame rate and another at a higher frame rate. A moving object is then detected based on the images captured at the higher frame rate, while the distance to the object is determined based on the disparity between the images captured by the multiple image acquisition units. List of prior art patent literature Patent literature 1: Japanese patent disclosure JP 2003 - 61 075 A Patent literature 2: US 2018 / 0 267 142 A1 Patent literature 3: WO 2019 / 058 755 A1 Patent literature 4: EP 3 413 015 A1 Patent literature 5: US 2018 / 0 082 136 A1 Summary of the invention: Technical problem

[0008] Patent literature 1 discloses the operation of tracking the object by selecting the recognition process between 3D recognition and 2D recognition in accordance with results derived from 3D recognition, such that the computational load can be reduced. If the object deviates from the image acquisition angle, there is a risk of reduced tracking accuracy.

[0009] Since it was made in view of a solution to the problem described above, it is an object of the present invention to provide an object position detection device that can accurately detect the object regardless of the viewing angle and the distance of the object. Solution to the problem

[0010] To solve this problem, the present invention provides an object position detection device. The device comprises a first distance detection unit that detects the distance of an object from images derived from a first image acquisition unit and a second image acquisition unit; a first position detection unit that detects the object's position based on the distance detected by the first distance detection unit; a processing area setting unit that sets a processing area containing the object detected by the first distance detection unit in one of the images acquired by multiple image acquisition units, including the first and second image acquisition units; a second position detection unit that detects the object's position within the processing area; and a second distance detection unit.a distance detection unit, based on the object's position detected by the second position detection unit; an orientation detection unit, based on the object's position detected by the second position detection unit; and a determination unit, which determines the object's distance and position in accordance with information from detections performed by the first distance detection unit, the second distance detection unit, the first position detection unit, the second position detection unit, and the orientation detection unit, respectively. Advantageous effects of the invention

[0011] The present invention enables accurate detection of the object regardless of the viewing angle and the distance of the object.

[0012] Further tasks, structures, functions and effects of the present invention will be clarified by explanations of embodiments, which will be described below. Brief description of the drawings Fig. Figure 1 shows a structure of an object position detection device according to a first embodiment of the present invention. Fig. Figure 2 shows exemplary recorded images derived from the object position detection device according to the first embodiment of the present invention. Fig. Figure 3 shows an exemplary recorded image derived from the object position detection device according to the first embodiment of the present invention, together with detection results. Fig. Figure 4 shows another exemplary recorded image derived from the object position detection device according to the first embodiment of the present invention, together with detection results. Fig. Figure 5 represents a processing time sequence of the object position detection device according to the first embodiment of the present invention. Fig. Figure 6 is a flow chart of a processing operation that is carried out in the object position detection device according to the first embodiment of the present invention. Fig. Figure 7A shows an example of an image taken by the object position detection device according to the first embodiment of the present invention at a time t1 while driving. Fig. Figure 7B shows an example of an image taken by the object position detection device according to the first embodiment of the present invention at a time t2 while driving. Fig. Figure 7C shows an example of an image taken by the object position detection device according to the first embodiment of the present invention at a time t3 while driving. Fig. Figure 8 shows exemplary detection and determination results derived from the object position detection device according to the first embodiment of the present invention. Fig. Figure 9 shows a structure of an object position detection device according to a second embodiment of the present invention. Fig. Figure 10 shows a structure of an object position detection device according to a third embodiment of the present invention. Fig. Figure 11 shows a structure of an object position detection device according to a fourth embodiment of the present invention. Fig. Figure 12 shows a structure of an object position detection device according to a fifth embodiment of the present invention. Description of the embodiments

[0013] Embodiments of the present invention are described with reference to the drawings. In the following embodiments, a vehicle traveling ahead is illustrated as an object that is subjected to position detection, such that the position of the vehicle traveling ahead is detected. The present invention is not limited to the one described above, but can be applied to the detection of the position of any object. (First embodiment)

[0014] Fig. Figure 1 shows a structure of an object position detection device according to a first embodiment of the present invention. A code 1 designates the object position detection device according to the embodiment. The object position detection device 1 is installed in the front part of the vehicle to detect objects such as the vehicle ahead, forming part of the system that assists in driving control while following the vehicle ahead.

[0015] The codes 101, 102, and 103 denote image acquisition units. Each of these units has its own image sensor equipped with an optical lens. Each of these units repeatedly captures a single image at predetermined intervals and outputs the captured image. The units 101 and 102 are positioned a predetermined distance apart on the left and right sides, respectively. The object position detection device 1 can calculate the object's distance based on the difference between the images captured by unit 101 and unit 102, i.e., the parallax.

[0016] Fig. Figure 1 shows that components of the object position detection device 1 are arranged in the same housing. It is possible to house the image acquisition units 101 and 102 separately from the other components (indicated by a dashed line 11 of Figure 1). Fig. ( 1 are enclosed) or each of the image acquisition units 101, 102 and 103 are housed in a separate enclosure, such that they are installed individually in the vehicle. In this case, image signals can be transmitted by connecting the respective structures using connecting cables (not shown). The transmission method using the differential transmission path of the LVDS type (low-voltage differential signaling type) is available as the method for transmitting images using the connecting cable.

[0017] The use of color image sensors as the image sensors for the image acquisition units 101, 102, and 103 enables the object position detection device 1 to capture color information from the recorded image. This makes it possible to determine the status of the traffic light or the taillight of the vehicle ahead based on the color information in addition to the luminance information.

[0018] A code 104 designates an image correction unit. The image correction unit 104 receives images from the image acquisition units 101 and 102 and corrects these images using a pre-measured correction value so that all luminances of the images are identical. The image correction unit also corrects image distortion caused by the lens and performs horizontal alignment between the images from the image acquisition units 101 and 102 using the pre-measured correction value.

[0019] The respective correction values ​​are measured in advance during the manufacturing process of the object position detection device 1. Before applying the correction value to each unit of the object position detection device 1, an image of the specific object is captured to obtain a luminance correction value for each pixel such that the luminance of the captured image is uniform, and a geometry correction value for each pixel such that the image is leveled by suppressing lens distortion. The captured values ​​are stored individually for each unit of the object position detection device 1 as correction tables in a non-volatile memory (not shown).

[0020] A code 105 designates a stereo distance detection unit. The stereo distance detection unit 105 detects the distance to an object and classifies the object based on an image input from the image correction unit 104. An example of the distance detection procedure is implemented as follows. The stereo distance detection unit 105 acquires the image from the image correction unit 104 to calculate the parallax. As described above, the image acquisition units 101 and 102 are positioned on the left and right sides at a predetermined distance from each other. Parallax exists between the acquired images. The parallax is then calculated; that is, the stereo image processing is performed.

[0021] Parallax is calculated, for example, by performing the block matching process. Specifically, the stereo distance detection unit 105 searches for an area in the image captured by the image acquisition unit 102 that corresponds to a block area of ​​a given small size, e.g., 8 x 8 pixels, segmented from the specified area of ​​the image captured by the image acquisition unit 101. The stereo distance detection unit 105 shifts the block area of ​​the same size horizontally in the image captured by the image acquisition unit 102 by the number of pixels specified as a search density. The resulting correlation value is evaluated. The stereo distance detection unit 105 sets either 128 pixels as the search area and 2 pixels as the search density, or 32 pixels as the search area and 1 pixel as the search density, such that the computational load and accuracy can be controlled together with the specification of the processing area.If the search density is high, the accuracy of the distance to the object being detected becomes coarse. However, the processing load in the search area is reduced. If the search density is low, the accuracy of the distance to be detected becomes high.

[0022] The positional difference of the block areas, which were matched between image acquisition units 101 and 102 in the captured images, is the parallax, expressed in terms of the number of pixels. The stereo distance detection unit 105 can obtain the distance of the object reflected in the block area in the actual environment using the parallax. In this example, the block area is used as an image element of the object that is supposed to have the detected distance. When the coincidence comparison process is implemented to assess the correlation value, the parallax is defined as the position at which the total sum of the luminance difference in pixels in the block areas to be compared is small.

[0023] It is known that the distance to be detected is derived from each focal length of the lens of the image acquisition units 101 and 102, the baseline length (as the distance between the image acquisition units 101 and 102), the parallax obtained as described above, and the pixel spacing of the image acquisition sensor. In the present invention, the method of calculating the distance is not limited to the one described above. Individual pixels forming the image acquisition sensor can be used, in addition to the block area described above, as the image element that is to possess the detected distance.

[0024] Regarding the object detection method, when adjacent pieces of distance information, each indicating essentially the same distance, are present, the stereo distance detection unit 105 groups such information into a single set. If the size of the group is equal to or greater than a given size, such a group is recognized as the object. Based on the size and shape of the detected group, the stereo distance detection unit 105 detects the vehicle as the object. In a further process, the size and shape of the object are detected by comparison with pre-stored pattern data, which serves as the reference data. This process provides the distance from the carrier vehicle to the preceding object, e.g., the vehicle, with high accuracy. The information is used for collision avoidance, such that the carrier vehicle is slowed down or stopped.The information obtained about the type and distance of the object is output to a position detection unit 106, a processing area setting unit 107, a recognition unit 108, a distance detection unit 110, an orientation detection unit 111 and a determination unit 112, all of which are described later.

[0025] A code 106 designates the position detection unit. The position detection unit 106 detects the object's position based on the result from the stereo distance detection unit 105. The position information includes a center position between the image acquisition units 101 and 102, and left and right positions expressed by the difference from the object's lateral center.

[0026] A code 107 designates the processing area setting unit. Based on the object detection results derived from the stereo distance detection unit 105, the processing area setting unit 107 sets and segments the area containing the object to be subjected to detection and position and distance detection, which is performed in the final stage by the detection unit 108, the position detection unit 109, and the distance detection unit 110, respectively, from images acquired by the image acquisition unit 103.

[0027] A code 108 designates the detection unit. Detection unit 108 detects the object contained within the area segmented by processing area setting unit 107. In this example, detection unit 108 detects the object by performing single-eye processing. Detection unit 108 detects the expected preceding vehicle from the input images. This detection method includes the detection of the object, such as the vehicle, based on the similarity of the light distribution and edge shape information of the image to the pattern data stored as reference data. Detection unit 108 can differentiate the type of preceding vehicle, such as a passenger vehicle and a truck, based on the stored pattern data.

[0028] Entering the type of object detected by the stereo distance detection unit 105 into the recognition unit 108 eliminates the need to compare it with the pattern data to detect the object, resulting in a reduced processing load.

[0029] A code 109 designates the position detection unit. The position detection unit 109 detects the position of the object, which was identified by the recognition unit 108 from the area segmented by the processing area setting unit 107. The position information includes an intermediate point between the image acquisition units 101 and 102 and left and right positions, expressed as the difference from the lateral center of the object. The position detection unit 109 establishes the position of a rear end face of the object and outputs this position to the orientation detection unit 111, which will be described later. The rear end face of the object represents, for example, the rear end face of the vehicle.If the side of the vehicle is included in the captured image after the vehicle has turned, the orientation detection accuracy is improved by making the orientation of the rear front surface detectable in addition to the vehicle's side surface. The method for determining the position of the rear front surface is implemented by detection from the image luminance distribution and the vertical edge shape information. The position detection unit 109 can be configured to optionally detect the object that has only been identified as the vehicle.

[0030] A code 110 designates the distance detection unit. The distance detection unit 110 detects the distance of the object detected by the recognition unit 108. The distance detection unit 110 can determine the distance in accordance with the height and size of the detected object on the screen.

[0031] The distance detection unit 110 can limit the number of sample data to be compared by inputting the distance and size of the object previously detected by the stereo distance detection unit 105. The object distance can be detected by performing the minimum required processing.

[0032] A code 111 designates the orientation detection unit. The orientation detection unit 111 detects the orientation of the object, e.g., a horizontal angular difference Δθ between the vehicle ahead and the carrier vehicle. This angular difference changes in accordance with a change in the yaw angle between the vehicle ahead and the carrier vehicle. The orientation detection unit 111 obtains the gradient by performing a linear approximation of the horizontal distance in the relative plane of the object.

[0033] It is assumed that the relationship between a coordinate X of the rear frontal surface of the object (the preceding vehicle) in the horizontal direction (the transverse direction) and a distance Z (a depth) is expressed by (X, Z), and measurement results expressed by (X1, Z1), (X2, Z2), ... and (X5, Z5) are obtained. The orientation detection unit 111 obtains a regression line (Z = a1xX + a2; a1, a1: constant) by the method of least squares and calculates the angle difference Δθ (= arctan(a1)) from the gradient a1.

[0034] The orientation detection unit 111 receives input from the position detection unit 109 specifying the rear frontal area of ​​the vehicle ahead and detects the orientation specifically for the rear frontal area by removing the vehicle's side surface, thus improving orientation detection accuracy. The orientation detection unit 111 can optionally detect the object that has only been identified as the vehicle.

[0035] A code 112 designates the determination unit. The determination unit 112 receives distance detection results from the stereo distance detection unit 105, position detection results from the position detection unit 106, orientation detection results from the orientation detection unit 111, position detection results from the position detection unit 109, and distance detection results from the distance detection unit 110, performs a determination on the results, and outputs the object detection results outside the object position detection device 1.

[0036] The image acquisition units 101, 102, 103, the image correction unit 104, and the stereo distance detection unit 105 of the object position detection device 1, enclosed by a frame 12 formed by a dashed line, are formed by electronic circuits. The other components of the object position detection device 1 are implemented by software processing using a microcomputer, which is not shown. It is also possible to implement the stereo distance detection unit 105 by software processing.

[0037] Fig. Figure 2 shows exemplary images taken by the object position detection device 1 according to the first embodiment of the present invention. With reference to the drawing, a code 1001 denotes a taken image derived by the image acquisition unit 101 and corrected by the image correction unit 104. A code 1002 denotes a taken image derived by the image acquisition unit 102 and corrected by the image correction unit 104. A code 1003 denotes a taken image derived by the image acquisition unit 103. A code 202 denotes the vehicle ahead as the object. A code 203 denotes a pedestrian as the object. A code 204 denotes a roadside tree as the object. A code 205 denotes a traffic light as the object.

[0038] Codes 201 and 209 in the drawing generally denote the common image acquisition areas of images 1001 and 1002. The deviation is present in these common image acquisition areas between images 1001 and 1002. The object's distance is calculated based on the magnitude of this deviation, i.e., the parallax.

[0039] Image acquisition unit 103 is configured to have an image acquisition angle that is substantially the same for each value of image acquisition units 101 and 102. The captured image 1003 has an area corresponding to area 201 of the captured image 1001 enclosed by the dashed line. When the image acquisition sensor for image acquisition unit 103, configured to have pixels larger than those of the image acquisition sensors for image acquisition units 101 and 102, is used, the captured image 1003 derived by image acquisition unit 103 has a higher resolution. This allows for greater accuracy in detecting the object and its distance.

[0040] Fig. Figure 3 shows a captured image derived by the object position detection device 1 according to the first embodiment of the present invention, and exemplary operations of the stereo distance detection unit 105, the position detection unit 106, and the orientation detection unit 111. In the drawing, area 201 is the captured image area common to the image captured by the image acquisition unit 102, which is a part of the captured image derived by the image acquisition unit 101 and corrected by the image correction unit 104.

[0041] A code 301 designates a processing area in which the stereo distance detection unit 105 detects the object's distance and type. In this embodiment, processing area 301 represents the entire area of ​​region 201. The stereo distance detection unit 105 detects the parallax by performing the block matching process described above within processing area 301 and identifies the object from the parallax group.

[0042] Each of the codes 302, 303, 304, and 305 designates a processing area that specifies the object detection result enclosed by the dashed line. Frames and numerical values ​​added to the images are not captured images but are superimposed on the images for explicit indication. In the embodiment, detection results for the respective processing areas are shown. For example, detection results for the vehicle 202 ahead in processing area 302 indicate a distance of 10.6 m, a position of 2.3 m, and an orientation of 0.5°. A detection result for the pedestrian 203 in processing area 303 indicates a distance of 5.2 m. A detection result for the street tree 204 in processing area 304 indicates a distance of 35.3 m. A detection result for the traffic light 205 in processing area 305 indicates a distance of 19.7 m.The object position detection device 1 ensures that the distance, position and orientation of the object are detected across the entirety of the captured image.

[0043] The data of the detected information regarding distance, position and orientation are output to the determination unit 112.

[0044] Fig. Figure 4 shows the captured image 1003, derived by the image acquisition unit 103, and exemplary operations of the processing area setting unit 107, the recognition unit 108, the position detection unit 109, and the distance detection unit 110. The processing area setting unit 107 selects the object information specifying the vehicle from the object information output by the stereo distance detection unit 105, segments the image corresponding to the selected information, and outputs the image to the subsequent recognition unit 108. In this example, the preceding vehicle 202 is present in processing area 301, as shown in Figure 4. Fig. Figure 2 shows that the processing area setting unit 107 selects the information from the preceding vehicle 202 such that the image is segmented and output. A code 401 designates a processing area of ​​the image that is segmented by the processing area setting unit 107 and output to the subsequent recognition unit 108, in such a way as to reduce the load on the subsequent processing. The drawing shows detection results from the position detection unit 109 and the distance detection unit 110; that is, 10.6 m and 2.3 m indicate the distance and position, respectively, of the preceding vehicle 202. The position detection unit 109 and the distance detection unit 110 output the distance and position of the detected object to the determination unit 112.

[0045] Fig. Figure 5 shows the time sequence of the processing carried out by the object position detection device 1 according to the first embodiment of the present invention. Fig. 5A shows each time sequence of the processing performed by the image correction unit 104, the stereo distance detection unit 105, the position detection unit 106 and the orientation detection unit 111. Fig. 5B shows each time sequence of the processing performed by the processing area setting unit 107, the detection unit 108, the position detection unit 109 and the distance detection unit 110. Fig. Figure 5C shows the time sequence of the processing carried out by the determination unit 112.

[0046] With reference to Fig. 5A, distance detection processing is performed by the image correction unit 104 and the stereo distance detection unit 105 to detect the object. The position detection unit 106 detects the object's position, and the orientation detection unit 111 detects the object's orientation.

[0047] With reference to Fig. 5B recognizes in response to the object information that is processed in Fig. As shown in Figure 5A, the detection unit 108 detects the object within the processing area defined by the processing area setting unit 107. The position detection unit 109 detects the object's position, and the distance detection unit 110 detects the distance.

[0048] With reference to Fig. 5C is based on detection results obtained from the processing that is in Fig. 5A and Fig. As shown in Figure 5B, the detection result of the object identified by the determination unit 112 was derived and output.

[0049] Fig. Figure 6 is a flowchart of a processing operation performed by the object position detection device 1 according to the first embodiment of the present invention. The image acquisition devices 101, 102, and 103 acquire images (step S601). Each image acquired by the image acquisition units 101 and 102 is subjected to luminance correction, lens distortion correction, and horizontal alignment, which are performed by the image correction unit 104 (step S602). The stereo distance detection unit 105 detects the object in the processing area 301 and the object's distance (step S603). The position detection unit 106 detects the object's position (step S604), and the orientation detection unit 111 detects its orientation (step S605).

[0050] Based on the detection result of the stereo distance detection unit 105, the processing area setting unit 107 segments the area to be processed from the image captured by the image acquisition unit 103 and outputs the area to the recognition unit 108 (step S606) to detect the object within the area containing the object (step S607). The position detection unit 109 detects the position of the object (step S608) and the distance detection unit 110 detects the distance of the object (step S609).

[0051] Finally, the determination unit 112 performs a determination based on the object's detection results in the respective steps and outputs a determination result (step S610). The object position detection device 1 repeatedly executes the series of processing steps for each frame.

[0052] Fig. Images 7A to 7C are exemplary images taken while driving, derived from the object position detection device 1 according to the first embodiment of the present invention. Fig. Figure 7A shows an image taken by the image acquisition unit 101 at time t1. The captured image indicates the state in which the vehicle 202 ahead fits completely within the image acquisition field of view. Fig. Figure 7B shows an image captured by the image acquisition unit 101 at time t2, which follows time t1. The captured image shows the state of approaching the vehicle 202 ahead, which does not fit completely into the image acquisition field of view while remaining in front. Fig. Figure 7C shows an image taken at time t3, which follows time t2. The captured image indicates the state in which the preceding vehicle 202 turns in such a way that it deviates from the image capture angle.

[0053] Fig. Figure 8 represents time-series detection results of captured images taken while driving, as shown in Fig. 7A, Fig. 7B and Fig. Figure 7C shows the resulting ratio of the detection results that were ultimately used for the determination. Each of the time points t1, t2, and t3 indicates a time at which the images were acquired, as shown in Figure 7C. Fig. 7A to 7B are shown.

[0054] Fig. Figure 8A shows distance detection results. The solid line indicates the result of the distance detection by the stereo distance detection unit 105, and the dashed line indicates the result of the distance detection by the distance detection unit 110. With reference to Fig. 8A denotes the x-axis as time and the y-axis as distance. At approximately time t1, the vehicle 202 ahead fits completely within the image capture field of view on the far side. Distance detection by the stereo distance detection unit 105 reduces the parallax value for the far side to increase the influence of the error. When the baseline length is reduced to make the device more compact, the accuracy of distance detection on the far side, which may contain the detection error, is lowered. Meanwhile, distance detection by the distance detection unit 110 allows the recognition unit 108 to perform tracking using the image and also performs time-series processing, resulting in stabilized detection results.As the vehicle approaches the preceding vehicle 202 at approximately time t2, the stereo distance detection unit 105 can detect the distance with high accuracy due to increased parallax. However, as the preceding vehicle 202 does not fully fit within the image acquisition field of view, the distance detection unit 110 struggles to compare the results with the reference data, leading to an increased detection error. As the vehicle approaches the preceding vehicle 202 further at approximately time t3, the stereo distance detection unit 105 can obtain accurate detection results. Meanwhile, the distance detection error in the distance detection unit 110 is further increased due to the sloping rear face of the vehicle. Ultimately, a comparison with the reference data is not possible, and distance detection fails.

[0055] Fig. Figure 8B shows position detection results. The solid line indicates the result of a position detection by position detection unit 106, and the dashed line indicates the result of a position detection by position detection unit 109. With reference to Fig. In diagram 8B, the x-axis represents time and the y-axis represents position. When the position detection unit 106 detects the object's position from the results of the distance detection by the stereo distance detection unit 105, the result at time t1 contains a detection error. As time progresses to times t2 and t3, the position detection result becomes more accurate. Meanwhile, similar to the distance detection unit 110, if the vehicle ahead deviates from the image acquisition angle, the position detection unit 109 increases its detection error to such an extent that the position detection eventually becomes undetectable.

[0056] Fig. Figure 8C shows results of the detection by the orientation detection unit 111. With reference to Fig. In 8C, the x-axis denotes time and the y-axis denotes orientation. In this embodiment, the direction in which the preceding vehicle 202 turns left is defined as a positive direction. The state at the time following time t3 indicates that the preceding vehicle is largely turning.

[0057] Fig. 8D and Fig. 8E indicate the ratio of distance detection results derived from one unit to those derived from the next unit ultimately used by the determining unit 112, or the ratio of position detection results derived from one unit to those derived from the next unit ultimately used by the determining unit 112. With reference to Fig. 8D and Fig. 8E, where the x-axis represents time and the y-axis represents the ratio.

[0058] With reference to Fig. 8D denotes part a below the curve as the detection results derived by the distance detection unit 110 as part of the total used by the determination unit 112, and part b above the curve as the detection results derived by the stereo distance detection unit 105 as the further part used by the determination unit 112. The sum of parts a and b is 1. The determination unit 112 is configured to selectively use the detection result with higher accuracy, while at any given time assigning it a higher priority (a stronger weighting). Around time t1, the determination unit 112 uses the detection result derived by the distance detection unit 110, while assigning it more priority.In this embodiment, part a assumes a value of 0.9 and part b assumes a value of 0.1 at approximately time t1. Assuming that the distance as the detection result derived by the distance detection unit 110 is z1 and the distance as the detection result derived by the stereo distance detection unit 105 is z2, the distance z, which is to be selectively used by the determination unit 112 at approximately time t1, is expressed by the equation z = 0.9 x z1 + 0.1 x z2.

[0059] The detection unit 112 uses more detection results derived by the stereo distance detection unit 105 at approximately time t2, and uses only detection results derived by the stereo distance detection unit 105 at approximately time t3. If the object position is located such that it differs from the image acquisition viewpoint, as indicated by the position detection result, the detection unit 112 uses the detection result derived by the stereo distance detection unit 105.

[0060] With reference to Fig. 8E denotes a portion c below the curve representing the detection results derived by position detection unit 109 as part of the total used by determination unit 112, and a portion d above the curve representing the detection results derived by position detection unit 106 as the additional part used by determination unit 112. Around time t1, determination unit 112 uses the detection result derived by position detection unit 109 with greater emphasis. Around time t2, determination unit 112 uses more detection results derived by position detection unit 106. Around time t3, determination unit 112 uses detection results derived solely by position detection unit 106.If the distance detection result shows that the object is adjacent to the position that differs from the image acquisition viewpoint, the determination unit 112 uses the detection results derived from the position detection unit 106.

[0061] If the detection result derived by the orientation detection unit 111 becomes larger than the angle that prevents the distance detection unit 110 and the position detection unit 109 from performing a detection, the determination unit 112 is able to use the distance detection results derived by the stereo distance detection unit 105 and the position detection result derived by the position detection unit 106.

[0062] If the distance detected by the stereo distance detection unit 105 is greater than a given distance, and the position detected by the position detection unit 109 is within a given range, the determination unit 112 can use the detection results derived from the distance detection unit 110 and the position detection unit 106.

[0063] If the distance detected by the stereo distance detection unit 105 is closer to the given distance on the side, and the position detected by the position detection unit 109 is outside the given range, the determination unit 112 can use the detection results derived from the stereo distance detection unit 105 and the position detection unit 109.

[0064] If the orientation of the object detected by the orientation detection unit 111 is greater than a given angle, the determination unit 112 can use detection results derived from the stereo distance detection unit 105 and the position detection unit 109.

[0065] The object position detection device 1 of the embodiment enables accurate continuous detection of the distance, position, and orientation of the preceding vehicle 202, even when the preceding vehicle 202 deviates from the image capture angle. That is, the device can acquire information about the position, distance, and orientation by continuously tracking the object, regardless of the viewing angle or the object's distance. (Second embodiment)

[0066] Fig. Figure 9 shows a structure of an object position detection device according to a second embodiment of the present invention. In this embodiment, the device is configured to automatically follow a preceding vehicle (not shown). Structures that perform the same functions as those shown in Fig. The objects shown in Figure 1 are designated with the same codes, and their precise explanations are omitted. The object position detection device 1 is installed in the vehicle, such as a passenger car. Code 901 in the drawing denotes a vehicle control unit. Data output by the destination unit 112 is input into the vehicle control unit 901.

[0067] The vehicle equipped with object position detection device 1 (the following vehicle while following the vehicle in front) follows the vehicle in front. The object position detection device 1 detects the distance, position, and orientation of the vehicle in front, as described above.

[0068] Based on the detection results derived by the determination unit 112, the vehicle control unit 901 controls other vehicle devices, which are not shown. A steering angle, a brake, a steering device, and the like are the targets to be controlled. Based on the detected results, the control unit controls these targets so that the vehicle moves by following the vehicle ahead. The vehicle control information is output by the object position detection device 1 to other devices, which are not shown, via an internal vehicle network such as CAN (control unit area network).Even if the following distance from the vehicle ahead is set to be short enough to fit within the image capture field of view, or if the vehicle ahead turns in such a way that it deviates from the image capture field of view, the object position detection device 1 of the embodiment can accurately and continuously detect the distance, position, and orientation of the vehicle ahead. Therefore, it is possible to follow the vehicle ahead safely.

[0069] Fig. Figure 9 shows an example in which the vehicle control unit 901 is housed together with the object position detection device 1. However, the structure is not limited to the one described above. The vehicle control unit can be housed separately in another housing in such a way that detection results are transmitted via the vehicle's internal network. Alternatively, the image acquisition units 101, 102, and 103 can be housed individually in their respective housings. (Third embodiment)

[0070] Fig. Figure 10 shows a structure of an object position detection device according to a third embodiment of the present invention. Structures that have the same functions as those described in Figure 10. Fig. Figure 1 shows the components with the same codes, and their detailed explanations are omitted. In this embodiment, the image acquisition units 101 and 102 are replaced simply by eliminating the image acquisition unit 103, which is shown in Figure 1. Fig. As shown in Figure 1, the image captured by the image acquisition unit 101 is input to the detection unit 108 via the processing area setting unit 107. The distance detection unit 110 also performs distance detection by means of stereo processing in the same way as the stereo distance detection unit 105. Images derived from the image acquisition units 101 and 102, which have been corrected by the image correction unit 104, are input to the processing area setting unit 107. After the detection results of the stereo distance detection unit 105 have been acquired, the processing area is set and input to the distance detection unit 110. The distance detection unit 110 performs distance detection with respect to the processing area set by the processing area setting unit 107 by means of stereo processing.The processing can be implemented through electronic circuitry or software. The distance detection unit 110 enables the stereo distance detection unit 105 to perform time-division multiplexing.

[0071] In this embodiment, the two image acquisition units 101 and 102 are enabled to perform stereo distance detection and single-eye distance detection, resulting in a reduction in the device's cost. If one of these image acquisition units is damaged to the point of failing to acquire an image, the other image acquisition unit can perform distance and position detection to maintain safety. (Fourth embodiment)

[0072] Fig. Figure 11 shows a structure of an object position detection device according to a fourth embodiment of the present invention. Structures that have the same functions as those described in Fig. The components shown in Figure 1 are designated with the same codes, and their detailed explanations are omitted. In this embodiment, the input information for the processing area setting unit 107 is derived from sensors providing distance and position information, which are generated by components other than the image acquisition units 101, 102, and 103. The input information is derived from a radar (not shown) and a sensor, such as an infrared sensor, such that the distance and position of the object lie within the target area. When the radar is used, the processing area can be controlled by the processing area setting unit 107 by narrowing the radar beam direction toward the object to be processed. Further operations are the same as those described above. (Fifth embodiment)

[0073] Fig. Figure 12 shows a structure of an object position detection device according to a fifth embodiment of the present invention. Structures that have the same functions as those described in Figure 12. Fig. The devices shown in Figure 1 are designated with the same codes, and their precise explanations are omitted. Code 1201 designates a network image capture section. Code 1203 designates a LAN (local area network). Code 1204 designates a control section. The network image capture section 1201 is, for example, installed on the vehicle windshield. The control section 1204, housed in a separate enclosure from the network image capture section 1201, is located at the point that ensures the safety of the vehicle interior. A network as defined by IEEE 802.3 is applicable as LAN 1203.

[0074] The network image acquisition section 1201 is connected to the control section 1204 via LAN 1203. Code 1202 designates an image compression / interface unit. Code 1205 designates a network interface unit. Code 1206 designates an image decompression unit.

[0075] Images captured by the image acquisition units 101, 102, and 103 undergo luminance correction, lens distortion correction, and horizontal alignment in the image correction unit 104. The image compression / interface unit 1202 compresses the image from the image correction unit 104, and the compressed image is sent to the control unit 1204 via the LAN 1203. Image compression is performed to reduce processing time by using the in-display compression process for compression within a single image without using time correlation of multiple images. Alternatively, the compression process can be selected between the in-display compression process and the video compression / encoding process.

[0076] The image compression / interface unit 1202 generates compression / encoding data and transmits the generated data in accordance with a defined network protocol. The image correction unit 104 can be located in the stage following the image decompression unit 1206 of the control section 1204. However, image compression following lens distortion correction in the preceding stage of the image compression / interface unit 1202 of the network image acquisition section 1201 is intended to achieve higher image compression efficiency and higher image quality. In this case, the processing range set by the processing range setting unit 107 is transmitted via the LAN 1203 from the network interface unit 1205 to the image compression / interface unit 1202 and the image correction unit 104.

[0077] The network interface unit 1205 of the control section 1204 receives the compressed image data via the LAN 1203. In the image decompression unit 1206, the compressed image data received by the network interface unit 1205 of the control section 1204 is decompressed back to the original image, which has the processing area set by the processing area setting unit 107, such that the distance is detected by the stereo distance detection unit 105. The subsequent processing is carried out in the same way as described above.

[0078] In this embodiment, the compressed image is transmitted via the LAN 1203 in such a way that the processing power required by the image acquisition units can be reduced. Weight reduction, low power consumption, and a compact housing on the image acquisition unit side mitigate limitations on the installation space required in the vehicle. Sufficient network bandwidth allows for image transmission without the need for compression / decompression.

[0079] The present invention incorporates various modifications, without being limited to the embodiments described above. For example, the embodiments, which are not necessarily limited to the one equipped with all the structures described above, are described in detail for ease of understanding. It is possible to replace a part of the structure of one embodiment with the structure of another embodiment. One embodiment can be provided with an additional structure of another embodiment. It is also possible to add, remove, or replace the additional structure with a part of the structure of the respective embodiments.

[0080] The respective structures can be implemented by hardware or the processor that executes the program. Only the control and data lines deemed necessary are shown. They do not necessarily represent all control and data lines for the product. In fact, it can be assumed that almost all components are interconnected.

[0081] Embodiments of the present invention can be implemented as described below.

[0082] (1) An object position detection device is configured to detect the position of an object from images of the object acquired by multiple image acquisition units. The multiple image acquisition units include a first and a second image acquisition unit. Assuming that the image derived by the first image acquisition unit is defined as a standard image and the image derived by the second image acquisition unit is defined as a reference image, the object position detection device includes a first distance detection unit that detects the distance of the object based on parallax obtained by searching for an image element in the reference image that corresponds to an image element in the standard image, and a first position detection unit that detects the position of the object based on the distance detected by the first distance detection unit.a processing area setting unit that sets a processing area containing the object detected by the first distance detection unit in one of the images acquired by multiple image acquisition units; a second position detection unit that detects a position of the object in the processing area; a second distance detection unit that detects a distance of the object based on the position of the object detected by the second position detection unit; an orientation detection unit that detects an orientation of the object based on the position of the object detected by the second position detection unit; and a determination unit that determines the distance and position of the object in accordance with the distance, position, and orientation determined by the first distance detection unit, the second distance detection unit,The first position detection unit, the second position detection unit, or the orientation detection unit were detected.

[0083] (2) In the object position detection device according to the one described in (1) above, the second position detection unit detects a predetermined part of the object being tracked, and the orientation detection unit detects an orientation of a position of a rear end face of the object, which is determined from the predetermined part detected by the second position detection unit. Reference symbol list 1 object position detection device, 101 to 103 Image acquisition unit, 104 image correction units, 105 Stereo Distance Detection Unit, 106 Position detection unit, 107 Processing area setting unit, 108 Recognition unit, 109 Position detection unit, 110 Distance Detection Unit, 111 Orientation detection unit, 112 Unit of determination, 201 Common image capture area, Objects 202 to 205 301 to 305 Processing area, 401 Processing area, 901 Vehicle control unit, 1201 Network image acquisition section, 1202 Image compression / interface unit, 1203 LAN 1204 Tax Section, 1205 Network interface unit, 1206 Image decompression unit

Claims

[1] Object position detection device comprising the following: a first distance detection unit that detects the distance of an object from images derived from a first image acquisition unit and a second image acquisition unit; a first position detection unit that detects a position of the object based on the distance detected by the first distance detection unit; a processing area setting unit that sets a processing area containing the object detected by the first distance detection unit in one of the images taken by multiple image acquisition units, including the first and second image acquisition units; a second position detection unit that detects the object's position in the processing area; a second distance detection unit that detects a distance of the object based on the object's position detected by the second position detection unit; an orientation detection unit that detects the orientation of the object based on the object's position detected by the second position detection unit; and a determination unit that determines the distance and position of the object in accordance with information from detections performed by the first distance detection unit, the second distance detection unit, the first position detection unit, the second position detection unit, and the orientation detection unit, respectively. [2] Object position detection device according to claim 1, wherein the second position detection unit detects a predetermined part of the object being tracked; and The orientation detection unit detects an orientation of a position of a rear end face of the object, which was determined from the predefined part that was detected by the second position detection unit. [3] Object position detection device according to claim 1 or 2, wherein, when the distance detected by the first distance detection unit is further than a predetermined distance on one side and the position detected by the first position detection unit is within a predetermined area, the determining unit uses detection results derived from the second distance detection unit and the second position detection unit. [4] Object position detection device according to claim 1 or 2, wherein, when the distance detected by the first distance detection unit is closer than a predetermined distance on one side, and the position detected by the first position detection unit is outside a predetermined area, the determining unit uses detection results derived from the first distance detection unit and the first position detection unit. [5] Object position detection device according to claim 1 or 2, wherein, when the orientation of the object detected by the orientation detection unit is greater than a predetermined angle, the determination unit uses detection results derived from the first distance detection unit and the first position detection unit. [6] Driving control system that controls the driving of a following vehicle that is following a preceding vehicle, wherein the following vehicle contains a position detection device that detects the position of the vehicle ahead from images of the vehicle ahead, which were captured by several image acquisition units, each serving to capture a forward-facing image; the multiple image acquisition units include a first image acquisition unit and a second image acquisition unit and The position detection device comprises a first distance detection unit for detecting a distance of the vehicle ahead based on parallax obtained by searching for an image element in a reference image defined as being derived from the second image acquisition unit, wherein the image element corresponds to an image element in a standard image defined as being derived from the first image acquisition unit; a first position detection unit that detects a position of the vehicle ahead based on the distance detected by the first distance detection unit; and a processing area setting unit that sets a processing area containing the vehicle ahead detected by the first distance detection unit in one of the images derived from the multiple image acquisition units.a second position detection unit that detects the position of the vehicle ahead within the processing area, a second distance detection unit that detects the distance of the vehicle ahead based on the position of the vehicle ahead detected by the second position detection unit, an orientation detection unit that detects the orientation of the vehicle ahead based on the position of the vehicle ahead detected by the second position detection unit, a determination unit that determines the distance and position of the vehicle ahead in accordance with the distance, position and orientation detected by the first distance detection unit, the second distance detection unit, the first position detection unit, the second position detection unit and the orientation detection unit respectively,and a vehicle control unit that controls the following vehicle based on a determination made by the determination unit. [7] Driving control procedure for controlling the driving of a following vehicle which is following a preceding vehicle, the procedure comprising: a step of detecting a first distance of the preceding vehicle based on a parallax obtained by searching for an image element in a reference image defined such that it is derived from the second image acquisition unit, wherein the image element corresponds to an image element in a standard image defined such that it is derived from the first image acquisition unit, and the first image acquisition unit and the second image acquisition unit each form multiple image acquisition units for capturing a forward-looking image of the following vehicle; a step of detecting an initial position of the vehicle ahead based on the initial distance detected by the initial distance detection step; a step of setting a processing area that includes the vehicle ahead, which was detected by the step of detecting the first distance in one of the images derived from the multiple image acquisition units; a step of detecting a second position of the vehicle ahead in the processing area; a step of detecting a second distance of the vehicle ahead based on the second position; a step of detecting an orientation of the vehicle ahead based on the second position; a step of determining a distance and a position of the vehicle ahead in accordance with the first distance, the second distance, the first position, the second position and the orientation; and a step of steering the following vehicle based on a determination made by the determination step.

Images