External recognition device and external recognition method

Abstract

This external recognition device comprises: a distance-measurement-information measurement unit that measures distance measurement information on a target; a movement path estimation unit that estimates a movement path of the target on the basis of distance measurement information measured at a reference time and distance measurement information measured at a time prior to the reference time; a distance-measurement-information correction unit that corrects past distance measurement information on the basis of a difference between distance measurement information estimated at the reference time on the basis of the past distance measurement information and actual distance measurement information measured at the reference time with higher measurement accuracy than the past distance measurement information; and a movement path adjustment unit that, on the basis of the corrected distance measurement information, adjusts the movement path of the target estimated on the basis of the past distance measurement information.

Description

External world recognition device and external world recognition method

[0001] The present invention relates to an external environment recognition device and an external environment recognition method.

[0002] Conventionally, there is a method for measuring the distance from the camera's installation position to a target captured in an image by stereo processing images captured by multiple cameras. However, when cameras are installed on a vehicle or other device, the accuracy of the sensors installed in the cameras may decrease, or the surrounding environment of the cameras may change. For example, the accuracy of a camera's near and central field of view may be higher than the accuracy of its wide-angle and far field of view. In addition, one camera may not be able to capture a good image of the field of view that includes the target due to factors such as lighting conditions, occlusion, and cut-off. In such a situation, it is not possible to perform high-precision distance measurement to calculate the target's accurate movement path when the target is recognized.

[0003] Patent document 1 states that "by utilizing multiple cameras installed on a vehicle, obstacles are detected and distances are measured using stereo vision in the common field of view area, and the distances measured in the stereo vision area are also utilized in the monocular field of view area."

[0004] Japanese Patent Application Laid-Open No. 2019-106026

[0005] The technology disclosed in Patent Document 1 uses multiple cameras to detect and measure distances to obstacles using stereo vision in a common field of view area, and also uses the distance measured in the stereo vision area in a monocular vision area. With this technology, when distance measurement using stereo vision cannot be used, a distance measurement method using monocular vision is used.

[0006] However, with conventional technology, even when an obstacle in an image moves from a monocular area where distance is measured with low accuracy to a stereo area where distance is measured with high accuracy, past measurement data that does not include distance information is used without correction. Therefore, when conventional appearance recognition devices use unstable distance measurement information, there is a risk that they will calculate a movement path that may lead to vehicle malfunctions (unnecessary alerts, unnecessary braking, etc.).

[0007] The present invention has been made in view of the above circumstances, and has as its object to accurately determine the movement path of a target.

[0008] The external environment recognition device of the present invention comprises a ranging information measurement unit that measures ranging information of a target using information acquired from a sensor; a movement path estimation unit that estimates the movement path of the target based on ranging information measured at a reference time and ranging information measured at a time earlier than the reference time; a ranging information correction unit that corrects past ranging information based on the difference between estimated ranging information at the reference time estimated based on past ranging information and actual ranging information at the reference time measured with higher measurement accuracy than the past ranging information; and a movement path adjustment unit that adjusts the movement path of the target estimated based on the past ranging information based on the corrected ranging information.

[0009] According to the present invention, the movement path of a target can be accurately determined.

[0010] FIG. 1 is a diagram for explaining a first problem in the past. FIG. 2 is a diagram showing a conventional prediction result of a movement path of a target. FIG. 3 is a block diagram showing an example of the internal configuration of an external environment recognition device according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of a correction amount of a movement path according to an embodiment of the present invention. FIG. 5 is a diagram showing the movement direction of a vehicle equipped with a conventional external environment recognition device and a camera, and the movement direction of a pedestrian who is a target. FIG. 6 is a diagram showing a second problem explaining a situation in which the parallax of an object target cannot be obtained according to an embodiment of the present invention, and a means for solving the problem. FIG. 7 is a diagram showing a third problem explaining a situation in which the measurement accuracy of a sensor that detects a target changes according to an embodiment of the present invention, and a means for solving the problem. FIG. 8 is a flowchart showing an example of the processing of a movement path estimation unit according to an embodiment of the present invention. FIG. 9 is a flowchart showing an example of the processing of a ranging information correction unit according to an embodiment of the present invention. FIG. 10 is a block diagram showing an example of the hardware configuration of a calculation device according to an embodiment of the present invention.

[0011] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted. The present invention is applicable to, for example, a computing device for vehicle control capable of communicating with an on-board ECU (Electronic Control Unit) for an Advanced Driver Assistance System (ADAS) or Autonomous Driving (AD).

[0012] [Problems with the Conventional Art] First, a first problem that occurs when a conventional external environment recognition device measures the distance to a pedestrian from an image captured by a camera will be described with reference to FIG.

[0013] FIG. 1 is a diagram illustrating the first problem of the related art. A left camera 121L and a right camera 121R are attached to the front of a vehicle 100. The left camera 121L and the right camera 121R are attached at different positions but facing the same direction, so that their field of view 122L and 122R partially overlap. The left camera 121L captures images of targets included in the field of view 122L, and the right camera 121R captures images of targets included in the field of view 122R. Examples of targets include a crosswalk at an intersection 155, a boundary between a roadway and a sidewalk, and a pedestrian 150. When the left camera 121L and the right camera 121R are not distinguished from each other, they may be collectively referred to as cameras.

[0014] 1 shows examples of images captured by the left camera 121L and the right camera 121R as time passes in the order of time (t-2), time (t-1), and time (t). Time (t) is an example of a reference time. Time (t-2) and time (t-1) are examples of times earlier than the reference time.

[0015] The external environment recognition device 10 predicts, as a movement path, the direction and position of entry of the pedestrian 150 when the vehicle 100 enters the intersection 155 after time (t), based on the position of the pedestrian 150 calculated in chronological order at time (t-2), time (t-1), and time (t).

[0016] However, at time (t-2) and time (t-1), due to the influence of the lighting environment or occlusion, right camera 121R is unable to capture an image of the target in a state in which the target can be identified. Therefore, the target is not captured in right camera image 123R captured at time (t-2) and time (t-1). On the other hand, the target is captured in left camera image 123L captured by left camera 121L at all times. Pedestrian 150 captured in left camera image 123L moves from left to right in the image as time passes.

[0017] Since conventional external environment recognition devices cannot perform stereo matching of the target between time (t-2) and time (t-1), monocular processing is performed using only the left camera image 123L in which the target is captured. In monocular processing, only the target captured in the left camera image 123L is recognized in the recognition frame 151. Other recognition frames (not shown) are also recognized from the left camera image 123L. After that, when the external environment recognition device identifies the target included in the recognition frame 151 as a pedestrian 150, the pedestrian 150 is tracked from the left camera image 123L at each time, and the distance from the vehicle 100 to the pedestrian 150 is measured at each time.

[0018] The lower part of FIG. 1 shows positions 131 and 132 of pedestrian 150 measured by the external environment recognition device at time (t-2) and time (t-1), as well as an example of the arrangement of the camera and field of view at each time. As shown in FIG. 5, as vehicle 100 moves forward from time (t-2) to time (t-1), the distance from vehicle 100 to pedestrian 150 decreases, and position 132, which is closer to the camera than position 131 of pedestrian 150 calculated at time (t-2), is measured. Both positions 131 and 132 of pedestrian 150 are positions measured using monocular processing. The legend in the figure states that left camera 121L is considered a monocular camera, and only left camera image 123L captured by left camera 121L is used to recognize the target as pedestrian 150 using monocular processing, and positions 131 and 132 are measured.

[0019] At time (t), the right camera 121R is able to capture a right camera image 123R in which a target object is captured. When the left camera image 123L and the right camera 121R captured at time (t) are input from each camera, the external environment recognition device performs stereo processing. In the stereo processing, the target object captured in the left camera image 123L and the target object captured in the right camera image 123R are recognized in recognition frames 151 and 152, respectively. When the external environment recognition device identifies the target object included in the recognition frames 151 and 152 as a pedestrian 150, the distance from the vehicle 100 to the pedestrian 150 is measured based on the parallax image of the target object. A position 133A of the pedestrian 150 represents the position measured in the stereo processing. The "stereo camera" in the legend in the figure indicates that the left camera 121L and the right camera 121R are considered as stereo cameras, and the target position 133A is measured by stereo processing using the left camera image 123L and the right camera image 123R.

[0020] In general, the distance to a target (e.g., a pedestrian 150) measured using stereo processing is more accurate than that measured using monocular processing. Conventional external environment recognition devices predict the movement path of a target at a future time based on the distance of the target moving over time measured using monocular processing and stereo processing.

[0021] 2 is a diagram showing the conventional prediction results of the movement path of a target. In Fig. 2, the Z axis is the traveling direction of the vehicle 100, and the X axis is the left-right direction of the vehicle 100. The target described here is the pedestrian 150 shown in Fig. 1.

[0022] 2 shows a linear movement path 161 connecting target positions 131 and 132 calculated by monocular processing at time (t-2) and time (t-1). At time (t), target position 133A is measured by stereo processing, which is switched to during the monocular processing. Therefore, a linear movement path 162 is shown that is fitted based on the target positions 131, 132, and 133A. However, if the process for measuring the target positions changes during the process, a movement path 162 that differs from the original movement path 161 is calculated.

[0023] A vehicle control device (not shown) provided in the vehicle 100 predicts, based on the movement path calculated by the external environment recognition device 10, that the vehicle 100 will come into contact with the pedestrian 150 when entering the intersection 155, and performs control such as applying the brakes before the vehicle 100 enters the intersection 155 or notifying the driver of an alert. However, since the movement paths 161 and 162 have different directions, the vehicle control device does not know which of the movement paths 161 and 162 to adopt. Furthermore, when the predicted future positions of the targets indicated by the movement paths 161 and 162 overlap with the future position of the moved vehicle 100, the position of the target cannot be determined.

[0024] If such unstable measurement results are used, a travel path that may lead to a malfunction of the vehicle 100 will be calculated. As a result, the accuracy of the determination result of whether or not the vehicle 100 will come into contact with a target, which is obtained from the travel path, will be low. For example, if the predicted position of the pedestrian 150 when entering the intersection 155 is closer than the actual position, there is a risk of malfunction, such as unnecessary braking or an alert being issued. Conversely, if the predicted position of the pedestrian 150 when entering the intersection 155 is farther away than the actual position, there is a risk of delaying the timing of braking the vehicle 100.

[0025]

[0033] Next, an example of the configuration of an external environment recognition device according to an embodiment of the present invention will be described. Fig. 3 is a block diagram showing an example of the internal configuration of an external environment recognition device 10 according to an embodiment of the present invention.

[0026] The external environment recognition device 10 includes an information acquisition unit 1, a target detection unit 2, a target tracking unit 3, a ranging information measurement unit 4, a movement path estimation unit 5, a ranging information correction unit 6, a movement path adjustment unit 7, an object selection unit 8, and a control determination unit 9.

[0027] The information acquisition unit 1 processes images acquired from a camera attached to the vehicle 100 and adjusts image characteristics for further processing. The camera attached to the vehicle 100 is, for example, a monocular camera or a stereo camera, and is an example of a sensor.

[0028] The image characteristic adjustment process performed by the information acquisition unit 1 includes image resolution adjustment, which changes the image size by reducing or enlarging the input image acquired from the camera. Other image characteristic adjustment processes may include, but are not limited to, image region of interest selection, which cuts out and crops a specific region of the input image from the original input image for further processing, image affine transformation such as rotation, scale, and shear of the input image, and processing to convert the input image into a bird's-eye view image by considering a flat ground as a reference. When the information acquisition unit 1 performs an affine transformation of the input image, a geometric formula or transformation table may be calculated or adjusted in advance.

[0029] Furthermore, the parameters used by the information acquisition unit 1 for adjusting the image resolution and selecting the image region of interest can be controlled based on the current driving environment and driving conditions (such as the moving speed or turning speed of the vehicle 100). When the information acquisition unit 1 acquires at least two images from the stereo camera, it performs stereo matching using the two images to create disparity data, which is a three-dimensional distance image of the scene ahead of the vehicle 100.

[0030] The information acquisition unit 1 may acquire not only images from a camera, but also information obtained from a distance measurement sensor such as a radar (radio detecting and ranging) or a lidar (light detecting and ranging), which are examples of sensors, or information obtained by sensor fusion. From any of the information, the external environment recognition device 10 can recognize a target to be measured and measure the distance to the target.

[0031] The target detection unit 2 has a target detection function of detecting a three-dimensional object from the image acquired by the information acquisition unit 1 and calculating the position of the three-dimensional object. "Target detection" includes, for example, a process of detecting the position of a target object in an image space. Target detection also includes a process of identifying, as the target object, for example, an intersection, a lane, an automobile, a motorcycle, a bicycle, a pedestrian, a pole, etc. Note that a target object is an object that the external environment recognition device 10 particularly pays attention to among multiple targets.

[0032] The target detection unit 2 can use various means to detect targets. For example, the information acquisition unit 1 calculates the distance from the vehicle equipped with the stereo camera to the target based on the parallax between two input images acquired from the stereo camera. The target detection unit 2 can also calculate target regions by grouping regions containing adjacent targets that are close to each other among the targets captured in the image. The target detection unit 2 can also calculate individual object regions using semantic segmentation and panoptic segmentation using deep learning. When an image containing an identification target is input, the target detection unit 2 outputs a score of the likelihood of the target being an identification target or uses a classifier to identify the target. The target detection unit 2 can also perform semantic segmentation and panoptic segmentation on the image containing the identification target to output the type of the identification target.

[0033] The target tracking unit 3 searches for targets detected by the target detection unit 2 at a time prior to the current time in the image acquired by the information acquisition unit 1 at the current time, or from targets detected by the target detection unit 2 at the current time. For example, it determines whether a template of a pedestrian detected before the current time matches the position of a pedestrian predicted before the current time. If the target tracking unit 3 finds the same target, it determines that the target has been detected continuously, and tracks the target.

[0034] The ranging information measurement unit (ranging information measurement unit 4) measures ranging information of the target using information acquired from the sensor. For example, the ranging information measurement unit 4 measures the position in three-dimensional space of the target to be tracked by the target tracking unit 3 as ranging information. The ranging information measurement unit 4 also stores information on a measurement range including the target in a storage unit (such as the RAM 23 or non-volatile storage 25 shown in FIG. 11 , which will be described later). The measurement range information includes, for example, a high-precision measurement range and a low-precision measurement range. If the target is continuously detected by the target detection unit 2, the ranging information measurement unit 4 calculates the speed of the target using, for example, a Kalman filter.

[0035] Note that when the distance measurement information measurement unit 4 obtains multiple distance measurement information with different accuracies and the measurement ranges of the distance measurement information overlap, it measures the distance to the target using the measurement range with the highest accuracy. However, all distance measurement information obtained by the distance measurement information measurement unit 4 is stored in a storage unit (such as the RAM 23 or non-volatile storage 25 shown in FIG. 11 , which will be described later). For example, if a monocular image obtained from a monocular camera is obtained by the information acquisition unit 1, the distance measurement information measurement unit 4 cannot calculate disparity information based on the monocular image alone. Therefore, the distance measurement information measurement unit 4 measures the distance to the target position through geometric calculations using the camera attitude parameters.

[0036] On the other hand, if the information acquisition unit 1 can obtain multiple images from a stereo camera or multiple cameras, it is possible to calculate disparity information based on the multiple images and perform stereo matching. Therefore, when the target object is located in an image area where the disparity information can be linked to the target object, the distance measurement information measurement unit 4 measures the distance to the target object position using the calculated disparity information.

[0037] The distance measurement information measurement unit 4 can also measure the distance to a target object by combining a method for processing monocular images and a method for processing stereo images. For example, when two cameras are used and an image cannot be obtained from one camera temporarily, monocular processing and stereo processing are used in combination. Therefore, the distance measurement information measurement unit (distance measurement information measurement unit 4) measures past distance measurement information using information acquired from at least one sensor at a past time among the multiple sensors, and measures distance measurement information at a reference time using information acquired from the multiple sensors at a reference time. As described above, since the past distance measurement information obtained by monocular processing does not include parallax information, the past distance measurement information is corrected by the downstream distance measurement information correction unit 6. Even when monocular processing and stereo processing are used in combination, the distance measurement information of the target object at a past time is corrected by the downstream distance measurement information correction unit 6 using information acquired from the multiple sensors, and the travel path is adjusted by the travel path adjustment unit 7, thereby improving the robustness of the distance measurement information.

[0038] The movement path estimation unit (movement path estimation unit 5) estimates the movement path of the target based on distance measurement information measured at a reference time and distance measurement information measured at a time earlier than the reference time. For example, the movement path estimation unit 5 estimates the movement path of the target based on multiple distance measurement information measured by the distance measurement information measurement unit 4 at different calculation times. The movement path estimation unit 5 estimates the movement path, for example, using past measurement data points of the target from the time the movement path is estimated. In this process, noise is removed from the measurement data of the past measurement data points, and the movement direction of the target is determined by linear or curve approximation of the measurement data points at multiple calculation times. Then, the movement path estimation unit 5 estimates the movement path of the target using the calculated target speed of the target. As shown in FIG. 4 (described later), the movement path is estimated to be at least the length of the movement direction of the target that intersects the traveling direction of the vehicle 100.

[0039] The ranging information correction unit (ranging information correction unit 6) corrects past ranging information based on the difference between estimated ranging information at a reference time estimated based on past ranging information and actual ranging information at a reference time measured with higher measurement accuracy than the past ranging information. For example, the ranging information correction unit 6 calculates the difference between ranging information Y(t) at time (t) estimated based on ranging information X(t-1) actually measured at time (t-1) while the target object is continuously recognized over time, and ranging information Z(t) measured at time (t). Then, the ranging information correction unit 6 calculates the amount of correction for the ranging information of the target object based on the difference. If the difference is within a predetermined range, the ranging information correction unit 6 determines that correction of the ranging information is unnecessary. Here, the ranging information correction unit (ranging information correction unit 6) corrects the ranging information at the reference time and past ranging information with a correction weight such that the amount of correction for past ranging information decreases as one goes back in time from the reference time.

[0040] Examples of "weighting" include the following calculation formulas (1) to (3): Weighting formula for time (t): Dist_Corr_t1 = Dist_Val1 + (Corr_Val * W1) ... (1) Weighting formula for time (t-1): Dist_Corr_t2 = Dist_Val2 + (Corr_Val * W2) ... (2) Weighting formula for time (t-2): Dist_Corr_t3 = Dist_Val3 + (Corr_Val * W3) ... (3)

[0041] The variables in the above equations (1) to (3) have the following meanings: Corr_Val = correction amount W1, W2, W3 = correction weight values ​​(W1 > W2 > W3) Dist_Val = distance measurement value (distance measurement information) Dist_Corr = corrected distance measurement value (distance measurement information)

[0042] On the other hand, if the difference exceeds a preset range, the ranging information correction unit 6 determines that correction is necessary and corrects the past ranging information. For example, the ranging information correction unit 6 corrects ranging information X(t-1), which is an example of estimated ranging information estimated at time (t-1), to ranging information XA(t-1).

[0043] Note that, as the distance measurement information obtained at the reference time (t) becomes older, the past distance measurement information is corrected using a predetermined correction weight. Therefore, the amount of correction for the past distance measurement information decreases. Generally, camera lenses have greater distortion at the edges than at the center. However, even if the amount of correction for a target captured at the edge is increased to account for lens distortion, it is still impossible to determine whether the target (e.g., pedestrian 150) walked straight across or over a step or other obstacle at a past point in time. Therefore, when correcting the estimated distance measurement information using the actual distance measurement information at time (t), the distance measurement information correction unit 6 reduces the amount of correction as the target's image is captured further back in time, thereby suppressing the influence of past target movement.

[0044] The movement path adjustment unit (movement path adjustment unit 7) adjusts the movement path of the target estimated based on past ranging information based on the corrected ranging information. When past ranging information is corrected, the movement path adjustment unit 7 corrects the movement path and compares it with the previously estimated movement path to determine whether to use the adjusted movement path. When correcting the movement path, the movement path adjustment unit 7 sets a correction flag for the movement path. Furthermore, when recalculating the movement speed of the target, the movement path adjustment unit 7 uses, for example, a Kalman filter. By using the Kalman filter, the movement path adjustment unit 7 can slow down the change in movement speed even if the horizontal movement speed of the target shown in FIG. 1 changes to a movement speed in the direction toward the vehicle 100. Note that the movement path adjustment unit 7 may use a time series filter or the like in addition to the Kalman filter. The correction flag set by the movement path adjustment unit 7 is used to determine that the result is acceptable because it is an adjusted movement path when a result different from the expected result is obtained using a movement path used in subsequent processing.

[0045] The object selection unit 8 has the function of selecting and prioritizing detected targets so that the ECU of the host vehicle can execute appropriate control routines or application programs. The target positions relative to the host vehicle's path and the movement path onto which the target positions are projected are used to select and prioritize targets based on the calculated proximity of the targets to the host vehicle's estimated movement path and current movement path. That is, if a target is deemed to be on a movement path that will result in a collision or near-collision situation with the host vehicle, that target is selected as the target, and if there are other selected targets, those targets are prioritized. The priority of the targets is assigned based on the distance (proximity) to collision and the time to collision calculated for each target. As a result, a higher priority is assigned to targets that are likely to collide with the host vehicle or are close to the host vehicle in terms of both distance and time to collision.

[0046] The control determination unit 9 has a function of determining an alarm routine or a control application to be executed in the vehicle equipped with the external environment recognition device 10 according to the result acquired from the object selection unit 8. When using a travel route adjusted by the travel route adjustment unit 7, the control determination unit 9 further leaves a flag such as "corrected" on the travel route. This flag allows the control application to select how to use and make decisions about the travel route after adjustment. For example, it is possible to change the use of the travel route after adjustment at the control application level.

[0047] 4 is a diagram showing an example of the correction amount of distance measurement information, in which the upper part of Fig. 4 shows a left camera image 123L and a right camera image 123R captured at the time (t) described with reference to Fig. 1 .

[0048] The lower left of Figure 4 shows an example of positions 131-133, 133A of pedestrian 150 at time (t) as described with reference to Figure 1. The distance measurement information measurement unit (distance measurement information measurement unit 4) measures past distance measurement information using information acquired at a past time from a sensor with a low-accuracy measurement range among multiple sensors whose low-accuracy measurement ranges partially overlap with their high-accuracy measurement ranges. The distance measurement information measurement unit (distance measurement information measurement unit 4) also measures distance measurement information at a reference time using information acquired at a reference time from a sensor with a high-accuracy measurement range. For example, a portion of field of view 122L of left camera 121L overlaps a portion of field of view 122R of right camera 121R. The range where field of view 122L and field of view 122R do not overlap is a low-accuracy measurement range measured by monocular processing. The range where field of view 122L and field of view 122R overlap is a high-accuracy measurement range measured by stereo processing. Therefore, the position 133 of the pedestrian 150 at time (t) is obtained by monocular processing, whereas the position 133A of the pedestrian 150 at time (t) is obtained by stereo processing.

[0049] Position 133 obtained by monocular processing is farther from left camera 121L than position 133A obtained by stereo processing. Therefore, a difference d1 between position 133 and position 133A is obtained in the forward direction of left camera 121L and right camera 121R.

[0050] The ranging information correction unit (ranging information correction unit 6) corrects past ranging information based on ranging information at a reference time. For example, the ranging information correction unit 6 corrects the position of pedestrian 150, which is ranging information at past times (t-2) and (t-1), based on the ranging information at time (t) and on difference d1. The corrected position of pedestrian 150 relative to position 133 before correction is shown as position 143 in the lower right of FIG. 4.

[0051] 4 shows movement paths 161 to 163 of the pedestrian 150. Movement path 161 represents the predicted movement direction and destination of the pedestrian 150 calculated using only conventional monocular processing. Movement path 162 represents the predicted movement direction and destination of the pedestrian 150 calculated using conventional monocular processing and stereo processing. The movement direction and destination of the pedestrian 150 shown in movement paths 161 and 162 are both as described with reference to FIG. 2 and are calculated without any correction according to this embodiment. Therefore, it is unclear whether the pedestrian 150 shown on movement paths 161 and 162 will come into contact with the vehicle 100 when the vehicle 100 moves in the z-axis direction.

[0052] The movement path 163 represents the predicted result of the movement direction and destination position of the pedestrian 150 calculated by the correction according to this embodiment. The correction amount of the distance measurement information indicating the position of the pedestrian 150 at each time is set to be largest at time (t), and smaller at time (t-1) and time (t-2) in that order. The external environment recognition device 10 adjusts the movement path 161 or 162 based on the position of the pedestrian 150 corrected using the correction amount at each time.

[0053] For example, positions 131 to 133 of pedestrian 150 calculated using only monocular processing are corrected to positions 141 to 143. Based on positions 141 to 143, movement path 161 or 162 of pedestrian 150 is adjusted. As a result, the position of pedestrian 150 in the traveling direction of vehicle 100 becomes clear.

[0054] 5 is a diagram showing the movement direction of the vehicle 100 equipped with the external environment recognition device 10 and the camera 120, and the movement direction of the pedestrian 150. In FIG. 5, x coordinates and y coordinates are shown.

[0055] Vehicle 100 moves from bottom to top in Figure 5 and enters intersection 155. Pedestrian 150 moves from left to right in Figure 5 and enters intersection 155. Figure 5 shows the actual positions of vehicle 100 and pedestrian 150 at time (t-2), time (t-1), and time (t). The movement amount of vehicle 100 is shown as movement amount 101_1 of vehicle 100 from time (t-2) to time (t-1), and movement amount 101_2 of vehicle 100 from time (t-1) to time (t).

[0056] The vehicle 100 is equipped with an external environment recognition device 10 and a camera 120. The camera 120 is composed of two cameras: a left camera 121L and a right camera 121R. Using the method according to this embodiment, the external environment recognition device 10 can correct past distance measurement information of a target and calculate an accurate movement path of the target based on the corrected distance measurement information. For example, the external environment recognition device 10 recognizes various targets appearing in images captured by the camera 120 and identifies a target representing a pedestrian 150 from these targets. Then, the position of the pedestrian 150 is corrected to obtain a movement path 163 adjusted based on the corrected position. This movement path 163 is more accurate than the movement paths 161 and 162 obtained by conventional methods, thereby improving the accuracy of collision determination between the vehicle 100 and the pedestrian 150. As a result, the vehicle control device (not shown) can determine the timing at which the direction of travel of the vehicle 100 intersects with the direction of travel of the pedestrian 150 based on the travel path 163, and can apply the brakes or output an alert at the appropriate time.

[0057] The external environment recognition device 10 also stores in a predetermined flag information indicating that the past distance measurement information of the target has been corrected and that the travel path has been recalculated. This flag allows a vehicle control device (not shown) to easily select and discard travel paths. Furthermore, the control application of the control determination unit 9 can fine-tune the corrected travel path or distinguish between the results obtained for the travel path before and after the correction, based on the flag information.

[0058] In this way, the external environment recognition device 10 takes into account the field of view of the camera 120, corrects the position of the pedestrian 150 at the timing when the pedestrian 150 enters from a wide-angle or distant low-accuracy measurement range of the field of view range to a high-accuracy measurement range at the center or nearby of the field of view range, and recalculates the movement path 163. Such processing is applicable to, for example, the NCAP (New Car Assessment Program) crossing vehicle (CCCscp (Car-to-Car Crossing Straight Crossing Path)) protocol.

[0059] The movement path adjustment unit 7 determines whether or not to adopt the adjusted movement path based on the difference between the target position at the reference time (t) estimated from the movement path before adjustment (e.g., movement path 162) and the target position at the reference time (t) for which the distance was actually measured. For example, if the difference d1 exceeds a preset range, as in the case of movement paths 164 and 165 shown in FIG. 5 , the movement path adjustment unit 7 determines that the adjusted movement path is unreliable and does not adopt the adjusted movement paths 164 and 165. In this case, the movement path adjustment unit 7 leaves an "unstable" flag for the adjusted movement paths 164 and 165 that were not adopted.

[0060] Conversely, the movement path adjustment unit 7 does not adopt the adjusted movement path even if the difference between the adjusted movement path and the movement path before adjustment (for example, the movement path 162) is within the difference d1. This is because there is almost no difference between the movement path before adjustment and the movement path after adjustment.

[0061] Next, the second and third problems and their solutions will be described with reference to Figures 6 and 7. Figure 6 shows the second problem, which explains a situation in which the parallax of the target object cannot be obtained, and a solution to that problem. In this example, since the target object (e.g., a pedestrian 150) is not captured in one of the camera images, there is a time when the parallax of the target object cannot be calculated.

[0062] 6, as time passes in the order of time (t-2), time (t-1), and time (t), pedestrian 150 in the shadow of a building is seen moving to the right, as captured by left camera image 123L and right camera 121R. Here, the images at each time and the top view showing the distance measurement information of pedestrian 150 are shown separated by dashed lines for each time.

[0063] The ranging information measurement unit (ranging information measurement unit 4) measures past ranging information using information acquired from at least one sensor among the multiple sensors that is capable of recognizing a target, and measures ranging information at a reference time using information acquired from multiple sensors that are capable of recognizing a target. For example, at time (t-2) and time (t-1), the pedestrian 150 is not captured in the left camera image 123L. Therefore, the external environment recognition device 10 recognizes the pedestrian 150 from the target and measures the distance to the pedestrian 150 using monocular processing, targeting the pedestrian 150 captured in the right camera image 123R at time (t-2) and time (t-1). The ranging information of the pedestrian 150 is calculated as positions 131 and 132 in the top view shown at the bottom of FIG. 6.

[0064] At time (t), the pedestrian 150 appears not only in the right camera image 123R but also in the left camera image 121. Therefore, the external environment recognition device 10 obtains parallax information of the pedestrian 150 from the left camera image 123L and the right camera image 123R at time (t). Therefore, the external environment recognition device 10 uses both monocular processing and stereo processing on the image captured at time (t), recognizes that the target is the pedestrian 150, and obtains positions 133, 133A of the pedestrian 150. A difference d2 occurs between the position 133 of the pedestrian 150 measured by monocular processing and the position 133A of the pedestrian 150 measured by stereo processing.

[0065] For this reason, the ranging information corrector (ranging information corrector 6) corrects past ranging information based on ranging information at a reference time. For example, the ranging information corrector 6 corrects positions 131 to 133 at time (t-2), time (t-1), and time (t) to positions 141 to 143, respectively. The movement path adjustor 7 predicts the movement direction and destination of pedestrian 150 shown on movement path 163 based on corrected positions 141 to 143. In this example, too, the movement path 163 is adjusted based on corrected ranging information such that the amount of correction at time (t) is greatest, and the amounts of correction decrease in order at time (t-1) and time (t-2).

[0066] 7 is a diagram illustrating a third problem that explains a situation in which the measurement accuracy of a sensor that detects a target object changes, and a means for solving the problem. In this example, it is assumed that a single stereo camera 125 is mounted on the vehicle 100 as a sensor.

[0067] The stereo camera 125 captures images of targets included in the field of view 126. The stereo camera 125 can capture images at a wide angle, but the measurement accuracy differs depending on the portion of the field of view 126. For example, the target measurement accuracy is higher in the narrow-angle portion 126B of the field of view 126 than in the wide-angle portion 126A of the field of view 126. This is because the resolution of the narrow-angle portion 126B is higher than the resolution of the wide-angle portion 126A. Furthermore, the target measurement accuracy is higher for targets closer to the stereo camera 125 than for targets farther away. This can be attributed to, for example, the specifications of the lenses or sensors constituting the stereo camera 125, changes in calibration accuracy, etc.

[0068] Therefore, the external environment recognition device 10 continues to calculate the position of the pedestrian 150, which is a target that has entered the field of view 126, and corrects the position of the target whose distance was measured at a past time when the pedestrian 150 enters the included angle portion 126B. At this time, the distance measurement information measurement unit (distance measurement information measurement unit 4) measures past distance measurement information using information acquired at a past time in the low-accuracy measurement range of the sensor, out of information acquired from the sensor having a low-accuracy measurement range and a high-accuracy measurement range. Then, the distance measurement information measurement unit (distance measurement information measurement unit 4) measures distance measurement information at a reference time using information acquired at a reference time in the high-accuracy measurement range of the sensor.

[0069] For this reason, the ranging information correction unit (ranging information correction unit 6) corrects past ranging information based on ranging information at a reference time. For example, positions 131 and 132 of pedestrian 150 are determined in order at time (t-2) and time (t-1). Then, at time (t), position 133 (not shown) of pedestrian 150 is determined. In the conventional method, position 133B is predicted based on positions 131 to 133 of pedestrian 150. On the other hand, since pedestrian 150 is within included angle portion 126B, position 133C is actually measured with high accuracy. Predicted position 133B and actual measured position 133C are separated by a difference d3.

[0070] The ranging information correction unit 6 corrects the positions 131 to 133C at time (t-2), time (t-1), and time (t) to positions 141 to 143, respectively. The movement path adjustment unit 7 adjusts the movement path 161 based on the corrected positions 141 to 143, and predicts the movement direction and destination of the pedestrian 150 shown in the adjusted movement path 163. In this example, too, the movement path 163 is adjusted based on the corrected ranging information so that the amount of correction is greatest at time (t), and decreases in order at time (t-1), time (t-2), and so on.

[0071] Note that a monocular camera equipped with a fisheye lens capable of capturing images at a wide angle may be used instead of the stereo camera 125. A fisheye lens has greater distortion at the edges than at the center. In other words, the position of the pedestrian 150 captured at the edge of the fisheye lens is less accurate than the position of the pedestrian 150 captured at the center of the fisheye lens. For this reason, the external environment recognition device 10 can correct the position of the pedestrian 150 when the position of the pedestrian 150 moves from the edge to near the center of the image captured by the fisheye lens, and calculate the movement path based on the corrected position.

[0072] Next, an example of the processing of each functional block of the external environment recognition device 10 will be described with reference to Figures 8 to 10. Figures 8 to 10 show an example of an appearance recognition method performed by each functional block of the external environment recognition device 10.

[0073] 8 is a flowchart showing an example of the process performed by the travel path estimation unit 5. When there are multiple target objects, this process is performed for each target object. The target objects include objects other than the pedestrian 150.

[0074] First, the travel path estimation unit 5 checks whether the number of historical distance measurement information records of the target object is equal to or greater than a preset threshold (S1). If the number of historical distance measurement information records is less than the preset threshold (NO in S1), the travel path estimation unit 5 sets the travel path of the target object as invalid (S4), ends this process, and moves on to the next task (processing by the distance measurement information correction unit 6).

[0075] If the number of historical distance measurement information records is equal to or greater than a preset threshold (YES in S1), the travel path estimation unit 5 calculates the travel direction of the target object using the historical distance measurement information (S2). The travel path estimation unit 5 calculates a model, for example, by linear approximation, and obtains the travel direction of the pedestrian 150 relative to the vehicle 100. This model is, for example, a straight line in a two-dimensional plane of x and y coordinates shown in FIG. 5. If it is a straight line, the slope and intercept are calculated.

[0076] The travel path estimation unit 5 also calculates the error of the ranging information of each history against the calculated model and determines whether any of the errors is an outlier from the model. The travel path estimation unit 5 marks any errors determined to be an outlier with an outlier flag for the past history. Furthermore, the travel path estimation unit 5 checks whether the number of ranging information in valid history that is not an outlier is equal to or greater than a preset threshold. If the travel path estimation unit 5 determines that the condition is met, it sets the moving direction of the target to "reliable." On the other hand, if the travel path estimation unit 5 determines that the condition is not met, it sets the moving direction of the target to "unreliable."

[0077] Next, the movement path estimation unit 5 estimates the movement path of the target (S3), ends this process, and moves on to the next task (processing by the distance measurement information correction unit 6). The movement path estimation unit 5 estimates the movement path of the target using the target speed of the target calculated from the time and movement distance, in addition to the movement direction of the target calculated in step S2. The movement path estimation unit 5 can estimate, for example, the target position in the next processing cycle, the target position several seconds from now (a range requiring an alarm or control), the time until the path of the vehicle and the target intersect [time-to-collision], etc.

[0078] 9 is a flowchart showing an example of the processing performed by the distance measurement information corrector 6. Here, the measurement correction determination processing and correction processing are performed for each target being tracked over several frames.

[0079] First, the ranging information correction unit 6 determines whether the target is being continuously recognized (S11). In step S3 of Fig. 8 described above, the movement path of the target being tracked up to a reference time (e.g., time (t)) is estimated by the movement path estimation unit 5. Therefore, if the number of tracking frames of the detected target is equal to or greater than a preset threshold, the ranging information correction unit 6 determines that the target is being continuously recognized (YES in S11) and proceeds to step S12. On the other hand, if it determines that the target is not being continuously recognized (NO in S11), proceeds to step S16.

[0080] Next, the distance measurement information corrector 6 determines what the plurality of distance measurement information of the target is (S12). There are two methods for determining the plurality of distance measurement information of the target.

[0081] (In the case of ranging information obtained by monocular processing and stereo processing) The ranging information correction unit 6 checks whether there are multiple pieces of ranging information for the target being tracked from a past time (e.g., time (t-2), time (t-1)) to a reference time (e.g., time (t)). For example, there is ranging information obtained by monocular processing using geometric calculations, etc., and ranging information obtained by stereo processing using parallax information, etc. If the ranging information correction unit 6 determines that there are multiple pieces of ranging information for the target (YES in S12), it proceeds to step S13.

[0082] (When the target moves from the low-accuracy measurement range to the high-accuracy measurement range) The ranging information correction unit 6 checks whether a target for which there is multiple ranging information and which has been tracked from a past time to a reference time (e.g., time (t)) has moved from the low-accuracy measurement range to the high-accuracy measurement range. When the target moves from the low-accuracy measurement range to the high-accuracy measurement range, the ranging information correction unit 6 estimates the position of the target from the low-accuracy measurement range to the reference time. Thereafter, the ranging information correction unit 6 proceeds to step S13 with the ranging information estimated in the low-accuracy measurement range and the ranging information obtained in the high-accuracy measurement range.

[0083] If there is not a plurality of pieces of distance measurement information for the target in step S12, or if the target has not moved from the low-accuracy measurement range to the high-accuracy measurement range (NO in S12), the process proceeds to step S16.

[0084] After the YES determination in step S12, the distance measurement information corrector 6 calculates the difference between the estimated position and the actually measured position of the target, and the correction amount for the distance measurement information (S13). In this process, the distance measurement information corrector 6 calculates the correction amount based on the difference between the distance measurement information Y(t) at time (t) estimated using the distance measurement information X(t-1) acquired at least at time (t-1), and the distance measurement information Z(t) measured at time (t).

[0085] Next, the ranging information correction unit 6 determines whether or not the ranging information needs to be corrected (S14). If the difference and correction amount calculated in step S13 are small and within a preset range, the ranging information correction unit 6 determines that correction of the ranging information is not necessary (NO in S14) and proceeds to step S16. On the other hand, if the calculated difference or correction amount is large and exceeds the preset range, the ranging information correction unit 6 determines that correction by a correction amount is necessary (YES in S14) and proceeds to step S15.

[0086] If step S14 returns YES, the ranging information correction unit 6 corrects the ranging information (S15). The ranging information correction unit 6 corrects the ranging information of the tracked target at the reference time and the ranging information at a past time. For example, the ranging information correction unit 6 corrects ranging information X(t-1) to ranging information XA(t-1) using the correction amount. As the ranging information becomes older relative to the reference time, the correction amount decreases according to a predetermined correction weight. Therefore, the ranging information correction unit 6 sets the correction weight so that, for example, up to 10 frames measured in the past can be corrected. Finally, the ranging information correction unit 6 sets a correction-completed flag indicating that the ranging information has been corrected, terminates this processing, and moves on to the next task (processing by the travel path adjustment unit 7).

[0087] If the determination in step S11, S12, or S14 is NO, the ranging information correction unit 6 sets information indicating that the ranging information has not been corrected (S16). The information indicating that the ranging information has not been corrected is, for example, an uncorrected flag. The ranging information correction unit 6 sets the uncorrected flag, ends this processing, and moves on to the next task (processing by the travel path adjustment unit 7).

[0088] Either a correction-completed flag set in step S15 or a correction-unexecuted flag set in step S16 is set for each frame processed and stored in the storage unit. Therefore, in subsequent processing, ranging information in which many correction-completed flags are set in multiple frames (e.g., correction-completed flags are set in four out of five frames) is determined to be less reliable than ranging information in which few correction-completed flags are set in multiple frames (e.g., correction-completed flags are set in only one frame). For this reason, the movement path may be adjusted using ranging information in which few correction-completed flags are set.

[0089] 10 is a flowchart showing an example of the processing of the movement path adjustment unit 7. Here, the processing of determining whether to correct the movement path and the correction processing are performed for each target being tracked.

[0090] First, the travel path adjustment unit 7 determines whether or not travel path adjustment is necessary (S21). If a travel path estimated before the current time exists and the past distance measurement information has been corrected by the distance measurement information correction unit 6, the travel path adjustment unit 7 determines that travel path adjustment is necessary (YES in S21) and proceeds to step S22. On the other hand, if a travel path estimated before the current time does not exist or the past distance measurement information has not been corrected by the distance measurement information correction unit 6, the travel path adjustment unit 7 determines that travel path adjustment is not necessary (NO in S21) and proceeds to step S25.

[0091] After the determination of YES in step S21, the travel path adjustment unit 7 uses the corrected distance measurement information to adjust the travel path estimated by the travel path estimation unit 5 (S22). This process is similar to the process of estimating a travel path by the travel path estimation unit 5 described with reference to FIG. 8.

[0092] Next, the travel path adjustment unit 7 determines whether or not to use the travel path adjusted in step S22 (S23). In this process, the travel path adjustment unit 7 compares the travel path adjusted in step S22 with the travel path previously estimated by the travel path estimation unit 5, and determines whether or not to use the adjusted travel path. If the difference between the travel paths is small or exceeds a preset range, the travel path adjustment unit 7 determines not to use the adjusted travel path (NO in S23) and proceeds to step S25. On the other hand, if the difference between the travel paths is large and is within a preset range, the travel path adjustment unit 7 determines to use the adjusted travel path (YES in S23) and proceeds to step S24.

[0093] After the YES determination in S23, the movement path adjustment unit 7 sets information about the adjusted movement path (S24). At this time, the movement path adjustment unit 7 outputs the information about the adjusted movement path to the object selection unit 8, sets an adjusted flag indicating that the movement path has been adjusted, ends this processing, and moves on to the next task (processing by the object selection unit 8).

[0094] On the other hand, after the determination of NO in steps S21 and S23, the movement path adjustment unit 7 sets information about an unadjusted movement path (S25). At this time, the movement path adjustment unit 7 outputs the information about the unadjusted movement path to the object selection unit 8, sets an unadjusted flag indicating that the movement path is unadjusted, ends this processing, and moves on to the next task (processing by the object selection unit 8).

[0095] <Example of Hardware Configuration of Computing Device> Next, a hardware configuration of the computing device 20 that constitutes the external environment recognition device 10 will be described.

[0096] 11 is a block diagram showing an example of the hardware configuration of the arithmetic device 20. The arithmetic device 20 is an example of hardware used as a computer that can operate as the external environment recognition device 10 according to this embodiment. In the external environment recognition device 10 according to this embodiment, each functional block is configured by the arithmetic device 20 (computer) executing a program, and the appearance recognition methods shown in FIGS. 8 to 10 are realized by the cooperation of each functional block.

[0097] The arithmetic device 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, and a RAM (Random Access Memory) 23, each of which is connected to a bus 24. The arithmetic device 20 further includes a non-volatile storage 25 and a network interface 26.

[0098] The CPU 21 reads out program code of software that realizes each function according to this embodiment from the ROM 22, loads it into the RAM 23, and executes it. Variables, parameters, etc. generated during the calculation process of the CPU 21 are temporarily written to the RAM 23, and these variables, parameters, etc. are read out by the CPU 21 as appropriate. The processing of each unit shown in FIG. 3 is realized by the CPU 21, the ROM 22, and the RAM 23. The various images and flags described above are stored in the RAM 23. However, an MPU (Micro Processing Unit) or a GPU (Graphics Processing Unit) may be used instead of the CPU 21, or the CPU 21 and the GPU may be used together.

[0099] The nonvolatile storage 25 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, or a nonvolatile memory. The nonvolatile storage 25 stores an operating system (OS), various parameters, and programs for operating the arithmetic device 20. The ROM 22 and the nonvolatile storage 27 store programs, data, and the like required for the CPU 21 to operate. In other words, the ROM 22 and the nonvolatile storage 25 are used as examples of computer-readable, non-transitory storage media that store programs executed by the arithmetic device 20.

[0100] The network interface 26 may be, for example, a network interface card (NIC). The network interface 26 can transmit and receive various data between devices via a local area network (LAN) connected to a terminal of the NIC, a dedicated line, or the like. For example, communication between the external environment recognition device 10 and a vehicle control device (not shown) is performed via the network interface 26.

[0101] The external environment recognition device 10 according to the embodiment described above corrects past ranging information based on high-precision ranging information among ranging information continuously measured in time series when calculating the movement path of a target continuously recognized based on information acquired from a sensor. The external environment recognition device 10 then corrects the movement path of the target based on the corrected past ranging information to calculate an accurate movement path of the target. The vehicle control device controls the operation of the vehicle using the accurate movement path of the target, thereby enabling the vehicle to operate safely.

[0102] Furthermore, the external environment recognition device 10 calculates a correction weight based on high-precision distance measurement information and uses this correction weight to correct past distance measurement information of the target. For example, based on high-precision distance measurement information calculated by stereo processing of a target captured in a stereo area, low-precision distance measurement information of the target calculated by monocular processing is corrected using the correction weight. This enables the external environment recognition device 10 to calculate the movement path of the target more accurately than before.

[0103] After performing low-accuracy distance measurement, the external environment recognition device 10 corrects the past distance measurement information of the target at the timing when high-accuracy distance measurement can be obtained, and adjusts the movement path of the target. Therefore, the external environment recognition device 10 can correct the past distance measurement information of targets that are continuously recognized, and adjust the movement path accurately.

[0104] Furthermore, when the external environment recognition device 10 recovers from a state in which it was unable to calculate highly accurate ranging information and becomes able to calculate highly accurate ranging information, it corrects the past ranging information using a correction weight calculated based on the highly accurate ranging information and adjusts the movement path. Therefore, the external environment recognition device 10 can increase the robustness of the adjusted movement path.

[0105] Furthermore, when a target moves from the low-accuracy measurement range to the high-accuracy measurement range, the external environment recognition device 10 corrects past ranging information using a correction weight calculated based on ranging information measured in the high-accuracy measurement range. This allows the external environment recognition device 10 to calculate an accurate movement path of the target. Note that the ranging information correction unit 6 may correct past ranging information using a method other than weighting.

[0106] Furthermore, the sensors may include not only imaging sensors such as multiple cameras and stereo cameras, but also a single monocular camera, a single ranging sensor such as radar or lidar, and sensor fusion using multiple sensors simultaneously. When the external environment recognition device 10 uses only a monocular camera, low-accuracy ranging information is obtained when the vehicle is far from the target, and high-accuracy ranging information is obtained as the vehicle approaches the target. This makes it possible to estimate a travel path based on high-accuracy ranging information. The external environment recognition device 10 may also use a monocular camera in combination with a stereo camera or ranging sensor. When low-accuracy ranging information cannot be obtained using only a monocular camera, the external environment recognition device 10 can obtain high-accuracy ranging information that can be calculated using a stereo camera or ranging sensor and estimate a travel path.

[0107] In addition, the external environment recognition device 10 does not need to correct the movement path if the image shows a target moving from the high-precision measurement range of the sensor to the low-precision measurement range and the vehicle does not move to the destination of the target.

[0108] The present invention is not limited to the above-described embodiments, and various other applications and modifications are possible without departing from the spirit of the present invention as defined in the claims. For example, the above-described embodiments provide detailed and specific descriptions of the device configuration in order to clearly explain the present invention, and are not necessarily limited to devices that include all of the described configurations. Furthermore, it is also possible to add, delete, or replace some of the configurations of the present embodiments with other configurations. Furthermore, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily represent all of the control lines and information lines in the product. In reality, it can be assumed that almost all of the configurations are interconnected.

[0109] 1... Information acquisition unit, 2... Target detection unit, 3... Target tracking unit, 4... Distance measurement information measurement unit, 5... Movement path estimation unit, 6... Distance measurement information correction unit, 7... Movement path adjustment unit, 8... Object selection unit, 9... Control determination unit, 10... External environment recognition device, 100... Vehicle

Claims

1. An external environment recognition device comprising: a ranging information measurement unit that measures ranging information of a target using information obtained from a sensor; a movement path estimation unit that estimates the movement path of the target based on the ranging information measured at a reference time and the ranging information measured at a time earlier than the reference time; a ranging information correction unit that corrects the past ranging information based on the difference between the estimated ranging information at the reference time estimated based on the past ranging information and the actual ranging information at the reference time measured with higher measurement accuracy than the past ranging information; and a movement path adjustment unit that adjusts the movement path of the target estimated based on the past ranging information based on the corrected ranging information.

2. The external environment recognition device according to claim 1, wherein the ranging information correction unit corrects the ranging information at the reference time and the past ranging information with a correction weight such that the amount of correction of the past ranging information decreases as the time goes back in time from the reference time.

3. The external environment recognition device described in claim 2, wherein the ranging information measurement unit measures the past ranging information using information acquired from at least one of the multiple sensors at the past time, and measures the ranging information at the reference time using information acquired from the multiple sensors at the reference time, and the ranging information correction unit corrects the past ranging information based on the ranging information at the reference time.

4. The external environment recognition device described in claim 3, wherein the ranging information measurement unit measures the past ranging information using information acquired from the sensor in the low-accuracy measurement range at the past time among the multiple sensors whose low-accuracy measurement range and high-accuracy measurement range overlap in part, and measures the ranging information at the reference time using information acquired from the sensor in the high-accuracy measurement range at the reference time, and the ranging information correction unit corrects the past ranging information based on the ranging information at the reference time.

5. The external environment recognition device described in claim 2, wherein the ranging information measurement unit measures the past ranging information using information obtained from at least one of the multiple sensors that was capable of recognizing the target at the past time, and measures the ranging information at the reference time using information obtained from the multiple sensors that became capable of recognizing the target at the reference time, and the ranging information correction unit corrects the past ranging information based on the ranging information at the reference time.

6. The external environment recognition device described in claim 2, wherein the ranging information measurement unit measures the past ranging information using information acquired in the low-accuracy measurement range of the sensor at the past time, among the information acquired from the sensor having a low-accuracy measurement range and a high-accuracy measurement range, and measures the ranging information at the reference time using information acquired in the high-accuracy measurement range of the sensor at the reference time; and the ranging information correction unit corrects the past ranging information based on the ranging information at the reference time.

7. A method for recognizing the external world, comprising: a step of measuring ranging information of a target using information obtained from a sensor; a step of estimating a movement path of the target based on the ranging information measured at a reference time and the ranging information measured at a time earlier than the reference time; a step of correcting the past ranging information based on the difference between estimated ranging information at the reference time estimated based on the past ranging information and actual ranging information at the reference time measured with higher measurement accuracy than the past ranging information; and a step of adjusting the movement path of the target estimated based on the past ranging information based on the corrected ranging information.

Images