Object recognition processing device, object recognition processing method and autonomous driving system

The object recognition processing device integrates and weights sensor outputs to accurately determine the distance and position of peripheral objects, addressing inaccuracies in existing systems by combining fusion signals of the same object, thereby improving autonomous driving reliability.

DE102017209772B4Active Publication Date: 2024-12-05MITSUBISHI ELECTRIC MOBILITY CORP
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
DE102017209772
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2016-06-10
Filing Date
2017-06-09
Publication Date
2024-12-05
Estimated Expiration
2037-06-09

AI Technical Summary

Technical Problem

Existing object recognition processing devices in autonomous driving systems face challenges in accurately determining the distance and position of peripheral objects due to discrepancies in measurements from sensors with different accuracies, leading to multiple tracks for the same object.

Method used

An object recognition processing device that integrates and weights sensor outputs from multiple sensors based on detection accuracy and reliability, selecting and combining fusion signals of the same object, and integrating them using weighted averages or medians considering estimation errors.

Benefits of technology

Accurately recognizes the distance and position of peripheral objects, reducing the risk of incorrect integration of different object measurements and enhancing the reliability of autonomous driving systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Object detection processing device for detecting objects existing around a vehicle, using a plurality of object detection units (3, 4) and receiving detection information of the objects from the respective object detection units (3, 4), the device comprising: an object fusion processing unit (6) for fusing detection information signals of the objects detected by the plurality of object detection units (3, 4) in order to output object fusion signals; a same-object fusion result combination candidate selection processing unit (7) for selecting, for a combination of object fusion signals, candidates of the object fusion signals to be combined; a same-object fusion result combination determination processing unit (10) for determining whether a combination of the selected object fusion signals is a combination of object fusion signals of the same object or not; and a same-object fusion combination integration processing unit (11) for recognizing the objects existing around the vehicle based on a result of the determination of the Same object fusion result combination determination processing unit (10), wherein the same-object fusion result combination candidate selection processing unit (7) selects candidates of the object fusion signals to be combined for a combination of object fusion signals based on object motion specification difference conditions, wherein, when the object fusion processing unit (6) adds an object feature information signal to each of the object fusion signals and outputs the object fusion signals, the object fusion processing unit (6) modifies the object motion specification difference conditions based on the object identification information, and wherein the object feature information signal includes at least one of direction, identification information, a color and a reception level of the corresponding object.
Need to check novelty before this filing date? Find Prior Art

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention

[0001] The present invention relates to an object recognition processing apparatus, an object recognition processing method, and an autonomous driving system. 2. Description of related prior art

[0002] Conventionally, in autonomous driving technologies and preventive safety driving technologies for automobiles, object detection processing devices are equipped with vehicle distance measuring devices. The vehicle distance measuring device measures, for example, the distance between a vehicle in which the distance measuring device is mounted (hereinafter referred to as the "own vehicle") and a peripheral object existing around the own vehicle. Values ​​obtained through the measurement are used in: a forward collision mitigation braking system for mitigating damage when the own vehicle collides with a forward object; an adaptive cruise control system for causing the own vehicle to follow a forward vehicle within a safe distance; an autonomous parking system for causing the own vehicle to park autonomously in a parking space; and the like.That is, the vehicle distance measuring device is a device for improving the safety and comfort of driving the own vehicle and is used in vehicle applications.

[0003] The inter-vehicle distance measuring device is equipped with a plurality of sensors, and sensor fusion technology for fusing a plurality of sensors is known as a technology for improving the performance of detecting a peripheral object existing around the host vehicle. For example, in JP 5 896 823 B2, the distance between the host vehicle and a peripheral object existing around the host vehicle, such as the position of the peripheral object, is measured using values ​​measured by a plurality of sensors with different observation accuracies.

[0004] In the sensor fusion technology disclosed in JP 5 896 823 B2, when the host vehicle follows a vehicle ahead, one sensor among the plurality of sensors with higher accuracy detects the vehicle ahead of the host vehicle (that is, the sensor with higher accuracy does not detect a peripheral object existing around the host vehicle), while one sensor among the plurality of sensors with lower accuracy detects a peripheral object existing around the host vehicle and measures the distance between the host vehicle and the peripheral object. The vehicle distance measuring device generates a data set (hereinafter referred to as a "track") of the transition of the position of the peripheral object for every predetermined time period using the measured distance.

[0005] In the sensor fusion technology disclosed in JP 5 896 823 B2, for example, if the sensor with higher accuracy detects a peripheral object existing around the host vehicle and the sensor with lower accuracy also detects a peripheral object existing around the host vehicle, the plurality of sensors detect the same object (the peripheral object). That is, the sensors with different accuracies detect the same object. As a result, the distance measured by the sensor with higher accuracy and the distance measured by the sensor with lower accuracy may have different values. Accordingly, two tracks are generated: one track is generated based on the distance measured by the sensor with higher accuracy, and another track is generated based on the distance measured by the sensor with lower accuracy.Thus, the sensor fusion technology disclosed in JP 5 896 823 B2 has the problem that a plurality of tracks are generated for one object.

[0006] DE 603 ​​16 226 T2 discloses a method for determining identical objects for determining whether objects are detected by multiple identical sensors. The method comprises determining a relative position and speed of the objects, determining a position error based on the determined positions, and calculating a first probability that the detected objects are identical in position from the determined position error based on a normal distribution with respect to the position error, determining a speed error based on the determined speed and calculating a second probability that the detected objects are identical in speed from the determined speed error based on a normal distribution with respect to the speed error, calculating a third probability that the detected objects are identical,based on the calculated first probability and the calculated second probability, and determining that the detected objects are identical if the third probability exceeds a first determination value.

[0007] US 2016 / 0 116 593 A1 discloses an object detection device comprising a sensor unit including a light distance and range sensor (LiDAR) for collecting LiDAR data and a camera sensor for collecting image data, an area dividing unit configured to divide a common detection area of ​​the LiDAR sensor and the camera sensor into a plurality of areas, a data analysis unit configured to analyze the LiDAR data to extract first object information and / or analyze the image data to extract second object information, and a sensor signal converging unit configured to determine whether an object exists for each of the divided areas based on the first object information and / or the second object information.

[0008] US 2014 / 0 266 651 A1 discloses a system and method for blind zone obstacle detection for a host vehicle. It includes a sensor array configured to generate measurement data for a blind zone of the host vehicle, a blind zone object detector having at least two detectors coupled to the sensor array to process the measurement data and generate outputs, which are then combined to form a final detection value used to detect an object in the blind zone of the host vehicle.

[0009] US 2017 / 0 043 771 A describes a device for real-time signaling of at least one object to a navigation module of a vehicle. The device includes a first sensor for producing first sensor data including a first detected position and a first detected speed of the object relative to the vehicle, at least one second sensor for producing second sensor data including a second detected position and a second detected speed of the object relative to the vehicle, a synchronization module for producing synchronized data including a first synchronized position from the first detected position and the first detected speed, and at least one second synchronization position from the second detected position and the second detected speed.and a merging module for generating merged data including a merged position from the first synchronized position and the second synchronized position for signaling the object to the navigation module by communicating all or part of the merged data thereto.

[0010] US 2015 / 0 112 570 A1 describes a driver assistance system with a prediction subsystem in a vehicle.

[0011] US 5 638 116 A describes an object detection device that can detect an object in an image captured by a vehicle-mounted camera with high efficiency and accuracy. SUMMARY OF THE INVENTION

[0012] The present invention has been made to solve the above-described problem, and it is an object of the present invention to provide an object recognition processing apparatus capable of accurately recognizing the distance between a subject vehicle and a peripheral object and the position of the peripheral object, as compared with the conventional object recognition processing apparatus.

[0013] This object is achieved by an object recognition processing device having the features specified in claim 1. Advantageous embodiments emerge from the subclaims. The object is further achieved by an autonomous driving system according to claim 14, an object recognition method according to claim 15, an object recognition processing device according to claim 16, and an object recognition method according to claim 17.

[0014] According to the present invention, it is possible to accurately detect the distance between the host vehicle and a peripheral object and the position of the peripheral object, compared with the conventional object detection processing apparatus.

[0015] The above and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing an object recognition processing apparatus according to a first embodiment of the present invention; Fig. 2A is a flowchart (first part) showing object recognition processing according to the first embodiment of the present invention; Fig. Fig. 2B is a flowchart (second part) showing the object recognition processing according to the first embodiment of the present invention; and Fig. 3 is a hardware configuration diagram showing the object recognition processing apparatus according to the first embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION FIRST EMBODIMENT

[0016] In the first embodiment of the present invention, an object recognition processing apparatus and an object recognition processing method are described, which include: determining whether or not a combination of fusion signals of outputs from a plurality of sensors is a combination of fusion signals of the same object, and integrating the combinations of fusion signals determined as fusion signals of the same object.

[0017] Examples of the outputs of the plurality of sensors include: a relative position of a peripheral object to the host vehicle; a relative speed of the peripheral object to the host vehicle; identification information (vehicle, person, etc.) of the peripheral object; sensor detection accuracy and sensor identification reliability. When the relative positions of the peripheral object to the host vehicle output by the plurality of sensors are combined, the respective relative positions are weighted based on the detection accuracies of the sensors, and the weighted values ​​are summed to obtain a value that is a relative position regarded as a recognition result.When pieces of peripheral object identification information output from the plurality of sensors are combined, the identification information output from a sensor with higher sensor identification reliability is considered a recognition result. Thus, "fusion" means combining the outputs from the plurality of sensors in the manner described above.

[0018] Hereinafter, the object recognition processing apparatus according to the first embodiment of the present invention based on Fig. 1, Fig. 2A and Fig. 2B. Fig. Fig. 1 is a block diagram showing the object recognition processing apparatus 1 according to the first embodiment of the present invention together with a vehicle control unit 2. It should be noted that arrows in Fig. 1 specify the sequence of signals.

[0019] A vehicle on which the object recognition processing device 1 according to the present invention is mounted is provided with the plurality of sensors, a first object detection unit 3, and a second object detection unit 4. Each of the first object detection unit 3 and the second object detection unit 4 is composed of, for example, a millimeter-wave radar, a laser radar, a camera, an ultrasonic sensor, an inter-vehicle communication device (a device for communicating between a plurality of automobiles), or the like. Each of the first object detection unit 3 and the second object detection unit 4 is configured to measure the distance between the own vehicle and a peripheral object existing around the own object, and the relative position of the peripheral object to the own object.The measured distance and relative position are output as a distance signal and a relative position signal to an object loss detection unit 5. Note that object detection information includes the distance signal and the relative position signal. Furthermore, the object detection information output from each of the first object detection unit 3 and the second object detection unit 4 also includes object identification information and speed information in addition to the distance signal and the relative position signal.

[0020] The first object detection unit 3 and the second object detection unit 4 mounted on the vehicle may be implemented by devices of the same type. For example, both units 3 and 4 may be millimeter-wave radars. Alternatively, the first object detection unit 3 and the second object detection unit 4 may be implemented by devices of different types. For example, the first object detection unit 3 may be a millimeter-wave radar, while the second object detection unit 4 is a camera. Furthermore, the first object detection unit 3 and the second object detection unit 4 may have the same detection accuracy or may have different detection accuracies.

[0021] The object loss detection unit 5 is a processing unit configured to detect erroneous detection (hereinafter referred to as “loss”) of a peripheral object by both the first object detection unit 3 and the second object detection unit 4, and counts the number of times of “loss”.

[0022] An object fusion processing unit 6 receives the object detection information output from each of the first object detection unit 3 and the second object detection unit 4, and receives the number of times of "loss" in object detection detected by the lost object detection unit 5. Further, the object fusion processing unit 6 performs a fusion of the object detection information from the first object detection unit 3 with the object detection information from the second object detection unit 4.

[0023] A same-object fusion result combination candidate selection processing unit 7 is a processing unit configured to select combination candidates of fusion results of the same object from the object fusion results obtained in the object fusion processing unit 6. That is, the same-object fusion result combination candidate selection processing unit 7 selects combination candidates of fusion results. This process corresponds to steps S101 to S111 in Fig. 2A, which will be described later.

[0024] A highly reliable same-object fusion result combination candidate extraction processing unit 8 is a processing unit configured to extract, from the combination candidates of fusion results of the same object, a combination of fusion results with higher detection reliability of the object detection information. This process corresponds to steps S121 to S122 and step S104 in the following steps. Fig. 2A and Fig. 2B.

[0025] An object detection number counter 9 is a processing unit configured to calculate the sum of the number of object detections fused in the object fusion processing unit 6.

[0026] A same-object fusion result combination determination processing unit 10 is a processing unit configured to determine whether signals from the high-reliability same-object fusion result combination candidate extraction processing unit 8 are fusion signals of the same object or not. This process corresponds to steps S131 to S134 and step S104 in the later-described Fig. 2B.

[0027] A same-object fusion combination integration processing unit 11 is a processing unit configured to integrate a combination determined as a combination of fusion signals of the same object. This process corresponds to step S141 and step S104 in the later-described Fig. 2B.

[0028] Finally, the vehicle control unit 2 is configured to control movements of the host vehicle, such as yaw rate, steering angle, vehicle speed, etc. of the host vehicle based on the detection result of the object detection processing device 1.

[0029] As described above, detected in the Fig. In the object recognition processing device shown in FIG. 1, the lost object detection unit 5 detects the "loss" of objects from the first object detection unit 3 and the second object detection unit 4 and outputs an object loss signal. Upon receiving the object loss signal, the object fusion processing unit 6 combines detection information signals of the objects detected by the plurality of object detection units and outputs object fusion signals. The same-object fusion result combination candidate selection processing unit 7 selects, as a combination of object fusion signals, candidates of the object fusion signals to be combined. The high-reliability same-object fusion result combination candidate extraction processing unit 8 selects, from among the candidates, a combination of fusion results with a higher detection probability of the object detection information.The object detection count counter 9 calculates the sum of the number of object detections combined by the object fusion processing unit 6. The same-object fusion result combination determination processing unit 10 determines whether the selected combination of object fusion signals is a combination of fusion signals of the same object. The same-object fusion combination integration processing unit 11 detects an object existing around the vehicle based on the determination result.

[0030] The following are the operations of the Fig. 1 shown corresponding components are described in more detail.

[0031] The same-object fusion result combination candidate selection processing unit 7 selects, as a combination of object fusion signals, the candidate object fusion signals to be combined based on the object motion specification difference conditions. Regarding the result of fusion of a plurality of objects determined by the process performed by the object fusion processing unit 6, the object motion specification difference conditions include at least one of (1) the position difference between the objects, (2) the speed difference between the objects, and (3) the acceleration difference between the objects.When the object fusion processing unit 6 adds an object feature information signal to the object fusion signal and outputs the object fusion signal, the same-object fusion result combination candidate selection processing unit 7 is allowed to change the object motion specification difference condition based on the object information.

[0032] When the object fusion processing unit 6 adds the object feature information signal (described later) to the object fusion signal and outputs the object fusion signal, the same-object fusion result combination candidate selection processing unit 7 selects, as the combination of object fusion signals, candidates of the object fusion signals to be combined based on the object feature information signal received from the object fusion processing unit 6. The object feature information signal includes at least one of the direction, identification information, color, and reception level of the corresponding object.

[0033] Based on a detection reliability condition signal (described later) received from both the object detection unit 3 and the object detection unit 4, the highly reliable same-object fusion result combination candidate extraction processing unit 8 narrows down the combination candidates of fusion results of the same object for a combination of object fusion signals that satisfies detection reliability conditions included in the detection reliability condition signal.

[0034] The detection reliability conditions include at least one of (1) the number of times the object fusion processing unit 6 cannot combine the object detection information signals (in other words, the number of “losses” of object fusion signals), (2) the detection position of the object, (3) the detection reliability of the object, and (4) the estimation error of the detection of the object.

[0035] In a combination of object fusion signals selected in the most recent n processing periods (n: natural number), if the number of object detections used to fuse objects for the most recent n processing periods is equal to or greater than a same object fusion result combination determination threshold, the same object fusion result combination determination processing unit 10 determines that the combination is a combination of fusion results of the same object.If a plurality of detection information signals of the objects used to unify objects for the most recent n processing periods are all detected by the same object detection unit, that is, only one of the first object detection unit 3 and the second object detection unit 4, the same-object fusion result combination determination processing unit 10 does not determine whether the combination is a combination of fusion results of the same object. Further, if the object detection unit that generated combination candidates of object fusion signals (that is, one of the first object detection unit 3 and the second object detection unit 4) detects the combination candidates of fusion results of the same object at the same time, the same-object fusion result combination determination processing unit 10 does not determine whether the combination is a combination of fusion results of the same object.

[0036] The same object fusion combination integration processing unit 11 detects the object existing around the vehicle based on at least one of the following values: an average value of a plurality of object fusion signals; a median value thereof; a mode value thereof; a shortest vertical position thereof; and a value obtained by weighting the plurality of object fusion signals, taking into account estimated errors thereof, and integrating the weighted object fusion signals.

[0037] The same-object fusion combination integration processing unit 11 can determine at least one of: an average value of relative positions, relative velocities, or relative accelerations included in a plurality of object fusion signals; a median value thereof; a mode value thereof; a shortest vertical position thereof; and a value obtained by weighting the plurality of object fusion signals while considering estimated errors thereof and integrating the weighted object fusion signals, and can detect the vehicle existing around the own vehicle based on the determined value. For example, when calculating the detection result using the average value, it is assumed that the plurality of object fusion signals are lane 1 and lane 2, and the relative position of lane 1 is x1 and the relative position of lane 2 is x2.Then the relative position of the object detection signal is expressed by (x1+x2) / 2.

[0038] The process of weighting and integrating the object fusion results while considering their estimation errors is expressed by the following expression, taking the horizontal position of the object detection result as an example. Note that a (0 ≤ a ≤ 1) depends on the estimation errors of track 1 and track 2.

[0039] If the estimation error of the horizontal position of track 1 is larger than the estimation error of the horizontal position of track 2, a is a value less than 0.5. If the estimation error of the horizontal position of track 1 is smaller than the estimation error of the horizontal position of track 2, a is a value greater than 0.5.

[0040] Horizontal position of the object detection result = a × horizontal position of track 1 + 1-a) × horizontal position of track 2.

[0041] It should be noted that, with respect to the vertical position, horizontal velocity and vertical velocity, the method of weighting and integrating the object fusion signals taking into account their estimation errors is the same as those described above.

[0042] As described above, the same-object fusion combination integration processing unit 11 detects the vehicle existing around the own vehicle using the object fusion signals corresponding to the most recent n processing periods.

[0043] Next, the operation processing of the object recognition processing apparatus 1 according to the first embodiment will be described in detail with reference to Fig. 2A and Fig. 2B. Fig. 2A and Fig. 2B are flowcharts illustrating the operation processing of the object recognition processing device 1.

[0044] As in Fig. 1, Fig. 2A and Fig. As shown in Fig. 2B, the inputs of the same-object fusion result combination candidate selection processing unit 7 are two fusion signals selected from all object fusion signals, and are referred to as a track 1 and a track 2, respectively. The term "track" means a transition data set of the position of the corresponding peripheral object for each predetermined time period.

[0045] First, in step S101, it is determined whether or not track 1 and track 2 have object identification information. If a result of the determination in step S101 is that both track 1 and track 2 have object identification information (Yes in step S101), the processing proceeds to step S102. In step S102, it is determined whether or not track 1 and track 2 have the same object identification signals.

[0046] When the result of the determination in step S102 is that track 1 and track 2 have the same object identification signal (Yes in step S102), an object specification difference threshold 1 is set as an object motion specification threshold in step S103.

[0047] If the determination result in step S102 is that track 1 and track 2 do not have the same object identification signal (No in step S102), processing proceeds to step S104. In step S104, it is determined that the combination of track 1 and track 2 is not subject to the process of integrating the fusion signals of the same object, and processing is terminated.

[0048] The reason is as follows. If the determination result in step S102 is that the object identification signals of track 1 and track 2 are different from each other, it is determined that it is highly likely that track 1 and track 2 are the fusion results of different objects, and therefore, they do not need to be subjected to the process of integrating the fusion signals of the same object, thereby reducing the risk of erroneous integration of the fusion signals of different objects.

[0049] If the result of the determination in step S101 is that both track 1 and track 2 do not contain object identification information (No in step S101), processing proceeds to step S105. In step S105, it is determined whether or not track 1 or track 2 contains object identification information.

[0050] If the result of the determination in step S105 is that track 1 or track 2 has object identification information (Yes in step S105), the process proceeds to step S106. In step S106, an object specification difference threshold 2 is set as an object motion specification threshold. If the result of the determination in step S105 is that neither track 1 nor track 2 has object identification information (No in step S105), the process proceeds to step S107. In step S107, an object motion specification threshold 3 is set as an object motion specification threshold.

[0051] Regarding the setting of the object motion specification thresholds, for example, the object motion specification threshold set when track 1 and track 2 have the same object identification information (Yes in step S102) is set higher than the object motion specification threshold set when track 1 or track 2 has object identification information (Yes in step S105) or when neither track 1 nor track 2 has object identification information (No in step S105).

[0052] If the object identification signals are the same, the probability of being the same object is high, and the threshold is raised to relax the same object determination criterion. On the other hand, if there is a probability of being different object identification information, the object motion specification difference threshold is lowered, thereby reducing the risk of erroneous integration of the fusion signals of different objects.

[0053] While in steps S101, S102, and S105, a determination is made based on the object identification information as described above, object direction, color, or reception level may be used. Also in this case, as with the determination based on object identification information, if the object directions, colors, or reception levels of track 1 and track 2 differ significantly from each other, it is determined that the combination of track 1 and track 2 should not be subjected to the process of integrating the fusion results of the same object.

[0054] Subsequently, in step S111, with respect to the track 1 and the track 2 in the most recent n processing periods, if each of the horizontal position difference, the vertical position difference, the horizontal speed difference, and the vertical speed difference between the track 1 and the track 2 in each period is equal to or smaller than the object motion specification difference threshold determined in step S103, step S106, or step S107 (Yes in step S111), the processing proceeds to step S121.

[0055] On the other hand, in step S111, with respect to track 1 and track 2 in the most recent final processing periods, even if any one of the horizontal position difference, vertical position difference, high-speed difference, and vertical speed difference between track 1 and track 2 in each period is greater than the object motion specification difference threshold determined in step S103, step S106, or step S107 (No in step S111), the processing proceeds to step S104. In step S104, it is determined that the combination of track 1 and track 2 is not to be subjected to the process of integrating the fusion signals of the same object, and the processing is terminated.

[0056] In step S111, for example, a motion specification difference evaluation value may be set by weighting the values ​​of the corresponding motion specifications, as shown by the following expression. Then, the combination of track 1 and track 2 may be determined to be subjected to the process of integrating the fusion signals of the same object if the motion specification difference evaluation value is less than a fixed threshold, and may be determined not to be subjected to the process of integrating the fusion signals of the same object if the motion specification difference evaluation value is equal to or greater than a fixed threshold.

[0057] Motion specification difference evaluation value = c × horizontal position difference + d × vertical position difference + e × high-speed difference + f × vertical speed difference where c, d, e and f are given values.

[0058] As described above, in combination extraction, if the object motion specification difference between track 1 and track 2 is equal to or smaller than the object motion specification difference threshold, track 1 and track 2 are determined as combination candidates of fusion signals of the same object, thereby making it possible to reduce the risk of erroneous integration of fusion signals of different objects.

[0059] Next, in step S121, with respect to track 1 and track 2 in the most recent n processing periods, if the number of times both track 1 and track 2 could not be correlated with the detection information of the object detection units in the same period (i.e., the number of "losses" in object detections detected by the lost object detection unit 5) is equal to or less than a predetermined threshold (e.g., n / 3 times) (Yes in step S121), the processing proceeds to step S122. In step S122, it is determined that track 1 and track 2 are combination candidates of fusion signals of the same object, and the processing proceeds to step S131.

[0060] On the other hand, in step S121, with respect to track 1 and track 2 in the most recent n processing periods, if the number of times of "lost" object detections calculated by the lost object detection unit 5 that generated track 1 and track 2 is greater than n / 3 times (No in step S121), the processing proceeds to step S104. In step S104, it is determined that the combination of track 1 and track 2 is not to be subjected to the process of integrating the fusion signals of the same object, and the processing is terminated.

[0061] That is, during the most recent n processing periods, if the number of "losses" in object detections calculated by the object loss detection unit 5 that generated track 1 and track 2 is greater than n / 3, it is determined that a detection reliability of one or both of track 1 and track 2 is low, and the combination of track 1 and track 2 is determined not to be subjected to the process of integrating the fusion signals of the same object, whereby the risk of erroneous integration of fusion signals of different objects can be reduced.

[0062] Based on the detection positions and estimation errors of track 1 and track 2, a combination of track 1 and track 2 with a low detection probability can be determined so that it will not be subjected to the process of integrating the fusion signals of the same object. If track 1 and track 2 have detection reliability information, the detection reliability information can be used to determine the combination with a low detection reliability so that it will not be subjected to the process of integrating the fusion signals of the same object.

[0063] The process corresponding to steps S121 and S122 described above is a process for extracting a combination with higher reliability, such as a combination with high detection reliability of the object detection unit, from combinations already selected in the process corresponding to steps S101 to S111 (i.e., selected by the same-object fusion result combination candidate selection processing unit 7). Therefore, there is no problem even if the process corresponding to steps S121 and S122 is omitted.

[0064] Subsequently, in step S131, if the track 1 and the track 2 are generated using detection information detected by a plurality of object detection units during the most recent final processing periods (Yes in step S131), the processing proceeds to step S132.

[0065] On the other hand, in step S131, if track 1 and track 2 are generated using detection information of the same object detection unit during the most recent n processing periods (No in step S131), the processing proceeds to step S104. In step S104, it is determined that the combination of track 1 and track 2 is not to be subjected to the process of integrating the fusion signals of the same object, and the processing is terminated.

[0066] That is, if track 1 and track 2 are generated using only the detection information of the same object detection unit during the most recent n processing periods, it is determined that track 1 and track 2 are most likely the fusion results of different objects, and track 1 and track 2 are determined not to be subjected to the process of integrating the fusion signals of the same object. Thus, the risk of erroneous integration of the fusion signals from different objects can be reduced.

[0067] Subsequently, in step S132, if the track 1 and the track 2 are not generated using detection information of the same object detection unit in the same period during the last n processing periods (No in step S132), the processing proceeds to step S133.

[0068] On the other hand, if in step S132, track 1 and track 2 are generated using the detection information of the same object detection unit in the same period during the most recent n processing periods (Yes in step S132), the processing proceeds to step S104. In step S104, it is determined that the combination of track 1 and track 2 is not to be subjected to the process of integrating the fusion signals of the same object, and the processing is terminated.

[0069] Subsequently, if in step S133, the sum of the number of object detections calculated by the object detection count counter 9 and used to generate track 1 and track 2 in the most recent n processing periods is equal to or less than a predetermined detection count threshold, for example, 3 × (2 × n) / 5 (Yes in step S133), the processing proceeds to step S134. In step S134, the same-object fusion result combination determination processing unit 10 determines that the combination of track 1 and track 2 is a combination of fusion signals of the same object.

[0070] On the other hand, if in step S133 the sum of the number of object detections calculated by the object detection number counter 9 and used to generate the track 1 and the track 2 in the most recent n processing periods is greater than the predetermined detection number threshold (e.g., 3 × (2 × n) / 5 (No in step S133), the processing proceeds to step S104. In step S104, it is determined that the combination of the track 1 and the track 2 is not to be subjected to the process of integrating the fusion signals of the same object, and the processing is terminated.

[0071] That is, it is estimated that the sum of the number of object detections used to generate track 1 and track 2 in the most recent final processing periods is small when track 1 and track 2 are fusion signals of the same object, compared to the case where track 1 and track 2 are fusion signals of different objects. Therefore, if the sum of the number of object detections used to generate track 1 and track 2 in the most recent final processing periods is greater than the predetermined detection number threshold, for example, 3 × (2 × n) / 5, track 1 and track 2 are not subjected to the process of integrating the fusion signals of the same object. Thus, it is possible to reduce the risk of erroneous integration of fusion signals of different objects.

[0072] Subsequently, in step S141, a new track is generated based on an average value of each of the motion specifications between track 1 and track 2, and the current track 1 and track 2 are canceled to end the processing. The above-described series of processes is performed on all combinations of object fusion signals.

[0073] In this embodiment, a new track is generated based on an average value of each of the motion specifications between track 1 and track 2. However, a new track may be generated based on a median value, a mode value, or a shortest vertical position of each of the motion specifications between track 1 and track 2. Alternatively, a new track may be generated by weighting the respective motion specifications while considering the estimation errors of track 1 and track 2.

[0074] A method of integrating a plurality of tracks can be determined based on the object identification information of the object fusion signals.

[0075] Alternatively, by using data of track 1 and track 2 for m processing periods (m: natural number) in the past, a new track can be generated based on an average value, a median value, a mode value, or a shortest vertical position of the data, or a new track can be generated by weighting the data taking into account the estimation errors of track 1 and track 2.

[0076] Although in the present embodiment, two object fusion signals are selected from all object fusion signals, three or more object fusion signals may be selected to implement the present embodiment.

[0077] The vehicle control unit 2 performs, for example, control of a forward collision mitigation braking system for mitigating damage when the host vehicle collides with a forward object, and an adaptive cruise control system for causing the host vehicle to follow a forward vehicle within a safe distance, using information such as the distance between the host vehicle and a peripheral object existing around the host vehicle, the position of the peripheral object, or the like, which is acquired by the object recognition processing device 1. That is, autonomous driving of the host vehicle is realized based on the object recognition result of the object recognition processing device 1.

[0078] Fig. 3 shows an example of hardware of the object recognition processing device 1. The object recognition processing device 1 consists of a processor 100 and a memory unit 101, as shown in Fig. 3. Although not shown, the storage unit 101 includes a volatile storage device such as a random access memory and a non-volatile auxiliary storage device such as a flash memory. The storage unit 101 may include an auxiliary storage device such as a hard disk instead of the flash memory.

[0079] The processor 100 executes a program loaded from the storage unit 101. In this case, the program is loaded from the auxiliary storage device to the processor 100 via the volatile storage device. The processor 100 can output data of the calculation result and the like to the volatile storage device of the storage unit 101 or can store the data in the auxiliary storage device via the volatile storage device.

Claims

[1] An object detection processing apparatus for detecting objects existing around a vehicle using a plurality of object detection units (3, 4) and receiving detection information of the objects from the respective object detection units (3, 4), the apparatus comprising: an object fusion processing unit (6) for fusing detection information signals of the objects detected by the plurality of object detection units (3, 4) to output object fusion signals; a same-object fusion result combination candidate selection processing unit (7) for selecting, for a combination of object fusion signals, candidates of the object fusion signals to be combined; a same-object fusion result combination determination processing unit (10) for determining whether a combination of the selected object fusion signals is a combination of object fusion signals of the same object or not; and a same-object fusion combination integration processing unit (11) for recognizing the objects existing around the vehicle based on a result of the determination of the Same object fusion result combination determination processing unit (10), wherein the same-object fusion result combination candidate selection processing unit (7) selects, for a combination of object fusion signals, candidates of the object fusion signals to be combined based on object motion specification difference conditions, wherein, when the object fusion processing unit (6) adds an object feature information signal to each of the object fusion signals and outputs the object fusion signals, the object fusion processing unit (6) changes the object movement specification difference conditions based on the object identification information, and wherein the object feature information signal includes at least one of direction, identification information, a color, and a reception level of the corresponding object. [2] An object recognition processing apparatus according to claim 1, further comprising: a high-reliability same-object fusion result combination candidate extraction processing unit (8) for extracting a combination of the object fusion signals with a higher detection reliability of object detection information from the candidates selected by the same-object fusion result combination candidate selection processing unit (7), wherein the same-object fusion result combination determination processing unit (10) determines whether the combination of the object fusion signals extracted by the high-reliability same-object fusion result combination candidate extraction processing unit (8) is a combination of object fusion signals of the same object or not. [3] The object recognition processing apparatus according to claim 2, wherein the high-reliability same-object fusion result combination candidate extraction processing unit (8) selects, for a combination of the object fusion signals, candidates of the object fusion signals to be combined based on the detection reliability condition signals received from the object detection units (3, 4). [4] The object recognition processing apparatus according to claim 3, wherein the highly reliable same object fusion result combination candidate extraction processing unit (8) selects, as the candidate object fusion signals, detection reliability conditions included in the detection reliability condition signals received from the object detection units (3, 4). [5] The object recognition processing apparatus according to claim 3 or 4, wherein the detection reliability conditions include at least one of: the number of times the object fusion processing unit (6) cannot fuse detection information signals of objects; detection positions of the objects; detection reliabilities of the objects; and estimation errors of detection of the objects. [6] The object recognition processing apparatus according to claim 1, wherein the object motion specification difference conditions include, with respect to a fusion result of a plurality of objects by the object fusion processing unit (6), at least one of: position difference between the objects; speed difference between the objects; and acceleration difference between the objects. [7] The object recognition processing apparatus according to claim 1, wherein, when the object fusion processing unit (6) adds an object feature information signal to each of the object fusion signals and outputs the object fusion signals, the same-object fusion result combination candidate selection processing unit (7) selects, for a combination of object fusion signals, candidates of the object fusion signals to be combined based on the object feature information signals received from the object fusion processing unit (6). [8] The object recognition processing apparatus according to claim 1, wherein, in a combination of the object fusion signals selected in the most recent n processing periods (n: natural number), if the number of object detections used to fuse objects for the most recent n processing periods is equal to or greater than a predetermined threshold for combining fusion signals of the same object, the same object fusion result combination determination processing unit (10) determines that the combination is a combination of fusion results of the same object. [9] The object recognition processing apparatus according to claim 1, wherein, with respect to a combination of the object fusion signals selected in the most recent n processing periods (n: natural number), if a plurality of detection information signals of the objects used to fuse objects for the most recent n processing periods are all detected by the same object detection unit, the same object fusion result combination determination processing unit (10) does not determine whether or not this combination is a combination of object fusion signals of the same object. [10] An object recognition processing apparatus according to claim 1, wherein if the object detection units (3, 4) that generated the combination candidates of the object fusion signals detect combination candidates of fusion results of the same object at the same time, the same object fusion result combination determination processing unit (10) does not determine whether the combination candidates are object fusion signals of the same object or not. [11] An object recognition processing apparatus according to claim 1, wherein said same-object fusion combination integration processing unit (11) recognizes the object existing around the vehicle based on an average value of a plurality of object fusion signals or estimation error values ​​of the object fusion signals. [12] The object recognition processing apparatus according to claim 1, wherein the same-object fusion combination integration processing unit (11) determines an average value of a plurality of object fusion signals or estimation error values ​​of the object fusion signals based on an object identification information signal added to each of the object fusion signals, and recognizes the object existing around the vehicle based on the determined value. [13] An object recognition processing apparatus according to claim 11 or 12, wherein said same-object fusion combination integration processing unit (11) recognizes the object existing around the vehicle using the object fusion signals for the most recent n processing periods. [14] An autonomous driving system comprising a vehicle control unit (2) for controlling an operation of a vehicle based on an output of the object recognition processing device (1) according to any one of claims 1 to 13. [15] Object recognition methods, comprising: a first step of detecting objects existing around a vehicle using a plurality of object detection units, and receiving detection information of the objects from the respective object detection units; a second step of fusing detection information signals of the objects detected by the plurality of object detection units to output object fusion signals; a third step of selecting, for a combination of object fusion signals, candidates of the object fusion signals to be combined based on object motion specification difference conditions; a fourth step of determining whether or not a combination of the selected object fusion signals is a combination of object fusion signals of the same object; a fifth step of recognizing the objects existing around the vehicle based on a result of the determination; and a sixth step of changing the object motion specification difference conditions based on the object identification information, wherein when an object feature information signal is added to each of the object fusion signals and the object fusion signals are output; wherein the object feature information signal includes at least one of direction, identification information, a color, and a reception level of the corresponding object. [16] An object detection processing apparatus for detecting objects existing around a vehicle using a plurality of object detection units (3, 4) and receiving detection information of the objects from the respective object detection units (3, 4), the apparatus comprising: an object fusion processing unit (6) for fusing detection information signals of the objects detected by the plurality of object detection units (3, 4) to output object fusion signals; a same-object fusion result combination candidate selection processing unit (7) for selecting, for a combination of object fusion signals, candidates of the object fusion signals to be combined; a same-object fusion result combination determination processing unit (10) for determining whether a combination of the selected object fusion signals is a combination of object fusion signals of the same object or not; and a same-object fusion combination integration processing unit (11) for detecting the objects existing around the vehicle based on a result of the determination of the same-object fusion result combination determination processing unit (10), wherein, with respect to a combination of the object fusion signals selected in the most recent n processing periods (n: natural number), if a plurality of detection information signals of the objects used to fuse objects for the most recent n processing periods are all detected by the same object detection unit, the same object fusion result combination determination processing unit (10) does not determine whether or not this combination is a combination of object fusion signals of the same object. [17] Object recognition methods, comprising: Detecting objects existing around a vehicle using a plurality of object detection units (3, 4); Receiving detection information of the objects from the respective object detection units (3, 4); fusing detection information signals of the objects detected by the plurality of object detection units (3, 4) to output object fusion signals; selecting, for a combination of object fusion signals, candidates of the object fusion signals to be combined; determining whether or not a combination of the selected object fusion signals is a combination of object fusion signals of the same object; Detecting the objects existing around the vehicle based on a result of the determination; wherein, in determining whether or not a combination of the selected object fusion signals is a combination of object fusion signals of the same object, if, in a combination of the object fusion signals selected in the most recent n processing periods (n: natural number), the number of object detections used to fuse objects for the most recent n processing periods is greater than or equal to a determination threshold for the combination of fusion signals of the same object, this combination is determined to be a combination of fusion results of the same object.

Citation Information

Patent Citations

  • Method and device for determining whether an object detected by several sensors is identical and method and device for position correction in a multi-sensor system

    DE60316226T2

  • Target tracking device and target tracking method

    JP5896823B2

  • System and method for blindzone object detection

    US20140266651A1

  • Confidence estimation for predictive driver assistance systems based on plausibility rules

    US20150112570A1

  • Object detecting apparatus, and method of operating the same

    US20160116593A1