Method of updating the existance probability of a target vehicle based on sensor perceived areas

The method updates the existence probability of a target vehicle by computing a dynamic perceived area within sensor fusion algorithms, addressing the challenge of occluded tracks and enhancing the accuracy and safety of autonomous vehicle tracking.

GB2599939BActive Publication Date: 2024-11-13CONTINENTAL AUTONOMOUS MOBILITY GERMANY GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
GB2020016383
Authority / Receiving Office
GB · GB
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-10-15
Publication Date
2024-11-13
Estimated Expiration
2040-10-15

AI Technical Summary

Technical Problem

Current sensor fusion algorithms in advanced driving assistance systems and autonomous driving struggle to accurately handle occluded tracks, leading to premature deletion of target vehicles due to limited understanding of sensor perceived areas, which can result in safety issues and delayed re-confirmation of tracks upon re-emergence.

Method used

A method to update the existence probability of a target vehicle by computing a dynamic perceived area using environment data and sensor information, distinguishing between occluded and non-visible tracks, and adjusting the probability of existence accordingly to prevent premature deletion.

Benefits of technology

This approach enhances the timely and accurate tracking of vehicles by differentiating between hidden and non-detected tracks, reducing false positives and negatives, and ensuring safer autonomous vehicle operations by utilizing a dynamic field of view within sensor fusion algorithms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000001_0000
    Figure 00000001_0000
  • Figure 00000001_0001
    Figure 00000001_0001
  • Figure 00000002_0000
    Figure 00000002_0000
Patent Text Reader

Abstract

The invention refers to a method of updating the existence probability of a track, configured to be performed by means of a fusion system of an ego vehicle, an electronic control unit mounted on the ego vehicle and a multitude of sensors operatively coupled to the ego vehicle. On the basis of the sensor data a perceived area is computed for each sensor using environment data that is available to the fusion system then individual sensor tracks are generated based on the received sensor data and it is determined whether an association between the individual sensor tracks exists to provide associated and unassociated tracks. For each unassociated track the probability of the existence of a target vehicle is updated then it is determined if a track is visible or not by determining if it is inside the dynamic perceived area of at least one sensor and if so then the probability of existence of a target vehicle for the unassociated track is decreased by a first factor to obtain a probability of existence within the perceived area. If the unassociated track is outside the perceived area then either the probability of existence is left alone or it is reduced by a second factor lower than the first to provide a probability of existence outside the perceived area. The respective probabilities of existence inside or outside the perceived area are compared against a threshold and those unassociated tracks with a probability below the threshold are deleted while associated tracks and unassociated tracks with probabilities above the threshold are updated from which sensor confirmed tracks are then generated.
Need to check novelty before this filing date? Find Prior Art

Description

Method of Updating the Existence Probability of a Target Vehicle Based on Sensor Perceived Areas 5 The invention relates to a method of updating the existence probability of a target vehicle based on sensor perceived areas . 10 Sensor fusion is a huge topic in advanced driving assistance systems (ADAS) and autonomous driving. From object detection to free-space and road estimation, fusion of information is mandatory for a consistent and precise result. Dynamic or static objects, i.e., pedestrians, bicyclists, vehicles, 15 trucks are detected and tracked by object detection fusion algorithms and the precision in detection is highly dependent on the quality of the sensors used. Algorithms that fuse sensor information are implemented so that they reach satisfactory results in certain scenarios; nevertheless, there are certain 20 scenarios and use cases not yet covered, although they must be when considering autonomy at higher levels (L3, L4) . High level fusion is the tern used for object level fusion. This approach is the lowest in complexity, but it is also 25 sometimes hard to configure and prone to errors as the sensor internal algorithms and outputs are not always trustworthy. Mid and low-level fusion uses mid or low-level information as inputs (features, point clouds or even pixels and peak points). Those algorithms are more complex to implement, that's why 30 they are preferred for low level fusion machine learning. However, for all these fusion algorithms it is mandatory to know and collect as many information possible that could contribute to better fusion output. One contribution that could generate better outputs and, in some cases, avoid late detec-5 tion is knowing the unknown: this means being aware of the space the sensors are and are not capable of detecting. In current approaches this is just limited to theoretical field of view of the sensors, but the following description will show the importance of detailed sensor perceiving area inside 10 fusion algorithms. In the following description there are used as example object detection fusion algorithms, but the underlying idea can be extrapolated to other fusion methods as well. In object detec-15 tion fusion algorithms, the term "measurements or sensor tracks" is used to describe the objects (vehicles, trucks, pedestrians, bicyclists, other) that are detected and / or tracked by sensors and "tracks or system tracks" is the term used for fusion objects, each track / object corresponding to a 20 single hypothesized target. The current approaches are using probability of existence (PoE) attached to each track that the fusion algorithm creates. The value of probability of existence for each track is increasing 25 or decreasing based on sensor inputs (if the track is identified freguently, by many sensors, the chance that the object is there gets high). When the probability of existence of a track gets high enough, the track is confirmed and will be part of the fusion output. Otherwise, when the probability of 30 existence does not pass a defined threshold, fusion does not output the track because many times the sensors are providing ghost objects that are just reflections or sensor false positives. Those should not be part of the output as they may affect the driving function decisions (for example, making a full break because of a false positive detection). There is also a lower threshold for the probability of existence that is used for track deletion. This means that if the track is no 5 longer confirmed by any sensor measurements, the track will be deleted ("killed") eventually after the probability of existence will decrease low enough. An issue arises when a track is deleted after the sensors have 10 not confirmed it and this is caused not because it is no longer there but because it is occluded, or it is not in the area visible to the sensor(s). The fact that the track is deleted too early can lead to dangerous and unwanted scenarios, since if the track will re-appear in the scene, it will take some CM 15 time until is confirmed again as a new track. This late con-firmation could have a big impact even on safety of the auton-’         omous vehicle. CO Therefore, the technical problem to be solved, in this context, 20 is to overcome the overall uncertainty related to occluded tracks due to the mechanism based on the probability of existence created by fusion algorithms. More detailed, the invention seeks answers to the following questions: how to identify occluded track in fusion algorithms; how to calculate a dynamic 25 area perceived by sensors; what occlusions to consider when calculating the dynamic perceived areas; and, finally, how to deal with occluded tracks in fusion algorithms. Therefore, the goal of the invention is to solve the deficien-30 cies of the mentioned prior art and to provide a method for computing and using a dynamic field of view of the sensors mounted on a vehicle, so that the fusion algorithms to be aware of the unknown parts (not visible areas) of this dynamic field of view and act according to this additional information. This goal is achieved according to the invention by means of 5 the technical characteristics mentioned in the independent claim, namely a method of updating the existence probability of a target vehicle based on sensor perceived areas. Further advantageous embodiments are the subject matter of the 10 dependent claims. 15 20 The subject-matter of the present invention is a method of updating the existence probability of a target vehicle, configured to be performed by means of a fusion system of an ego vehicle, an electronic control unit mounted on the ego vehicle and a multitude of sensors operatively coupled to the ego vehicle, the method comprising the next steps: receiving sensor data from the multitude of sensors; computing a dynamic perceived area for each sensor using environment data that is available to the fusion system; generating individual sensor tracks based on the received sensor data; determining whether an association between the individual sensor tracks exists and further providing associated and 25       unassociated tracks; for each unassociated track, updating a probability of existence of a target vehicle, determining if a track is visible or not, meaning determining if it is inside the dynamic perceive area of any 30        sensor as following: if the respective unassociated track is inside the dy namic perceived area of at least one sensor, then decreasing the updated probability of existence with a first factor and obtaining a probability of existence inside the dynamic perceived area; 5     - if the respective unassociated track is outside the dy namic perceived area of at least one sensor, then lowering the decrease of the updated probability of existence by applying a second factor lower than the first factor and obtaining a probability of existence outside the dynamic 10        perceived area; comparing the respective probability of existence inside or outside the dynamic perceived area, against a predefined threshold, deleting the unassociated tracks with respective proba-15       bility of existence under the predefined threshold and updating the associated tracks and the unassociated tracks with respective probability of existence above the predefined threshold The main advantages of adopting the method according to inven-20 tion consist in an efficient, timely use of track-related information (the occluded tracks are no longer dismissed too quickly). This way it is created a clear distinction between a track that is no longer detected by a sensor (although it should) and a track that is hsdden / occluded by other objects. 25 Any ambiguities caused by the difference between non-visible tracks and no track detections around the vehicle are closed and solved. Further special features and advantages of the present inven-30 tion can be taken from the following description of advantageous embodiments by way of the accompanying drawings. Figures 1 present a schematic diagram of a track management unit, part of a multi-sensor data fusion system, according to prior art; Figures 2 show several stages of computing a sensor perceived 5 area, namely: Fig.2a shows an object inside an ideal field of view of a sensor, namely an area perceived by sensor as unobstructed by any occlusion; Fig. 2b,2c illustrate the subtraction of non-visible areas 10 behind the object detected within the ideal field of view; Fig. 2d shows an example of dynamic field of view of a sensor detecting two different objects; Fig. 2e shows an illustration of a sensor field of view affected by object occlusions; 15 Fig. 3 present a slope scenario; Fig. 4 present a blocked view scenario; Fig. 5 is a schematic representation of a method of updating the existence probability of a track, according to invention. 20 Referring now to Fig. 1, there is shown a diagram of a track management unit, part of a classic fusion system from prior art. Based on raw sensor data provided by a multitude of sensors (e.g., radar, camera, acoustic, lidar sensors) and processed by a multi-sensorial object detection module, a detec- 25 tion list is supplied as input to a track management unit. The track management unit performs a series of operations such as track association, prediction, updating and deleting. The output consists into a list of confirmed tracks. 30 Figures 2 show several stages of computing a dynamic field of view of a sensor, also called a sensor perceived area, starting from an ideal (i.e. theoretical) field of view (Fig. 2a) within which an object is detected (Fig. 2b, 2c), and where non-visible areas occluded by a detected object are not anymore perceived by the sensor and, consequently, are subtracted from the ideal field of view when computing the dynamic field of view. There is also presented by Fig. 2d a dynamic field of 5 view for two detected objects, one of which may be static and the other mobile. Moreover, fig. 2e illustrates how occlusions affect a sensor field of view. 10 15 20 25 Further on, it needs to be mentioned that the dynamic representation of a sensor field of view is used to deal with occluded tracks in an object detection fusion algorithm. Further explanation is given about this dynamic representation of a sensor field of view (or sensor perceived areas and the way these areas are integrated by fusion systems). Autonomous vehicles usually have a sensor configuration that is supposed to cover as much as possible of the ego vehicle surrounding area. Object fusion algorithms collect information from every sensor of this configuration and output a set of static and dynamic objects (sensor tracks) that are present around the autonomous vehicle. Using these lists of objects - static and dynamic -the fusion algorithm infers the area that is detectable around the vehicle. Starting with a theoretical area (i.e., the ideal field of view), using polygon representation for example, the algorithm subtracts the non-visible area generated by an object occlusion as can be seen in Fig. 2b, c (the non-visible areas are created by projecting the edges of objects on the horizon from the sensor position). Along with object occlusions, road geometry influences may be 30 considered. For example, a slope of road may be read from high-definition maps to determine if the visibility is affected by it. Figures 3a. 3b, 3c illustrate a slope scenario with an ego vehicle going on a slopped road, the ego vehicle being equipped with a fusion system comprising ADAS sensors and an electronic control unit including at least one processor able to process data received from sensors and fused them by means of a fusion algorithm. At a first moment T1 (see Fig. 3a), an oncoming 5 vehicle is detected by the sensors of the ego vehicle and tracked as target (all front sensors confirm its presence, probability of existence exceeds a predefined threshold, for ex., 80%). As the ego vehicle advances, at a successive moment T2 (see Fig. 3b), the ego vehicle is not anymore able to detect 10 the target, due to the slope, the probability of existence of the target decreases and the afferent track would be abandoned (since the probability of existence drops under the predefined threshold). Furthermore, at another successive moment T3, the target re-appears in the field of view of the ego vehicle 15 sensors and front sensors start detecting again the target (see Fig. 3c) . Until the two vehicles pass one next to the other the probability of existence has not enough time to increase, therefore the target-associated track is confirmed very late or not at all. 20 Weather conditions and road infrastructure must also be considered here. Using sensor manufacturer information or data driven approaches, the fusion algorithm may infer if the field of view of one sensor is affected by external conditions 25 (weather or infrastructure, for example tunnels) . Those factors might reduce the range or azimuth of the sensor field of view and this information should be used when calculating the dynamic field of view of a sensor. 30 Related to this aspects, Figures 4a-4c show a blocked view scenario handled by a classic fusion algorithm, with a similar representation of different successive moments of time when an ego vehicle and another oncoming vehicle arrive to a junction 10 15 20 on a road from perpendicular directions. There are objects (building or other infrastructure) placed on one side of the road that obstruct, at some point, the field of view of the ego vehicle sensors. At a first moment T1 (according to Fig. 4a) , the field of view of the ego vehicle sensors is not obstructed, so the oncoming vehicle is detected and tracked as target, front and side sensors of the ego vehicle confirm its presence, the probability of existence for the target exceeds the predefined threshold. As both ego vehicle and the target advance, at a second moment T2 (as seen in Fig. 4b), the ego vehicle cannot detect the target anymore, due to the presence of the occluding objects interposed between them, the probability of existence for the target is decreased in this period and the track is abandoned right after the probability of existence dropped under the predefined threshold. From the moment the target re-appears in the field of view of the egovehicle sensors (moment T3, as seen in Fig. 4c) and front or side sensors of the ego vehicle start detecting the target again, until the target passes, its probability of existence has not enough time to increase back, therefore the track is confirmed very late - which is neither desired nor efficient. Fig. 5 shows a schematic diagram of the method of updating the existence probability of a track, according to the invention. 25 major steps: view for each available sensor (sensor pose, range and angle) that form the respective sen- f view for each available sen-information (e.g. road geometry , traffic participants, weather The inventive method has three SI: Computing an ideal field of based on all sensor properties by creating a polygon of points 30 sor's area of visibility; S2. Computing a dynamic field sor, by analysing environment limitations, static obstacles CM conditions, infrastructure and so on), detecting occlusions and creating "shadow" polygons for each occlusion in respect to each sensor (as two sensors can see the same occlusion different because of the position and angle where they are 5 mounted). These "shadow" polygons are areas behind occlusions not visible for the sensor anymore, therefore in order to compute the dynamic field of view, such non-visible areas are going to be subtracted from the respective ideal field of views for each sensor. Basically, the dynamic field of view is an 10 area dynamically perceived by sensor, and as such it is called in the following description a dynamic perceived area; S3. Including the computed dynamic perceived area as information to a track management unit of a fusion system by per-the next sub-steps: when updating sensor tracks with new sensor measureperforming an additional check for all unassociated tracks, in order to determine if a sensor track is or not or - in other words - if it is inside the perceived area or not - and updating the probability 20 of existence of each sensor track; b. If the sensor track is inside the dynamic perceived area, then decreasing the updated probability of existence with a first factor; c. If the sensor track is outside the dynamic perceived 25 area, then no intervening on the updated probability of existence or lowering the decrease by applying a second factor lower than the first factor. In this case the sensor track deleting is delayed, while an afferent confidence ellipse increases accordingly (the prediction should be made using the same mo-30 tion model used before). forming 15 ments, visible dynamic The updating method according to invention may be considered in any fusion algorithm, not only object detection. Road model fusion may apply this, but also free space estimation could have benefits by applying this method. However, while certain embodiments of the present invention 5 have been described in detail, those familiar with the art to which this invention relates will recognize various alternative designs and embodiments for practicing the invention as defined by the following claims. 13 1022 10 15 20 25

Claims

Patent claims1. A method of updating the existence probability of a target, configured to be performed by means of a fusion system of an 5 ego vehicle, an electronic control unit mounted on the ego vehicle and a multitude of sensors operatively coupled to the ego vehicle, the method comprising the next steps: receiving sensor data from the multitude of sensors, computing a dynamic perceive area for each sensor using envi-10 ronment data that is available to the fusion system,generating individual sensor tracks based on the received sensor data, wherein sensor tracks describe objects that are detected by sensors,updating sensor tracks with new sensor measurements and updat-15 ing the probability of existence of each sensor track,determining whether an association between the individual sensor tracks exists and further providing associated and unassociated tracks, wherein track means fused object, each track corresponding to a single target,20 for each unassociated track, performing an additional check, namelydetermining if the unassociated track is visible or not, meaning determining if it is inside the dynamic perceive area of any sensor as following:25     - if the respective unassociated track is inside the dynamic perceived area of any sensor, then decreasing the updated probability of existence with a first factor and obtaining a probability of existence inside the dynamic perceived area,30     - if the respective unassociated track is outside the dynamic perceived area of any sensor, then no intervening on the updated probability of existence or lowering thedecrease of the updated probability of existence by applying a second factor lower than the first factor and obtaining a probability of existence outside the dynamic perceived area,5     - comparing the respective probability of existence insideor outside the dynamic perceived area, against a predefined threshold and10deleting the unassociated tracks with respective probability of existence under the predefined threshold or updating the associated tracks and the unassociated tracks with respective probability of existence above the predefined threshold, andgenerating confirmed sensor tracks comprising updated associated tracks and updated unassociated tracks.

2. Fusion system for tracking a target, comprising:an electronic control unit mounted in a vehicle equipped with a multitude of sensors, the electronic control unit including a memory configured to store program instructions and at least one processor configured to execute the stored program instructions, wherein, upon executing the stored program instructions, the at least one processor is configured to operate as:a multi-sensor tracking unit which receives sensor data 25        from the multitude of sensors, computes a dynamic perceive area for each sensor and generates individual sensor tracks; anda track management unit which manages the individual sensor tracks and outputs confirmed tracks by employing the method of updating the existence probability of a target according to claim 1.