A Multi-sensor Information Fusion Algorithm Based on DS Evidence Theory

Abstract

The present invention discloses a multi-sensor information fusion algorithm based on DS evidence theory, wherein the algorithm performs corresponding weight assignments on different sensors in different states, and then utilizes the evidence combination method provided by DS evidence theory to realize information fusion of multiple sensors. The present invention proposes a multi-sensor fusion algorithm based on DS evidence theory. Specifically, the algorithm performs corresponding weight assignments on different sensors in different states, and then utilizes the evidence combination method provided by DS evidence theory to realize information fusion of multiple sensors, thereby improving target perception effect. More specifically, specific information fusion is performed for the two perception fusion tasks of lifecycle confidence management and target category classification management to achieve improved perception performance.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a multi-sensor information fusion algorithm based on DS evidence theory proposed by the present invention. Background Art

[0002] In recent years, autonomous vehicles have demonstrated the application of smart city life in many scenarios such as intelligent transportation, logistics distribution, and cleaning operations. Autonomous vehicles rely on various sensors to perceive the environment, and each type of sensor has its own advantages and disadvantages. LiDAR can provide accurate distance information and work well at night, but it cannot provide information such as the color of obstacles. Cameras can perceive information such as the color and texture of obstacles and can be used to classify targets, but the detection range is limited and sensitive to lighting conditions. Millimeter-wave radar can perceive obstacles at a long distance and provide speed information, but it cannot classify obstacles. Therefore, by fusing information from multiple sensors, it is possible to complement each other and improve perception performance.

[0003] At present, the tracking of targets by the autonomous driving perception system is mainly carried out by fusing information provided by different sensors. The main tasks include: target motion information fusion, target life cycle management, target category management, etc. The target motion information refers to information such as the target's speed, position, and acceleration, which is mainly obtained through the Kalman filter. The target life cycle management refers to the generation of new targets, the health assessment of tracked targets, and the disappearance of tracked targets. The target category management refers to the classification management of the target category attributes obtained by the perception sensor, including: vehicles, pedestrians, bicycles, etc.

[0004] The existing target lifecycle management is mainly based on the number of updates of sensor measurement values. Different confidence values ​​are set for different types of sensor measurement values, and the establishment of new targets and the extinction of targets are achieved by increasing or decreasing the numerical value of the tracking target confidence. However, this type of mechanism lacks the estimation of uncertainty. For example: for a certain sensor measurement value, it can be directly used to generate a new target with a certain probability, but it is uncertain whether it can be used to generate a new target with a certain probability. This situation cannot be reflected in the confidence of the target through a single numerical accumulation.

[0005] Existing target category management mainly relies on the weighted combination of single sensor detection results. However, for different sensors, under different measurement conditions, the accuracy of target category detection results is different. If only a simple rule weighting is used to determine the category of the target, the target classification accuracy will be reduced, and problems such as category switching will occur. Summary of the invention

[0006] In order to solve the above problems, the purpose of this application is to provide a multi-sensor information fusion algorithm based on DS evidence theory.

[0007] To achieve the purpose of the present invention, the present invention provides a multi-sensor information fusion algorithm based on DS evidence theory.

[0008] The algorithm is to perform corresponding weight assignments on different sensors in different states, and then use the evidence combination method provided by DS evidence theory to achieve information fusion of multiple sensors.

[0009] in,

[0010] The algorithm includes: (1) basic probability distribution and (2) combination rules,

[0011] (1) Basic probability allocation: Let Θ be an identification frame, or hypothesis space, and basic probability allocation, or BPA for short. The BPA on the identification frame Θ is a function m of 2Θ->[0,1], called the Mass function, and satisfies:

[0012] and

[0013]

[0014] Among them, A that makes m(A)>0 is called the focal element;

[0015] (2) Combination rule: The DS evidence theory combination rule is also called the evidence combination formula, which is defined as follows: The Dempster composition rule for two Mass functions m1 and m2 on Θ is:

[0016]

[0017] Where K is the normalization constant,

[0018]

[0019] in,

[0020] For target lifecycle management,

[0021] The new target generation rule is a combination of a certain Mass function allocation based on DS evidence theory and the indicators of new target generation and virtual target extinction. If the allocation of the Mass function or the evaluation indicator of the new evidence is changed, the corresponding new target generation rule will also change.

[0022] in,

[0023] For target lifecycle management,

[0024] The rule for target health assessment is to manage the health of the established targets, and use the position variance value of the tracking target filter to manage the confidence of whether there is an update of the measurement value;

[0025] The details are as follows:

[0026] (1) If the current target is associated with a reliable measurement value, then this condition can be used as evidence to improve the target confidence. This evidence can be combined with the existing confidence of the target to obtain a new confidence.

[0027] (2) If the current filter variance of the target is too large, then this condition can be used as evidence of a reduction in the target confidence. A weaker confidence reduction criterion can be used and this evidence can be combined with the target's latest confidence.

[0028] (3) If the current target is not associated with a measurement value, then a stronger confidence reduction criterion can be used to combine the target confidences to obtain a new confidence;

[0029] After all the evidence is combined, the confidence of the obtained target is judged, and the target with low confidence is deleted. Under the current Mass function allocation, if the index of deleting the target is set to m(Delete)>0.9, the target that meets the conditions can be deleted under the corresponding conditions;

[0030] (4) Similar to the principle of Mass function allocation for generating new targets, the target confidence increase and decrease Mass function allocation here is also just one possible combination. During implementation, it is necessary to adjust the Mass function according to the rules of target extinction or add more evidence.

[0031] in,

[0032] Classification management for target categories,

[0033] When fusing target category detection information from multiple sensors, it is necessary to reasonably assign confidence levels to the detection results of different sensors. The target obstacle category at time k+1 is the fusion of the target category at time k and the target category detection results of all sensors at time k+1.

[0034] The mass function allocation needs to be configured according to the actual performance of the sensor. After the obstacle categories are fused, the category with the highest output probability is set as the final obstacle target category according to the corresponding threshold value.

[0035] Compared with the prior art, the invention has the beneficial effect that the invention proposes a multi-sensor fusion algorithm based on DS evidence theory. Specifically, the algorithm performs corresponding weight assignments on different sensors in different states, and then uses the evidence combination method provided by DS evidence theory to achieve information fusion of multiple sensors, thereby improving the target perception effect.

[0036] More specifically, for the two perception fusion tasks of lifecycle confidence management and target category classification management, specific information fusion is carried out to improve perception performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Shown is the obstacle perception target category management flow chart of this application. DETAILED DESCRIPTION

[0038] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0039] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0040] The present invention proposes a multi-sensor information fusion method, which specifically refers to two tasks: target lifecycle management based on DS evidence theory, and target category management.

[0041] Specifically, the algorithm assigns corresponding weights to different sensors in different states, and then uses the evidence combination method provided by DS evidence theory to achieve information fusion of multiple sensors, thereby improving the target perception effect.

[0042] Basic working principle

[0043] As an important uncertainty reasoning theory, DS evidence theory provides a good solution for dealing with the ambiguity and uncertainty of sensor information. It not only emphasizes the objectivity of things, but also emphasizes the subjectivity of human estimation of things. Its biggest feature is that it uses "interval estimation" rather than "point estimation" to describe uncertainty information, and it shows great flexibility in distinguishing unknown and uncertain aspects and accurately reflecting evidence collection.

[0044] This patent describes the specific information fusion algorithm for the two perception fusion tasks: lifecycle confidence management and target category classification management.

[0045] (1) Basic probability distribution

[0046] Let Θ be a recognition frame, or hypothesis space. Basic Probability Assignment, referred to as BPA (Basic Probability Assignment). BPA on the recognition frame Θ is a function m of 2Θ->[0,1], called Mass function, and satisfies:

[0047] and

[0048]

[0049] Among them, A that makes m(A)>0 is called focal elements.

[0050] (2) Combination rules

[0051] The DS evidence theory synthesis rule is also called the evidence synthesis formula, which is defined as follows. The Dempster composition rule for two Mass functions m1 and m2 on Θ is:

[0052]

[0053] Where K is a normalization constant.

[0054]

[0055] 1. Target lifecycle management

[0056] New target generation

[0057] There are three common target tracking sensors in the autonomous driving perception system: LiDAR, Radar, and Vision. For the probability of generating new targets, the three types of sensors have the following Mass function allocation (simple example). In addition, Mass function allocation is also performed for the case where there is no observation in a certain frame. N / A represents an empty set, YES represents the probability of generating a new target, NO represents the probability of not generating a new target, and UNKNOWN represents uncertainty as to whether a new target will be generated.

[0058] Table 1: Mass function distribution under different evidence (sensor type and presence or absence)

[0059]

[0060] Table 1

[0061] Based on the Mass function allocation in the above table, the generation of new targets under the combination of multi-frame multi-measurement values ​​(minimum observations) is statistically analyzed, as shown in Table 2. It can be seen that the multi-class combinations can be roughly divided into six categories. Each category contains several small categories of combinations.

[0062]

[0063] Table 2

[0064] Under the current Mass function allocation, if the indicator for new target generation is set to m(YES)>0.9, and the indicator for the extinction of the fake track is set to m(NO)>0.7, as shown in Table 3, four of the six combinations can generate new targets, and two will not.

[0065] Note: When the perception fusion module obtains the measurement information of the target, it will not create a real target immediately, but will first create a virtual target (which will not be sent to the lower-level application). When the confidence level of the virtual target reaches a certain value (threshold value), the virtual target will be converted into a real target (sent to the lower-level application).

[0066]

[0067] Table 3

[0068] However, it should be noted that the new target generation rule in the figure above is based on a certain combination of Mass function allocation and indicators of new target generation and virtual target extinction in DS evidence theory. If the allocation of the Mass function or the evaluation indicator of the new evidence is changed, the corresponding new target generation rule will also change. In actual use, it is often necessary to adjust the Mass function based on existing experience to obtain a suitable new target generation rule.

[0069] Target health assessment

[0070] For the established targets, health management is performed. Here, the confidence management is performed using the position variance value of the tracking target filter, whether there is an update of the measurement value, and other evidence. The corresponding Mass function allocation is shown in Table 4 (a simple example). N / A represents an empty set, Maintain represents the probability of maintaining the target, Delete represents the probability of deleting the target, and UNKNOWN represents uncertainty about what operation to perform.

[0071]

[0072] Table 4

[0073] Newly created targets can be assigned a lower confidence level, represented by Track Level B.

[0074] (1) If the current target is associated with a reliable measurement value (determined by the confidence of the sensor measurement value), then this condition can be used as evidence to upgrade the target's confidence. The evidence Confidence Upgrade is combined with the existing confidence of the target to obtain a new confidence.

[0075] (2) If the current filter variance of the target is too large (set threshold), then this condition can be used as evidence of reduced confidence in the target. A weaker confidence reduction standard, Confidence Weaken, can be used and this evidence can be combined with the latest confidence of the target.

[0076] (3) If the current target is not associated with a measurement value, then a stronger confidence reduction standard Confidence Degrade can be used to combine the target confidences to obtain a new confidence.

[0077] After all the evidence is combined, the confidence of the obtained target is judged and the target with low confidence is deleted. Under the current Mass function allocation, if the indicator for deleting the target is set to m(Delete)>0.9, the target that meets the condition can be deleted under the corresponding conditions.

[0078] (4) Similar to the principle of mass function allocation for new target generation, the target confidence increase and decrease mass function allocation here is also just one possible combination. In the implementation process, it is necessary to adjust the mass function according to the rules of target extinction, or add more evidence. It is very flexible, but the more evidence there is, the more complicated the situation after combination, which is not conducive to the formulation and analysis of system rules.

[0079] 2. Target Category Classification Management

[0080] Autonomous driving perception systems often also need to provide category information of target obstacles, such as vehicles, people, bicycles, and unknown obstacles. Different types of sensors have different detection accuracy for target categories. Generally speaking, for target category detection, the detection accuracy of visual sensors is better than that of lidar, while millimeter-wave radar has almost no target category detection capability. Therefore, when fusing target category detection information from multiple sensors, it is necessary to reasonably assign confidence levels to the detection results of different sensors. Figure 1 This is a management flow chart for perceived obstacle target categories. The target obstacle category at time k+1 is obtained by fusion of the target category at time k and the target category detection results of all sensors at time k+1.

[0081] Similar to the algorithm used in life cycle confidence management, the target category classification management algorithm assigns Mass functions to different sensors and then fuses multiple information sources based on the DS evidence combination theory.

[0082] In this example, three commonly used types of sensors are assigned Mass functions. Taking the case where the obstacle target is a vehicle as an example, as shown in Table 5.

[0083]

[0084] Table 5

[0085] In practical applications, the above Mass function allocation needs to be configured according to the actual performance of the sensor. After the obstacle categories are fused, according to the corresponding threshold value settings, the category with the highest output probability is the final obstacle target category. The specific fusion process is not described here.

[0086] The present invention proposes a multi-sensor fusion algorithm based on DS evidence theory. Specifically, the algorithm performs corresponding weight assignments for different sensors in different states, and then uses the evidence combination method provided by DS evidence theory to achieve information fusion of multiple sensors, thereby improving the target perception effect. More specifically, for the two perception fusion tasks of lifecycle confidence management and target category classification management, specific information fusion is performed to achieve the improvement of perception performance.

[0087] It should be noted that the technical solutions not described in detail in this application adopt publicly known technologies.

[0088] The above is only a preferred embodiment of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-sensor information fusion method based on DS evidence theory, characterized in that: The method is to assign corresponding weights to different sensors in different states, and then use the evidence combination method provided by DS evidence theory to achieve information fusion of multiple sensors, specifically referring to the two tasks of target life cycle management and target category management based on DS evidence theory; The method comprises: (1) basic probability distribution and (2) combination rules, (1) Basic probability allocation: Let Θ be an identification frame, or hypothesis space, and basic probability allocation, or BPA for short. The BPA on the identification frame Θ is a function m of 2Θ->[0,1], called the Mass function, and satisfies: and Among them, A that makes m(A)>0 is called the focal element; (2) Combination rule: The DS evidence theory combination rule is also called the evidence combination formula, which is defined as follows: The Dempster composition rule for two Mass functions m1 and m2 on Θ is: Where K is the normalization constant, Target lifecycle management includes the following: First: New target generation A certain Mass function allocation combination based on DS evidence theory and the indicators of new target generation and virtual target extinction; if the allocation of Mass function or the evaluation indicator of new evidence is changed, the corresponding new target generation rule will also change; Second: Target Health Assessment For established targets, health management is performed, and confidence management is performed using the position variance value of the tracking target filter and whether there is any update of the measurement value; The details are as follows: (1) If the current target is associated with a reliable measurement value, then this condition is used as evidence to improve the target confidence, and this evidence is combined with the existing confidence of the target to obtain a new confidence; (2) If the current filter variance of the target is too large, then this condition is used as evidence of reduced confidence in the target, a weaker confidence reduction standard is used, and this evidence is combined with the latest confidence of the target; (3) If the current target is not associated with a measurement value, then a stronger confidence reduction criterion is used to combine the target confidences to obtain a new confidence; After all the evidence is combined, the confidence of the obtained target is judged, and the target with low confidence is deleted. Under the current Mass function allocation, if the index of deleting the target is set to m(Delete)>0.9, the target that meets the condition is deleted under the corresponding condition; (4) During the implementation process, it is necessary to adjust the Mass function according to the rules of target extinction, or add more evidence; Classification management for target categories, When fusing target category detection information from multiple sensors, it is necessary to reasonably assign confidence levels to the detection results of different sensors. The target obstacle category at time k+1 is the fusion of the target category at time k and the target category detection results of all sensors at time k+1. The mass function allocation needs to be configured according to the actual performance of the sensor. After the obstacle categories are fused, the category with the highest output probability is set as the final obstacle target category according to the corresponding threshold value.

Images