Multi-sensor target classification fusion method and system based on reliability evaluation

Abstract

The application provides a multi-sensor target classification fusion method and system based on reliability evaluation, and relates to the technical field of target fusion.The application provides a two-stage fusion method based on sensor reliability evaluation in an evidence framework.According to the working performance of a single sensor, representative evidence of each sensor can be obtained, and the influence of offset data on the fusion result is weakened.Meanwhile, the difference between the representative evidence of multiple sensors is compared, and the accuracy of the fusion result can be further improved.The technical problem that the target classification fusion method can cause the fusion result to be distorted due to the offset of most sensor data is solved.

Description

Technical Field

[0001] This invention relates to the field of target fusion technology, and specifically to a multi-sensor target classification and fusion method and system based on reliability assessment. Background Technology

[0002] Autonomous intelligent systems are equipped with various types of sensors and utilize perception algorithms to detect objects in the external environment, acquiring target category and state information to construct an external environment model. Target classification is one of the key tasks for autonomous intelligent systems to achieve environmental perception; it categorizes identified targets and determines the probability of them belonging to a certain category. To avoid the influence of internal sensor failures or external environmental factors, intelligent systems typically equip themselves with redundant sensors to improve reliability. For example, self-driving cars are equipped with LiDAR and camera sensors to record target images and point cloud information for object classification. Different types of sensors operate on different principles and their performance varies significantly in different scenarios. Coupled with occasional sensor failures, classification results from different sensors may conflict. Therefore, it is necessary to fuse the redundant and complementary information provided by different sensors to obtain a more accurate classification of target categories.

[0003] Existing fusion schemes mainly include data-level fusion, feature-level fusion, and decision-level fusion. Data-level fusion directly fuses raw sensor data from different modes, while feature-level fusion extracts features from the raw data and fuses cross-modal data in a feature space. These two fusion strategies are strongly coupled in their sensor information processing methods; if the sensor types and structure in the sensor system change, the fusion method will lose its effectiveness. Decision-level fusion strategies fuse the decision results of each sensor. This process separates the sensor information processing from the fusion process, offering greater flexibility. Decision-level fusion strategies better leverage the strengths of each sensor and are widely used in target classification fusion tasks.

[0004] In decision-level fusion, numerous schemes have been applied to sensor data fusion, such as voting methods, weighted averaging, and evidence theory. In these methods, a single sensor can be considered as a support for a certain decision outcome. By considering the number of supports or the impact of each support, the final decision is made by integrating data from multiple sensors. In these methods, the number of supports or the impact of each support is determined by the differences in decision outcomes among sensors; the smaller the differences with other sensor data, the higher the support level. Therefore, this majority-reliability rule is effective when most sensor data is accurate. However, due to environmental factors, sensor malfunctions, and other factors, it is not uncommon for most sensor data in a sensor system to deviate from the true value, leading to a final fusion result based on the above methods that will deviate from the true value. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides a multi-sensor target classification and fusion method and system based on reliability assessment, which solves the technical problem that the target classification and fusion method will cause the fusion result to be distorted due to the offset of data from most sensors.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0009] In a first aspect, the present invention provides a multi-sensor target classification and fusion method based on reliability assessment, comprising:

[0010] The raw information from multiple sensors within a certain time window is processed by a classification algorithm to obtain the recognition results of each sensor for the target category within the certain time window;

[0011] Based on a pre-constructed reliability criterion system, a two-stage fusion method is used to fuse the target category identification results of various sensors within a certain time window to obtain average evidence. The reliability criterion system includes sensor performance and the variability of data from different sensors. Performance includes the uncertainty and stability of each sensor, while variability includes the degree of deviation and conflict between evidence samples. The first stage of the two-stage fusion method is: correcting the target category identification results of the corresponding sensors based on their uncertainty and stability to obtain representative evidence from each sensor within a certain time window; the second stage is: calculating the average evidence based on the degree of deviation and conflict between the representative evidence from each sensor.

[0012] The average evidence is fused to obtain the final classification decision.

[0013] Secondly, the present invention provides a multi-sensor target classification and fusion system based on reliability assessment, comprising:

[0014] The data processing module is used to process the raw information from multiple sensors within a certain time window using a classification algorithm to obtain the recognition results of each sensor for the target category within the certain time window.

[0015] The two-stage fusion module is used to fuse the target category identification results of various sensors within a certain time window based on a pre-constructed reliability criterion system, to obtain average evidence. The reliability criterion system includes sensor performance and the variability of data from different sensors. Performance includes the uncertainty and stability of each sensor, while variability includes the degree of deviation and conflict between evidence samples. The first stage of the two-stage fusion method is: correcting the target category identification results of the corresponding sensors based on the uncertainty and stability of each sensor to obtain representative evidence from each sensor within a certain time window; the second stage is: calculating the average evidence based on the degree of deviation and conflict between the representative evidence from each sensor.

[0016] The average evidence fusion module is used to fuse average evidence to obtain the final classification decision result.

[0017] Thirdly, the present invention provides a computer-readable storage medium storing a computer program for a multi-sensor target classification and fusion method based on reliability assessment, wherein the computer program causes a computer to execute the multi-sensor target classification and fusion method based on reliability assessment as described above.

[0018] Fourthly, the present invention provides an electronic device, comprising:

[0019] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing a multi-sensor target classification fusion method based on reliability assessment as described above.

[0020] (III) Beneficial Effects

[0021] This invention provides a multi-sensor target classification and fusion method and system based on reliability assessment. Compared with existing technologies, it has the following advantages:

[0022] This invention first processes raw information from multiple sensors within a certain time window using a classification algorithm to obtain the recognition results of each sensor for the target category within that time window. Then, based on a pre-constructed reliability criterion system, a two-stage fusion method is used to fuse the recognition results of each sensor within the time window to obtain average evidence. The reliability criterion system includes sensor performance and the variability of data from different sensors. Performance includes the uncertainty and stability of each sensor, while variability includes the degree of deviation and conflict between evidence. The first stage of the two-stage fusion method involves correcting the recognition results of the corresponding sensors for the target category based on the uncertainty and stability of each sensor, obtaining representative evidence from each sensor within the time window. The second stage involves calculating the average evidence based on the degree of deviation and conflict between the representative evidence from each sensor. Finally, the average evidence is fused to obtain the final classification decision result. This invention proposes a two-stage fusion method based on sensor reliability assessment within an evidence framework. By assessing the performance of individual sensors, representative evidence from each sensor can be obtained, reducing the impact of offset data on the fusion result. Furthermore, comparing the differences in representative evidence from multiple sensors can further improve the accuracy of the fusion result. This solves the technical problem that the target classification fusion method suffers from distortion of fusion results due to the offset of data from most sensors. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a block diagram of a multi-sensor target classification and fusion method based on reliability assessment according to an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram showing the average probability values ​​of support category θ1 under different scenarios in Experiment 2;

[0026] Figure 3 This is a schematic diagram showing the change of the probability value of supporting category θ1 in scenario 1 of Experiment 2 over time.

[0027] Figure 4 This is a schematic diagram showing the change of the probability value of supporting category θ1 in scenario 2 of Experiment 2 over time.

[0028] Figure 5 This is a schematic diagram showing the change of the probability value of support category θ1 in scenario 3 of Experiment 2 over time. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] This application provides a multi-sensor target classification and fusion method and system based on reliability assessment, which solves the technical problem that the target classification and fusion method will cause the fusion result to be distorted due to the offset of data from most sensors, thereby improving the accuracy of the fusion result.

[0031] The technical solution in this application is to solve the above-mentioned technical problems, and the general idea is as follows:

[0032] Most reliable rules rely on indirect comparisons and combinations of sensor data, neglecting to analyze sensor reliability from the performance of individual sensors, which has limitations in practical applications. Many technological inventions have now proposed improved methods for measuring sensor reliability, including analyzing the performance of individual sensors, new methods for measuring sensor variability, or a combination of both.

[0033] In the study of sensor performance, some methods consider the sensor's performance over a past period, using the accuracy of historical decision-making results as a reliability factor. Other methods consider time-series data, assessing reliability based on stability, data dispersion, or mean time between failures (MTBF). For example, analyzing the average difference and dispersion of sensor data over a certain time span, and determining the sensor's reliability and weighting factors based on a Logistic model and coefficient of variation, solves the problem of traditional evidence-based reasoning rules failing in cases of conflicting sensor data. Still other methods determine sensor performance by analyzing the correlation between observed parameters. For instance, based on target positions detected by lidar and millimeter-wave radar, analyzing the correlation between abscissa and ordinate data, and combining stability and consistency criteria, a weighted fusion method is proposed, improving fusion accuracy in cases of sensor data offset. However, when the relationship between parameters changes multiple times over a period, such as when the target's direction of motion updates rapidly, it becomes difficult to establish a unified mathematical model to describe the dynamic and variable relationship between parameters.

[0034] From the perspective of correlating sensor data, some methods rely on optimization algorithms to obtain the reliability factors of each sensor. For example, within the evidence framework, optimization algorithms minimize the distance between the fusion result and the target output to obtain the sensor reliability factor. This method fully utilizes the complementarity of different sensors, improving classification accuracy. However, such methods struggle to adapt dynamically to environmental changes, limiting their application scenarios. Other methods determine reliability factors by comparing the differences between sensor data. For instance, the paper J. Zhao, R. Xue, Z. Dong, D. Tang, and W. Wei, "Evaluating the reliability of sources of evidence with a two-perspective approach in classification problems based on evidencetheory," Information Sciences, vol. 507, pp. 313-338, 2020, proposes a novel difference measure—decision difference—combined with a genetic algorithm to minimize the decision difference between the fusion result and the true value, thus obtaining the sensor reliability factor and improving the performance of the classification model. However, this type of method neglects the contribution of the degree of conflict in sensor preference decisions to the difference. Furthermore, some methods combine multiple criteria for sensor reliability measurement, assigning weights to each criterion and the sensor based on multi-attribute decision-making methods. In these methods, the selection of criteria, their rationality and comprehensiveness, and the correlation between criteria all affect the fusion result. As described above, existing methods all suffer from the following drawbacks:

[0035] 1. The fusion results may be distorted due to the offset of data from most sensors;

[0036] 2. The subjectivity and uncertainty of the criterion measurement method affect the accuracy of the fusion results;

[0037] 3. There may be overlaps or mutual influences between the criteria, which may affect the accuracy of the fusion results.

[0038] To address the aforementioned issues, this invention proposes a multi-sensor target classification and fusion method based on sensor reliability assessment within an evidentiary framework. This method resolves the problem of distorted fusion results caused by the offset of data from multiple sensors in target classification and fusion tasks, and improves the criteria for sensor reliability assessment.

[0039] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0040] This invention provides a multi-sensor target classification and fusion method based on reliability assessment, such as... Figure 1 As shown, the method includes:

[0041] S1. The raw information from multiple sensors within a certain time window is processed by a classification algorithm to obtain the recognition results of each sensor for the target category within the certain time window;

[0042] S2. Based on a pre-constructed reliability criterion system, the identification results of various sensors for target categories within a certain time window are fused using a two-stage fusion method to obtain average evidence. The reliability criterion system includes the working performance of the sensors and the differences in data from different sensors. Working performance includes the uncertainty and stability of each sensor, and differences include the degree of deviation and conflict between evidence bodies. The first stage of the two-stage fusion method is: correcting the identification results of the corresponding sensors for target categories based on the uncertainty and stability of each sensor to obtain representative evidence from each sensor within a certain time window; the second stage is: calculating the average evidence based on the degree of deviation and conflict between the representative evidence from each sensor.

[0043] S3. The average evidence is fused to obtain the final classification decision result.

[0044] This invention proposes a two-stage fusion method based on sensor reliability assessment within an evidentiary framework. By analyzing the performance of individual sensors, representative evidence from each sensor can be obtained, mitigating the impact of offset data on the fusion result. Furthermore, comparing the differences in representative evidence from multiple sensors further improves the accuracy of the fusion result. This addresses the technical problem of distorted fusion results in target classification fusion methods due to data offset from multiple sensors.

[0045] The following is a detailed explanation of each step:

[0046] In step S1, the raw information from multiple sensors within a certain time window is processed by a classification algorithm to obtain the recognition results of each sensor for the target category within the certain time window.

[0047] Align the operating frequencies of each sensor and collect raw data from each sensor within a certain time window.

[0048] This study uses a deep learning framework to process raw data from various sensors and obtains the recognition results (belief distribution) of each sensor for the target category. For example, the YOLO algorithm can be used for image data acquired by a camera sensor, and the CenterPoint algorithm can be used for point cloud data acquired by a LiDAR sensor. The final classification result represented by a belief distribution is as follows:

[0049] m i,t(·)={m i,t (θ1),m i,t (θ2),…,m i,t (θ 2|Θ| )}

[0050] In step S2, based on a pre-constructed reliability criterion system, the identification results of various sensors for the target category within a certain time window are fused using a two-stage fusion method to obtain average evidence. The reliability criterion system includes the working performance of the sensors and the differences in data from different sensors. Working performance includes the uncertainty and stability of each sensor, and differences include the degree of deviation and conflict between evidence samples. The first stage of the two-stage fusion method is: correcting the identification results of the corresponding sensors for the target category based on the uncertainty and stability of each sensor to obtain representative evidence from each sensor within a certain time window; the second stage is: calculating the average evidence based on the degree of deviation and conflict between the representative evidence from each sensor. The specific implementation process is as follows:

[0051] The pre-built reliability criteria system includes: assessment of the working capability of a single sensor and assessment of the differences in data from different sensors. The criteria for assessing the working capability of a single sensor include uncertainty and stability, while the criteria for assessing the differences in data from different sensors include the degree of deviation and the degree of conflict.

[0052] When data from most sensors in a sensor system deviates, especially when multiple sensor data support pseudo-classes, the fusion result derived solely from data discrepancies will deviate from the true class and fail to effectively reflect the real situation. In this embodiment of the invention, the reliability of a sensor over a period of time is related to the sensor's operating performance and the discrepancies in the sensor data. The uncertainty and stability of the data reflect the sensor's classification ability, while the discrepancies in the sensor data stem from the degree of bias and conflict. This criterion system can more effectively evaluate sensor readings.

[0053] The classification capability of a single sensor is related to the uncertainty and stability of the evidence body (i.e., the sensor's identification result of the target category). When the sensor is in a suitable operating environment, the probability of the detected target belonging to the correct category should be significantly higher than the probability of other categories; the probability of each category should not show significant deviation within a given time window. This embodiment of the invention uses information entropy and mean difference to describe the uncertainty and stability of the evidence body.

[0054] Assume A is the body of evidence. Let |A| be a focal element in the framework Θ, and |Θ| be the cardinality of focal element A. Let |Θ| be the number of elements in the framework Θ. The uncertainty of the evidence body at time t is... The definition is as follows:

[0055]

[0056] If the focal element contains only one element, i.e., |A| = 1, then the belief entropy degenerates into the Shannon entropy, defined as follows:

[0057]

[0058] Sensor stability is typically reflected by the degree of data fluctuation within a certain time window. Greater fluctuation indicates higher data instability, suggesting that the sensor's observations are affected by external or internal factors, and consequently, lower sensor reliability. Assuming a time window length of l, a criterion for measuring the instability of the evidence body can be defined by comparing the average difference between the fundamental probabilities assigned to each focal element at the current moment and those at previous moments.

[0059]

[0060] Assessment of the differences in data from different sensors:

[0061] The variability in multi-sensor data stems from the bias and conflict between the data. From a global variability perspective, the overall bias between the basic probabilities assigned to each focal element reflects the degree of bias between evidence bodies. From a local variability perspective, the differences between focal elements assigned the highest basic probabilities indicate different decisions regarding evidence body preferences, reflecting the degree of conflict between evidence bodies. Combining both approaches provides a more comprehensive description of the variability between evidence bodies.

[0062] A focal element containing multiple elements increases the computational burden of the degree of bias. To reduce the computational burden and better represent the degree of bias between two pieces of evidence, the basic probabilities are first converted into pignistic probabilities, resulting in a vector composed of pignistic probabilities:

[0063] P = (p(θ1), p(θ2)...p(θ)) n ))

[0064] Where, θ i ∈Θ, Θ={θ1, θ2...θ n}, i = 1, 2...n.

[0065] Therefore, the deviation between two pieces of evidence is transformed into the deviation between two vectors. There are many methods for comparing the deviations between vectors. For example, the Tanimoto coefficient:

[0066]

[0067] in, It is a vector The norm of .

[0068] If there are n pieces of evidence, then use Define the average degree of deviation between evidence body i and the remaining n-1 evidence bodies:

[0069]

[0070] The conflict coefficient is defined as the product of the basic probabilities corresponding to incompatible focal elements in two pieces of evidence.

[0071]

[0072] If there are n pieces of evidence, use Define the average degree of conflict between evidence body i and the remaining n-1 evidence bodies:

[0073]

[0074] It should be noted that the above criteria are all for minimization. The lower the score of the sensor on the above criteria, the higher the reliability of the sensor.

[0075] In this embodiment of the invention, the first stage involves correcting the recognition results of the corresponding sensors for the target category based on the uncertainty and stability of each sensor, thereby obtaining representative evidence from each sensor within a certain time window. Specifically, this includes:

[0076] S201a. Based on the identification results and the reliability criteria, calculate the score of each sensor on uncertainty and stability at each moment within a certain time window, and construct a score matrix.

[0077] include:

[0078] Assuming the sliding time window has a length of l, and there are M sensors in the system, the belief distribution of each sensor at a certain time t is as follows: Each component represents the probability of support for each category. In the evidence framework, m i,t (·) is considered as a piece of evidence, i = 1, 2, ... M.

[0079]

[0080] The uncertainty and stability of sensor data reflect its performance. The scores for uncertainty (C1) and stability (C2) of each sensor at each time step are calculated, and the score matrix is ​​constructed as follows. It should be noted that the scores are minimized here; the smaller the score, the higher the sensor's performance.

[0081]

[0082] Where N represents the number of criteria, and here N = 2.

[0083] S201b, Calculate the weights of uncertainty and stability based on the score matrix. This includes:

[0084] Step 1: Normalize the score matrix using Max-Min-Scaling to ensure that the scores are defined on the same measurement scale.

[0085]

[0086] and This indicates that within the time window l, sensor m i In criterion C k Score The maximum and minimum values ​​of , where k = 1, ..., N.

[0087] The normalized score matrix is ​​shown below.

[0088]

[0089] Step 2: Construct the criterion comparison matrix. For a pair of criteria C k and C h (i.e., uncertainty and stability), C k Relative to C h The importance coefficient is expressed by the following formula:

[0090]

[0091] Normalization of importance coefficients:

[0092]

[0093] The criterion comparison matrix is ​​as follows:

[0094]

[0095] Step 3: Finally, Criterion C k The weight calculation results are as follows:

[0096]

[0097] S201c. Calculate the reliability factor of the sensor readings at each time step based on the weights of uncertainty and stability, and determine the representative evidence for each sensor based on the reliability factor and the identification results.

[0098] include:

[0099] Step 1: The score matrix here is minimized. Based on the following formula, the score matrix is ​​first transformed into a maximization matrix, and then standardized.

[0100]

[0101] Step 2: Calculate the overall score, sensor m i At time t j The overall score is

[0102]

[0103] Step 3: Sensor m i At time t j The reliability factor is

[0104]

[0105] Therefore, the representative evidence of the sensor within a time window l for

[0106]

[0107] The representative evidence from each sensor within a time window l.

[0108]

[0109] Where M is the number of sensors.

[0110] Real-time calculation of sensor reliability based on the scoring matrix can effectively correct offset data that occurs within the time window, providing a more reliable data source for further fusion.

[0111] If a sensor performs poorly within a specified time window, the representative evidence obtained after the first stage of uncertainty and stability correction may still have a certain degree of bias. In this case, a second stage of processing is needed to measure the degree of deviation and conflict among the representative evidence, and to further strengthen the fusion process.

[0112] In this embodiment of the invention, the second stage involves calculating the average evidence based on the degree of deviation and conflict among the representative evidence from each sensor. Specifically, this includes:

[0113] Using the same procedure described above, calculate the score matrices representing the evidence on bias (C3) and conflict (C4) to obtain the relative weights of each criterion. and the reliability factor representing evidence

[0114] The final average evidence obtained is:

[0115]

[0116] In step S3, the average evidence is fused to obtain the final classification decision result. The specific implementation process is as follows:

[0117] The Murphy method is used to fuse evidence M-1 times on average, and the class with the highest support probability is the final classification decision, where M is the number of sensors. It should be noted that in practice, other fusion methods can also be used to fuse the evidence, such as the Dempster rule and the Frikha method.

[0118] The effectiveness of the embodiments of the present invention is verified below through experimental data:

[0119] Experiment 1:

[0120] The fusion scheme provided in this embodiment of the invention is applied to the target classification problem in an autonomous driving scenario. In this case, the recognition framework composed of categories is Θ={θ1,θ2,θ3}. The system has three sensors that continuously track a target for a period of time and return the corresponding classification results (belief distribution) in real time, as shown in Tables 1, 2 and 3.

[0121] Table 1. Belief distribution of sensor m1

[0122]

[0123] Table 2. Belief distribution of sensor m2

[0124]

[0125] Table 3. Belief distribution of sensor m3

[0126]

[0127] Analyzing the data in Tables 1, 2, and 3, it can be seen that the target detected by the three sensors should be classified as category θ1. However, the data output by sensor m2 at time t5 and by sensor m3 at times t4 and t5 are offset, incorrectly classifying the target as category θ2. Next, with a time window length of l = 3, a two-stage fusion method is used to fuse the sensor data within the time window [t3, t4, t5].

[0128] (1) Calculate the score matrix of each sensor.

[0129] Calculate the scores of each sensor at times t3, t4, and t5 on uncertainty (C1) and stability (C2), and construct the score matrices as shown in Table 4(a)(b)(c).

[0130] Table 4(a) Score matrix of sensor m1

[0131]

[0132] Table 4(b) Score matrix of sensor m2

[0133]

[0134] Table 4(c) Score matrix of sensor m3

[0135]

[0136] As can be seen from Tables 4(a)(b)(c), the scores of the sensors are ranked differently on different criteria, which also shows that the performance of sensors needs to be considered from multiple criteria.

[0137] (2) Calculate the relative weights of each criterion.

[0138] After normalizing the score matrix based on Max-Min-Scaling, the criterion comparison matrix can be further obtained, and the relative weights of each criterion are shown in Table 5.

[0139] Table 5 Weights of Criteria C1 and C2

[0140]

[0141]

[0142] (3) Calculate the representative evidence for each sensor.

[0143] Taking sensor m3 as an example, this section explains the reliability factor of the evidence body at each time point in the time window [t3, t4, t5] and the calculation of the representative evidence. The score matrix of m3 in Table 4(c) is standardized as shown in Table 6.

[0144] Table 6. Standardized score matrix of sensor m3

[0145]

[0146] The overall score of sensor m3 at time t3 is S(m 3,t3 = 0.2899, and similarly, the comprehensive score S(m) of sensor m3 at times t4 and t5 can be obtained. 3,t4 ) and S(m 3,t5After normalizing the overall score, the reliability factors of the evidence at each time point [t3, t4, t5] for sensor m3 are shown in Table 7. Observing the data in Table 7, the data output by sensor m1 did not shift during the time window, therefore the reliability factors of the evidence at each time point are roughly the same. However, the data output by sensor m2 shifted at time t5. Based on the uncertainty and stability measures of sensor performance, its performance at time t5 was poor, therefore the reliability factor of the evidence at that time was relatively low. Similarly, the data output by sensor m3 shifted at times t4 and t5, and its reliability factor was lower than that at time t3.

[0147] Table 7 Reliability factors of each sensor at times t3, t4, and t5

[0148]

[0149] Finally, representative evidence m' of sensor m3 within the time window [t3,t4,t5] is obtained. 3,t5 Using the same procedure, representative evidence for sensors m1 and m2 can also be obtained, as shown in Table 8:

[0150] Table 8. Representative evidence for each sensor

[0151]

[0152] (4) Calculate and integrate average evidence

[0153] The second-stage processing flow is similar to the first stage, except that the data object is representative evidence from each sensor within a time window, and the measurement criteria are bias (C3) and conflict (C4). The reliability factors for each representative piece of evidence and the calculation results for the average evidence are shown in Tables 9 and 10. Observing the data in Table 9, the representative evidence m' 3,t5 With m' 1,t5 and m' 2,t5 Although both support the same category θ1, m' 3,t5 Compared to the other two representative pieces of evidence, it shows significant differences, and therefore its reliability factor is relatively low.

[0154] Table 9 Reliability factors for representative evidence

[0155]

[0156] Table 10 Average Evidence

[0157]

[0158] Using the Murphy method to analyze the average evidence m' t5 The fusion results of each sensor at time t5 after combining the two sensors are shown in Table 11.

[0159] Table 11 Comparative Experiment Results

[0160]

[0161]

[0162] The methods in Table 11 are represented as follows:

[0163] Dempster rule: Use Dempster's combination rule to fuse evidence at the last moment of the time window.

[0164] Murphy's method: First, process the evidence at the last moment of the time window using the simple averaging method to obtain average evidence, and then combine the average evidence M-1 times.

[0165] The Frikha method involves using a multi-attribute decision method to obtain average evidence for the evidence body at the last moment of the time window, and then combining the average evidence M-1 times.

[0166] Dempster discount method: First, use the simple averaging method to process the evidence within the time window to obtain representative evidence, and then use Dempster's combination rule to merge the representative evidence.

[0167] Using only the first stage of fusion: First, process the evidence within the time window using the method of the first stage of the embodiment of the present invention to obtain representative evidence, then obtain average evidence based on the simple averaging method, and finally combine the average evidence M-1 times.

[0168] Using only the second-stage fusion: First, the evidence within the time window is processed using a simple averaging method to obtain representative evidence. Then, average evidence is obtained based on the second-stage method of the present invention. Finally, the average evidence is combined M-1 times.

[0169] Based on the comparison results in the table, the following conclusions can be drawn: Since most sensor data in the sensor system is offset, when using the Dempster rule to fuse the evidence body, most confidence values ​​will be assigned to the offset data, i.e., focal element θ2. The Murphy method ignores the impact of sensor reliability on the fusion result, and the simple averaging method for processing the evidence body is not ideal. The Frikha method introduces multiple criteria to measure sensor reliability, but the correlation between the criteria may interfere with the fusion result. The above methods fail to effectively handle the problem of distorted fusion results caused by most offset data. The remaining methods, by introducing the concept of a time window and analyzing the characteristics of sensor data within the time window, obtain more ideal fusion results. Compared to the Dempster discount method, introducing corresponding criteria to measure the real-time reliability of the sensor in the first and second stages of fusion, instead of a simple averaging method, yields more persuasive and accurate fusion results. Furthermore, using only the first or second stage fusion method results in a lower support probability for the classification decision than the two-stage fusion method, indicating that the two-stage fusion method is more comprehensive in analyzing sensor reliability.

[0170] Experiment 2:

[0171] To further verify the effectiveness of the proposed solution in this embodiment of the invention, Experiment 2 simulated scenarios where the data from three sensors exhibited different degrees of shift, and observed the magnitude of the probability value supporting the final classification decision. In the simulated sensor system, each sensor tracked and identified a target over time spans of t1, t2, ..., t... 20 Calculate and record t 11 , t 12 , ..., t 20 The fusion results at each time step. Sensor m1 operates normally, while sensors m2 and m3 perform poorly, and their data are prone to shift. The recognition framework composed of categories is {θ1, θ2, θ3}. The belief distribution of the sensor outputs at each time step is generated by random numbers that meet certain conditions, as shown in Table 12. The shift rates of the sensor data under the three scenarios are 20%, 30%, and 40%, respectively.

[0172] Table 12 Belief distribution of each sensor under different scenarios

[0173]

[0174] Among them, x, y, z∈[-0.10,0.10], x+y+z=0.

[0175] Figure 2The average probability values ​​supporting the classification decision result θ1 are shown under different scenarios. Even in scenarios with high offset rates for sensors m2 and m3, the average probability value obtained using the two-stage fusion method proposed in this embodiment is greater than 0.7, which is highly convincing for the target to belong to category θ1. Scenario 1 is when the offset rate of sensors m2 and m3 is 20%; Scenario 2 is when the offset rate of sensors m2 and m3 is 30%; and Scenario 3 is when the offset rate of sensors m2 and m3 is 40%.

[0176] Furthermore, Experiment 2 compared the impact of different time window lengths on the fusion results. In scenarios with low sensor offset rates, the length of the time window had a relatively small impact on the fusion results. For example... Figure 3 When the offset rate of sensors m2 and m3 is 20%, the support probabilities of the fusion results obtained using time window lengths of 3, 4, 5, and 6 are not significantly different. However, when the offset rate is high, the shorter the time window, the greater the impact of the offset data on the representative evidence generated in the first stage of the fusion process. This results in the representative evidence still having a certain degree of deviation, ultimately leading to a relatively low support probability in the fusion result. Figure 4 and Figure 5 .

[0177] This invention also provides a multi-sensor target classification and fusion system based on reliability assessment, comprising:

[0178] The data processing module is used to process the raw information from multiple sensors within a certain time window using a classification algorithm to obtain the recognition results of each sensor for the target category within the certain time window.

[0179] The two-stage fusion module is used to fuse the target category identification results of various sensors within a certain time window based on a pre-constructed reliability criterion system, to obtain average evidence. The reliability criterion system includes sensor performance and the variability of data from different sensors. Performance includes the uncertainty and stability of each sensor, while variability includes the degree of deviation and conflict between evidence samples. The first stage of the two-stage fusion method is: correcting the target category identification results of the corresponding sensors based on the uncertainty and stability of each sensor to obtain representative evidence from each sensor within a certain time window; the second stage is: calculating the average evidence based on the degree of deviation and conflict between the representative evidence from each sensor.

[0180] The average evidence fusion module is used to fuse average evidence to obtain the final classification decision result.

[0181] It is understood that the multi-sensor target classification and fusion system based on reliability assessment provided in this embodiment of the invention corresponds to the multi-sensor target classification and fusion method based on reliability assessment described above. The explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding content in the multi-sensor target classification and fusion method based on reliability assessment, and will not be repeated here.

[0182] This invention also provides a computer-readable storage medium storing a computer program for multi-sensor target classification fusion based on reliability assessment, wherein the computer program causes a computer to execute the multi-sensor target classification fusion method based on reliability assessment as described above.

[0183] This invention also provides an electronic device, comprising:

[0184] One or more processors;

[0185] Memory; and

[0186] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing a multi-sensor target classification fusion method based on reliability assessment as described above.

[0187] In summary, compared with existing technologies, it has the following beneficial effects:

[0188] 1. This invention proposes a two-stage fusion method based on sensor reliability assessment within an evidentiary framework. By analyzing the performance of individual sensors, representative evidence from each sensor can be obtained, reducing the impact of offset data on the fusion result. Simultaneously, comparing the differences in representative evidence from multiple sensors further improves the accuracy of the fusion result, solving the technical problem that target classification fusion methods suffer from distorted fusion results due to offset data from multiple sensors.

[0189] 2. In the embodiments of the present invention, the weights of each criterion and the reliability factor of the sensor are automatically calculated based on the score matrix, eliminating subjectivity and further improving the accuracy of the fusion results.

[0190] 3. Based on the feature analysis of sensor data, the embodiments of the present invention summarize a set of criteria (uncertainty, stability, degree of deviation and degree of conflict) for real-time evaluation of sensor reliability. These criteria reflect the reliability of the sensor from different perspectives, have low correlation, reduce the possibility of overlap or mutual influence between criteria, and thus further improve the accuracy of fusion results.

[0191] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0192] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-sensor target classification and fusion method based on reliability assessment, characterized in that, include: The raw information from multiple sensors within a certain time window is processed by a classification algorithm to obtain the recognition results of each sensor for the target category within the certain time window; Based on a pre-constructed reliability criterion system, a two-stage fusion method is used to fuse the target category identification results of various sensors within a certain time window to obtain average evidence. The reliability criterion system includes sensor performance and the variability of data from different sensors. Performance includes the uncertainty and stability of each sensor, while variability includes the degree of deviation and conflict between evidence samples. The first stage of the two-stage fusion method is: correcting the target category identification results of the corresponding sensors based on their uncertainty and stability to obtain representative evidence from each sensor within a certain time window; the second stage is: calculating the average evidence based on the degree of deviation and conflict between the representative evidence from each sensor. The average evidence is fused to obtain the final classification decision.

2. The multi-sensor target classification and fusion method based on reliability assessment as described in claim 1, characterized in that, The process of correcting the target category identification results of corresponding sensors based on the uncertainty and stability of each sensor to obtain representative evidence from each sensor within a certain time window includes: S201a. Based on the identification results and the reliability criteria, calculate the score of each sensor on uncertainty and stability at each moment within a certain time window, and construct a score matrix. S201b, Calculate the weights of uncertainty and stability based on the score matrix; S201c. Calculate the reliability factor of the sensor readings at each time step based on the weights of uncertainty and stability, and determine the representative evidence for each sensor based on the reliability factor and the identification results.

3. The multi-sensor target classification and fusion method based on reliability assessment as described in claim 2, characterized in that, Based on the criteria system of recognition results and reliability, the score of each sensor on uncertainty and stability at each moment within a certain time window is calculated, and a score matrix is ​​constructed, including: The sliding time window has a length of l, and there are M sensors in the system. The belief distribution output by each sensor at a certain time t is as follows: Each component represents the probability of support for each category. In the evidence framework, m i,t (·) is considered a piece of evidence, i = 1, 2, ... M; Calculate the score of each sensor at each time step on uncertainty C1 and stability C2, and construct the score matrix as follows: Where: N represents the number of criteria. Each sensor minimizes its score on uncertainty C1 and stability C2. The smaller the score, the higher the sensor's performance.

4. The multi-sensor target classification and fusion method based on reliability assessment as described in claim 2, characterized in that, The calculation of the weights for uncertainty and stability based on the score matrix includes: Step 1: Normalize the score matrix based on Max-Min-Scaling so that the scores are defined under the same measurement scale, as shown in the following expression: in, and This indicates that within the time window l, sensor m i In criterion C k Score The maximum and minimum values, where k = 1, ..., N; The normalized score matrix is ​​shown below. Step 2: Construct the criterion comparison matrix, for a pair of criterion uncertainties C k and stability C h C k Relative to C h The importance coefficient is expressed by the following formula: Normalization of importance coefficients: The criterion comparison matrix is ​​as follows: Step 3: Criterion C k The weight calculation results are as follows:

5. The multi-sensor target classification and fusion method based on reliability assessment as described in claim 2, characterized in that, The process of calculating the reliability factor of sensor readings at each time step based on the weights of uncertainty and stability, and determining representative evidence for each sensor based on the reliability factor and the identification results, includes: Step 1: Based on the following formula, first transform the score matrix into a maximal form, then standardize it to obtain the standardized score. Step 2: Calculate the overall score based on the weights and standardized scores, for sensor m. i At time t j The overall score is: Step 3: Sensor m i At time t j The reliability factor is: Representative evidence of the sensor within a time window l for: The representative evidence from each sensor within a time window l. Where M is the number of sensors.

6. The multi-sensor target classification and fusion method based on reliability assessment as described in any one of claims 1 to 5, characterized in that, The fusion of average evidence includes: The average evidence is fused M-1 times based on the Murphy method, where M is the number of sensors.

7. The multi-sensor target classification and fusion method based on reliability assessment as described in any one of claims 1 to 5, characterized in that, The process of classifying the raw information from multiple sensors within a certain time window yields the recognition results of each sensor for the target category within that time window, including: Align the operating frequencies of each sensor and collect raw data from each sensor within a certain time window; The system uses a deep learning framework to process the raw data from each sensor and obtain the recognition results of each sensor for the target category.

8. A multi-sensor target classification and fusion system based on reliability assessment, characterized in that, include: The data processing module is used to process the raw information from multiple sensors within a certain time window using a classification algorithm to obtain the recognition results of each sensor for the target category within the certain time window. The two-stage fusion module is used to fuse the target category identification results of various sensors within a certain time window based on a pre-constructed reliability criterion system, to obtain average evidence. The reliability criterion system includes sensor performance and the variability of data from different sensors. Performance includes the uncertainty and stability of each sensor, while variability includes the degree of deviation and conflict between evidence samples. The first stage of the two-stage fusion method is: correcting the target category identification results of the corresponding sensors based on the uncertainty and stability of each sensor to obtain representative evidence from each sensor within a certain time window; the second stage is: calculating the average evidence based on the degree of deviation and conflict between the representative evidence from each sensor. The average evidence fusion module is used to fuse average evidence to obtain the final classification decision result.

9. A computer-readable storage medium, characterized in that, It stores a computer program for a multi-sensor target classification and fusion method based on reliability assessment, wherein the computer program causes a computer to execute the multi-sensor target classification and fusion method based on reliability assessment as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing the multi-sensor target classification fusion method based on reliability assessment as described in any one of claims 1 to 7.

Images