Multi-source data fusion method, system, electronic device and storage medium

Abstract

The present invention relates to a multi-source data fusion method, system, electronic device and storage medium. The fusion method comprises: aligning the timestamps of measurement data to be fused with existing valid fusion targets; performing weighted summation of the Mahalanobis distance of the position and the Euclidean distance of the velocity between the measurement data and the aligned valid fusion targets to obtain a weighted distance, and calculating an association threshold range and an association probability between the measurement data and the aligned valid fusion targets based on the weighted distance; judging whether the measurement data is associated and matched to any valid fusion target to generate a matching pair based on the association threshold range and the association probability; for each matching pair, updating a filtering algorithm in the valid fusion target using the measurement data; performing weighted calculation of the Mahalanobis distance of the position vector and the Euclidean distance of the velocity vector between the fusion target and the sensor measurement, and calculating the joint association probability between the fusion target and the sensor measurement based on the weighted distance, thereby improving the association matching accuracy.

Description

Technical Field

[0001] The present invention relates to the field of sensor perception fusion, and in particular to a multi-source data fusion method, system, electronic device and storage medium. Background Art

[0002] Sensor perception fusion is one of the most critical components of intelligent assisted driving systems, crucial for their reliability. Most market-leading automotive OEMs have invested significant resources in developing sensor fusion modules to ensure the accuracy and real-time nature of the resulting perception results. Furthermore, in recent years, numerous academic research teams have been continuously researching and developing sensor fusion algorithm models with higher precision and wider applicability.

[0003] Currently, mainstream perception fusion technologies can be roughly divided into three categories: data-level fusion, feature-level fusion, and decision-level fusion. Data-level fusion utilizes the features of raw sensor data as input, often relying on deep learning algorithms. Feature-level fusion utilizes representative features extracted from sensor data as input, including features such as edges, direction, speed, and shape. Decision-level fusion utilizes the results of sensor data processed through specific algorithms as input, achieving results that exceed the accuracy of single-sensor perception. Because decision-level fusion technology consumes less computing power and significantly improves accuracy, it is increasingly favored by some in-vehicle platforms that require less computing power and operate efficiently.

[0004] In practical applications, the overall algorithmic framework of decision-level fusion algorithms is largely similar: after target parsing, the resulting measurement targets from different sensor data are fused through a filtering algorithm to produce a fused target. When sensors output multiple targets, a data association algorithm can be used to match the measurement targets with the fused targets, updating existing fused targets or establishing new ones to optimize the accuracy of the fusion results. It's easy to see that decision-level fusion primarily involves data association and filtering algorithms.

[0005] There are several mainstream data association algorithms: the Hungarian algorithm (also known as the KM algorithm, a nearest neighbor data association method), the joint probabilistic data association method (also known as the JPDA algorithm), the multi-hypothesis multi-target tracking data association method (also known as the MHT algorithm), and the maximum flow algorithm (also known as the MaxFlow algorithm). The Hungarian algorithm runs the fastest, but its fusion effect on targets of similar distances is poor; the MHT algorithm is the most effective, but it requires judging the probability of each association in multiple consecutive frames, resulting in its computing power requirements increasing exponentially with the number of frames; the JPDA algorithm can meet the real-time requirements of the algorithm while considering multiple association hypotheses, and is therefore used in many assisted driving systems. Considering the convenience of actual calculations, the joint probabilistic data association algorithm (i.e., the JPDA algorithm) usually uses the simplified joint probabilistic data association algorithm (also known as the CJPDA algorithm) to calculate the association probability.

[0006] In perception fusion algorithms for vision and millimeter-wave radar, the target data provided by the sensor perception algorithm typically includes relative position and velocity information. When applying a joint probabilistic data association algorithm, the algorithm typically uses the motion state vector composed of relative position and relative velocity, combined with the covariance matrix calculated from the tracked target, to calculate the associated Mahalanobis distance. This Mahalanobis distance is then used to calculate the association threshold range and association probability. This approach often leads to a problem:

[0007] That is, when the frequency of sensor data transmission is limited and the vehicle makes a sharp turn, the measurement that should be associated with the fusion target will fall outside the association threshold calculated using the Mahalanobis distance due to the sudden change in the velocity vector, resulting in the measurement being unable to be associated with the fusion target.

[0008] For example, if the sensor transmits target detection information at a frequency of no more than 10Hz and the vehicle suddenly makes a sharp 90-degree turn, the sensor's measured target will have a large lateral velocity component, causing the Mahalanobis distance between the measurement and the corresponding fused target to exceed a preset threshold, resulting in a mismatch between the measurement and the fused target. This situation will trigger a continuous jump in the fused target ID, severely limiting the accuracy of the fusion result.

[0009] Currently, research approaches in industry and academia addressing these issues include focusing on improving the overall robustness of perception fusion systems using certain data association algorithms, or optimizing known data association algorithms for specific scenarios. For example, the technical solution disclosed in patent CN112285700A utilizes a dual-wave gate IMM-JPDA data association and filtering algorithm to improve the accuracy of perception fusion systems based on lidar and millimeter-wave radar. In patents CN106872955A and CN101783020B, improvements are made to the joint probabilistic data association algorithm (JPDA) based on the specific characteristics of radar multi-target tracking and video multi-target tracking, enabling this association algorithm to improve accuracy in these scenarios.

[0010] Among these solutions and academic research, patent CN112285700A primarily explores the benefits of combining the joint probabilistic data association (JPDA) algorithm and the interactive multi-model (IMM) filtering algorithm for lidar and millimeter-wave radar perception fusion. Patent CN106872955A optimizes the JPDAA and Kalman filtering processes to more accurately track multiple radar targets with intersecting trajectories. Patent CN101783020B, similar to patent CN106872955A, optimizes the JPDAA and Kalman filtering processes to improve target tracking accuracy in image and video data. It can be seen that all of these studies utilize or optimize the JPDAA algorithm to achieve accuracy in perception fusion systems or multi-target tracking systems under scenarios where the sensor data transmission frequency is normal and the ego vehicle does not make sudden changes in direction. Under conditions where the sensor data transmission frequency is limited and the ego vehicle makes sharp turns, these algorithms can result in frequent ID jumps in the fusion or tracking results, affecting model accuracy. Summary of the Invention

[0011] The present invention aims to solve the technical problems existing in the prior art and provides a multi-source data fusion method to improve the accuracy of the association matching between the fusion target and the sensor measurement.

[0012] According to a first aspect of the present invention, a multi-source data fusion method is provided, wherein the multi-source data includes at least two types of measurement data from different sensors, and the measurement data each includes position information and speed information. The fusion method includes:

[0013] Step 1: align the timestamps of the measurement data to be fused with the timestamps of the existing valid fusion targets;

[0014] Step 2: performing a weighted summation of the Mahalanobis distance of the position and the Euclidean distance of the velocity between the measurement data and the aligned effective fusion target to obtain a weighted distance, and calculating an association threshold range and an association probability between the measurement data and the aligned effective fusion target based on the weighted distance;

[0015] Step 3: Determine whether the measurement data is associated and matched to any of the valid fusion targets to generate a matching pair based on the association threshold range and association probability; for each matching pair, use the measurement data to update the filtering algorithm in the valid fusion target; for unmatched measurement data, use the unmatched measurement data as the initialization condition of the filtering algorithm to create a new fusion target.

[0016] On the basis of the above technical solution, the present invention can also make the following improvements.

[0017] Optionally, step 1 includes:

[0018] After obtaining the measurement data to be fused at time t, the position and velocity information of each of the effective fusion targets at time t is predicted using the motion state equation.

[0019] Optionally, the calculation formula of the weighted distance is:

[0020]

[0021] Among them, α and β are adjustable weight values. and are the position vector and velocity vector of the nth effective fusion target at time t; and are the position vector and velocity vector of the mth measurement data at time t respectively; is the position covariance of the nth effective fusion target at time t.

[0022] Optionally, the association threshold range G between the measurement data and the aligned valid fusion target calculated based on the weighted distance is:

[0023]

[0024] Optionally, the association probability between the measurement data and the aligned valid fusion target is calculated based on the weighted distance as follows:

[0025]

[0026] in, represents the association probability between the mth measurement data and the nth effective fusion target at time t;

[0027]

[0028]

[0029] p D represents the sensor detection probability, M represents the total number of measurement data at time t, and λ is the Poisson distribution parameter.

[0030] Optionally, step 3 includes:

[0031] The measurement data and the valid fusion target are classified into: a matching pair of measurement data and a valid fusion target, unmatched measurement data, and an unmatched valid fusion target according to a matching result between the measurement data and the valid fusion target.

[0032] Optionally, after step 3, the following steps may be further performed:

[0033] Perform target lifecycle management on all fusion targets, filter and output the valid fusion targets.

[0034] According to a second aspect of the present invention, a multi-source data fusion system is provided, wherein the multi-source data includes at least two types of measurement data from different sensors, and the measurement data both include position information and speed information. The fusion system includes: a parsing module, an association matching module, and a filtering update module;

[0035] The parsing module is used to align the timestamps of the measurement data to be fused with the timestamps of the existing valid fusion targets;

[0036] The association matching module is configured to obtain a weighted distance by performing a weighted summation on the Mahalanobis distance of the position and the Euclidean distance of the velocity between the measurement data and the aligned effective fusion target, and calculate an association threshold range and an association probability between the measurement data and the aligned effective fusion target based on the weighted distance;

[0037] The filter update module is used to determine whether the measurement data is associated and matched to any of the valid fusion targets to generate a matching pair based on the association threshold range and the association probability; for each matching pair, the measurement data is used to update the filtering algorithm in the valid fusion target; for unmatched measurement data, the unmatched measurement data is used as the initialization condition of the filtering algorithm to create a new fusion target.

[0038] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the processor is configured to implement the steps of the multi-source data fusion method when executing a computer management program stored in the memory.

[0039] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer management program is stored. When the computer management program is executed by a processor, the steps of the multi-source data fusion method are implemented.

[0040] The present invention provides a multi-source data fusion method, system, electronic device and storage medium. When the frequency of sensor data transmission is limited, the sudden turn of the vehicle causes all perceived targets within the sensor's field of view to have a large lateral velocity, resulting in the joint probability data association algorithm based on the elliptical threshold being unable to associate and match the correct fusion target and measurement ratio pairing. This effectively solves the problem of continuous ID jumps caused by the traditional joint probability association algorithm under the above problem. The improved algorithm can ensure that the output of the fusion target is stable when the vehicle's speed suddenly changes while the association accuracy remains unchanged in ordinary scenes. In addition, the introduction of weighted distance does not increase the overall computational complexity of the algorithm. Therefore, the algorithm proposed in this technical solution can still maintain efficiency and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A structural diagram of a multi-source data fusion system provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart of a multi-source data fusion method provided by an embodiment of the present invention;

[0043] Figure 3 An example diagram of a method for defining associated events in a joint probabilistic association algorithm provided by an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of two measurement data and two valid fusion targets in a two-dimensional plane at two adjacent moments provided by an embodiment of the present invention;

[0045] Figure 5 A schematic diagram comparing the fusion of a traditional joint probabilistic data association algorithm and the association algorithm provided by the present invention provided in an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0047] Figure 7 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0048] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0049] The embodiment of the present invention provides a multi-source data fusion method, wherein the multi-source data includes at least two types of measurement data from different sensors, and the measurement data includes position information and speed information, such as Figure 1 FIG. 1 is a structural diagram of a multi-source data fusion system provided by an embodiment of the present invention. Figure 2 The figure shows a flow chart of a multi-source data fusion method provided by an embodiment of the present invention, combined with Figure 1 and Figure 2 It can be seen that the fusion methods include:

[0050] Step 1: Align the timestamps of the measurement data to be fused with the timestamps of the existing valid fusion targets.

[0051] Step 2: Perform weighted summation of the Mahalanobis distance of the position and the Euclidean distance of the velocity between the measurement data and the aligned effective fusion target to obtain a weighted distance, and calculate the association threshold range and association probability between the measurement data and the aligned effective fusion target based on the weighted distance.

[0052] Step 3: Determine whether the measurement data is associated and matched to any valid fusion target based on the association threshold range and association probability to generate a matching pair; for each matching pair, use the measurement data to update the filtering algorithm in the valid fusion target; for unmatched measurement data, use the unmatched measurement data as the initialization condition of the filtering algorithm to create a new fusion target.

[0053] The present invention provides a multi-source data fusion method that expands and innovates the joint probabilistic data association algorithm in the data association algorithm and proposes an improved multi-source data fusion algorithm. The algorithm performs weighted calculations on the Mahalanobis distance of the position vector and the Euclidean distance of the velocity vector between the fusion target and the sensor measurement. The above results are used to calculate the association threshold in the joint probabilistic data association algorithm, and this distance formula is used to calculate the joint association probability of the fusion target and the sensor measurement, thereby improving the association matching accuracy.

[0054] Example 1

[0055] Embodiment 1 provided by the present invention is an embodiment of a multi-source data fusion method provided by the present invention. Figure 1 In the given embodiment, the data collected by the camera and the millimeter-wave radar are respectively transmitted to the corresponding sensors, and the visual targets and radar targets containing position information and speed information are parsed through the relevant perception algorithm, and then the time and spatial positions of different sensor data are synchronized through sensor calibration and time synchronization. For the sake of ease of description, the visual targets and radar targets are uniformly referred to as sensor measurements (which can be simplified to "measurement data") below. The entire perception fusion algorithm can be divided into three major sections: data association matching, filter creation and update, and fusion target management. Specifically, combined with Figure 1 and Figure 2It can be seen that the embodiment of the fusion method includes:

[0056] Multi-source data includes at least two types of measurement data from different sensors, both of which contain position and velocity information. The fusion method includes:

[0057] Step 1: Align the timestamps of the measurement data to be fused with the timestamps of the existing valid fusion targets.

[0058] In one possible embodiment, step 1 includes:

[0059] After obtaining the measurement data to be fused at time t, the position and velocity information of each effective fusion target at time t is predicted using the motion state equation.

[0060] After each frame measurement enters the perception fusion algorithm module, the existing effective fusion target is first aligned with the measurement timestamp, that is, the position and velocity information of the effective fusion target is predicted through the motion state equation to obtain the predicted target.

[0061] Step 2: Perform weighted summation of the Mahalanobis distance of the position and the Euclidean distance of the velocity between the measurement data and the aligned effective fusion target to obtain a weighted distance, and calculate the association threshold range and association probability between the measurement data and the aligned effective fusion target based on the weighted distance.

[0062] The traditional joint probability data association algorithm uses the data association algorithm to associate and match the measurement and prediction targets. The input of the data association algorithm is the measurement at a certain moment and the corresponding effective fusion target. Assume that there are M measurements at time t and N effective fusion targets in All motion state information of the target detected by the mth sensor, including relative position and relative speed, The motion state information contained in the nth fusion target, including relative position and relative speed. Taking 2D sensor data as an example, the above information can be simply defined as:

[0063]

[0064]

[0065] Where [M]0 represents the set of measurement IDs, with 0 representing an incorrect or unmatched measurement; and [N] represents the set of fused target IDs. Below, we will use m and n to uniformly represent measurement and fused target IDs.

[0066] Assume that the association between the mth measurement and the nth fusion target is expressed as in Is a binary value to indicate whether it is associated. Then, at time t, the probability that the mth measurement is associated with the nth fusion target is Simplified to

[0067]

[0068] Among them, p D represents the sensor detection probability; represents Gaussian normal distribution; Represents the fusion target The predicted state vector at time t; Σ S Represents the prediction information covariance of the Kalman filter algorithm contained in the fusion target.

[0069] The calculation of association probability by the joint probabilistic data association algorithm is based on the data association hypothesis event set Θ. The set Θ contains all possible events of the association combination of measurement and fusion target. Each event meets two conditions: each measurement can be matched to at most one fusion target; each fusion target can only be associated with one measurement. Then, at time t, the association hypothesis event set Θ t It can be defined as:

[0070]

[0071] in, It is a binary matrix used to represent each possible data association event.

[0072] In the definition of the joint probabilistic data association algorithm, whether each measurement and each fusion target in each association event are associated depends on the range of the association threshold, and the range of the association threshold is calculated based on the Mahalanobis distance. That is, at time t, the association threshold between the mth measurement and the nth fusion target is

[0073]

[0074] Among them, S tn represents the covariance of the nth fusion target at time t; is the Mahalanobis distance between the mth measurement and the nth fusion target at time t; G is the established threshold range. Then the mth measurement and the nth fusion target may form an associated matching pair, that is, Possibly 1.

[0075] Let's use an example to explain how to form a set of related events. Suppose there are two measurements in a two-dimensional plane at a certain moment. and 2 fusion targets like Figure 3 After time synchronization, the predicted fusion target is obtained According to the predetermined threshold range G, Two ellipses are formed as the center, and these two elliptical areas are the fusion targets Therefore, each measurement in the figure may have three matching situations, namely matching the fusion target Or match to the fusion target Or no fusion target is matched. Therefore, a 2×3 matrix can be used to represent the associated events, and all associated events can be expressed as:

[0076]

[0077] Θ={θ1,θ2,θ3,θ4} (7)

[0078] Among them, θ i The first and second rows represent the measurement The first, second, and third columns indicate no match or match to the fusion target. Matching to fusion target Θ = {θ1, θ2, θ3, θ4} is the set of associated events in this example.

[0079] After defining the set of associated events, the joint probabilistic data association algorithm needs to consider all key events to determine the joint association probability of the mth measurement and the nth fusion target at time t. Assumptions represents a subset of all events that can contain m measurements associated with the nth fusion target, then the probability of association is It can be calculated as:

[0080]

[0081] This joint association probability It will be used as input to update the new information vector of the Kalman filter in the fusion target, that is,

[0082]

[0083] in, represents the new information vector of the nth fusion target at time t; represents the innovation vector generated by the mth measurement for the nth fusion target at time t. The updated innovation vector is used to update the Kalman filter to achieve the purpose of updating the fusion target.

[0084] It can be seen that in the joint probability data association algorithm, the calculation of the association probability will use formula (8) and formula (3), and the calculation is relatively complicated. Therefore, in the simple joint probability data association algorithm, The calculation can be simplified to:

[0085]

[0086]

[0087]

[0088] Where λ is the Poisson distribution parameter.

[0089] Traditional joint probabilistic data association algorithms rely on the Mahalanobis distance to calculate the association threshold and joint association probability. This distance, however, can make it difficult to correctly match measurements and fusion targets in certain scenarios, such as when sensor data transmission frequency is limited and the ego vehicle makes sudden turns. The following example illustrates the challenges of traditional joint probabilistic data association algorithms in such scenarios.

[0090] like Figure 3 FIG. 1 is an example diagram of a method for defining associated events in a joint probability association algorithm provided by an embodiment of the present invention; Figure 4 FIG. 1 is a schematic diagram showing two measurement data and two effective fusion targets in a two-dimensional plane at two adjacent moments provided by an embodiment of the present invention; Figure 5 FIG2 is a comparative diagram of the fusion of the traditional joint probabilistic data association algorithm provided by an embodiment of the present invention and the association algorithm provided by the present invention. Figure 4 The center coordinate system represents the ego vehicle, and the rectangle represents the target vehicle. At time t, the target vehicle is moving relative to the ego vehicle at velocity v. At this time, a large counterclockwise angular velocity is applied to the ego vehicle to simulate a sudden, sharp turn. At time t+1, the target vehicle's position changes from the dotted position in the right figure to the position indicated by the solid rectangle, generating a large lateral velocity component. Figure 5 In Figure 3, the correlation thresholds calculated using Mahalanobis distance and Euclidean distance are compared for position and velocity vectors. The left figure shows the distance correlation threshold using Mahalanobis distance, while the right figure shows the velocity correlation threshold. It can be seen that in the case of sudden changes in lateral velocity, the Euclidean distance improves the probability of correlation between the fused target and the corresponding sensor measurement compared to the Mahalanobis distance.

[0091] Figure 5 The coordinate system in is defined as the ego vehicle, with the origin at the ego vehicle's center point, y as the ego vehicle's forward direction, and x to the right of the ego vehicle. Suppose at time t, a target vehicle is detected moving at velocity v relative to the ego vehicle. At this point, the ego vehicle suddenly turns left. Therefore, a large counterclockwise angular velocity needs to be added to the coordinate system at this time. Then, at time t+1, the target vehicle will change from its predicted position (i.e., the dashed rectangle in the right figure) to the position of the solid rectangle in the right figure, generating a large lateral velocity.

[0092] As can be seen, the position of the sensor target changes little from time t to time t+1, but the target experiences a significant increase in lateral velocity due to the ego vehicle's sudden turn. Assuming that a new valid fused target is generated for this sensor target using a filtering algorithm at time t, then at time t+1, using the Mahalanobis distance to calculate the association threshold for the target's position and velocity will result in a mismatch between the valid target and the sensor target due to velocity mismatch, leading to loss of the fused target or a change in its ID.

[0093] According to such situations, the embodiment of the present invention proposes an improved joint probability data association algorithm. First, when calculating the association threshold, the present invention redefines the distance calculation method. According to formulas (1) and (2), and Separating these formulas into position and velocity yields:

[0094]

[0095]

[0096] in, Represents the position and speed measured by the sensor; Represents the position and velocity of the fused target.

[0097] According to formulas (13) and (14), the present invention proposes a weighted distance between the Mahalanobis distance of position and the Euclidean distance of velocity to calculate the association threshold in the joint probabilistic data association algorithm, that is,

[0098]

[0099] Where G is the same as G in formula (5), which represents the established threshold range; is S in formula (5) tn The position component of represents the position covariance of the nth fusion target at time t; α and β are weight values, which are obtained based on experimental test results; It is expressed as the weighted distance between the mth measurement and the nth fusion target at time t.

[0100] by Figure 4 For example, Figure 5 The difference between using Mahalanobis distance and weighted distance to calculate the association threshold is shown. According to the above example description, from time t to time t+1, the position measured by the sensor does not change much, and the association matching can still be calculated by Mahalanobis distance, such as Figure 5 Left half; the velocity measured by the sensor changes greatly, and there is a surge in the lateral velocity. At this time, compared with the Mahalanobis distance of the elliptical threshold (such as Figure 5The Euclidean distance can effectively increase the correlation threshold range in the lateral velocity direction (as shown in the dotted ellipse in the right half). Figure 5 The embodiment of the present invention adds two additional adjustable parameters α and β, so that Figure 5 The elliptical threshold of the target position and the circular threshold range of the target speed can be further adjusted according to the actual scene requirements.

[0101] In order to calculate the joint association probability more accurately, the embodiment of the present invention improves the original formula (11) in the simple joint probability data association algorithm to:

[0102]

[0103] The values of α and β are the same as those in formula (15).

[0104] In a possible embodiment, formula (12) can be modified accordingly as follows:

[0105]

[0106] That is, the calculation of the joint association probability in the present invention is formed by combining formulas (10)(16)(17), which can more accurately locate the association probability between the fusion target and the sensor measurement.

[0107] The present invention is derived through theoretical analysis of traditional fusion algorithms and perception fusion experiments (visual data and millimeter-wave radar data), but the method itself can be extended to a variety of decision-level multi-sensor perception fusions for different application scenarios.

[0108] Step 3: Determine whether the measurement data is associated and matched to any valid fusion target based on the association threshold range and association probability to generate a matching pair; for each matching pair, use the measurement data to update the filtering algorithm in the valid fusion target; for unmatched measurement data, use the unmatched measurement data as the initialization condition of the filtering algorithm to create a new fusion target.

[0109] In one possible embodiment, step 3 includes:

[0110] According to the matching results between the measurement data and the effective fusion target, the measurement data and the effective fusion target are classified into: matching pairs of measurement data and effective fusion targets, unmatched measurement data and unmatched effective fusion targets.

[0111] In the specific implementation, for the unmatched fusion target, the total number of unmatched frames is recorded.

[0112] In a possible embodiment, step 3 further includes:

[0113] Perform target lifecycle management on all fusion targets, filter and output valid fusion targets.

[0114] That is, determine which fusion targets are valid fusion targets and which fusion targets are invalid fusion targets, etc. Finally, all valid fusion targets are output through screening.

[0115] Example 2

[0116] The embodiment 2 provided by the present invention is an embodiment of a multi-source data fusion system provided by the present invention, wherein the multi-source data includes at least two types of measurement data from different sensors, and the measurement data includes position information and speed information, combined with Figure 1 and Figure 2 It can be seen that the fusion system includes: parsing module, association matching module and filtering update module;

[0117] The parsing module is used to align the timestamps of the measurement data to be fused with the existing valid fusion targets.

[0118] In a possible embodiment, the processing of the parsing module further includes:

[0119] After obtaining the measurement data to be fused at time t, the position and velocity information of each effective fusion target at time t is predicted using the motion state equation.

[0120] After each frame measurement enters the perception fusion algorithm module, the existing effective fusion target is first aligned with the measurement timestamp, that is, the position and velocity information of the effective fusion target is predicted through the motion state equation to obtain the predicted target.

[0121] The association matching module is used to obtain a weighted distance by performing a weighted summation of the Mahalanobis distance of the position and the Euclidean distance of the velocity between the measurement data and the aligned effective fusion target, and calculate the association threshold range and association probability between the measurement data and the aligned effective fusion target based on the weighted distance.

[0122] In a possible embodiment, the weighted distance is calculated as follows:

[0123]

[0124] Among them, α and β are adjustable weight values, which can be obtained based on experimental test results. and are the position vector and velocity vector of the nth effective fusion target at time t; and are the position vector and velocity vector of the mth measurement data at time t respectively; is the position covariance of the nth effective fusion target at time t.

[0125] In specific implementation, the present invention improves the traditional joint probabilistic data association algorithm (JPDA algorithm). The core innovation lies in performing a weighted summation of the Mahalanobis distance of position and the Euclidean distance of velocity, and using the above results as the basis to calculate the association threshold range and association probability to solve the problem of possible velocity vector images.

[0126] The position vector of the nth fusion target at time t and velocity vector and the position vector of the mth measurement and velocity vector as follows:

[0127]

[0128]

[0129] In a possible embodiment, the association threshold range G between the measurement data and the aligned valid fusion target is calculated based on the weighted distance:

[0130]

[0131] In one possible embodiment, the association probability between the measurement data and the aligned valid fusion target is calculated based on the weighted distance:

[0132]

[0133] in, It represents the association probability between the mth measurement data and the nth effective fusion target at time t.

[0134]

[0135]

[0136] p D represents the sensor detection probability, M represents the total number of measurement data at time t, and λ is the Poisson distribution parameter.

[0137] The filter update module is used to determine whether the measurement data is associated and matched to any valid fusion target to generate a matching pair based on the association threshold range and association probability; for each matching pair, the measurement data is used to update the filtering algorithm in the valid fusion target; for unmatched measurement data, the unmatched measurement data is used as the initialization condition of the filtering algorithm to create a new fusion target.

[0138] In a possible embodiment, the processing of the filter update module further includes:

[0139] According to the matching results between the measurement data and the effective fusion target, the measurement data and the effective fusion target are classified into: matching pairs of measurement data and effective fusion targets, unmatched measurement data and unmatched effective fusion targets.

[0140] In the specific implementation, for the unmatched fusion target, the total number of unmatched frames is recorded.

[0141] In a possible embodiment, the fusion system further includes: a target screening management module;

[0142] The target screening management module is used to manage the target lifecycle of all fusion targets, screen and output valid fusion targets.

[0143] That is, determine which fusion targets are valid fusion targets and which fusion targets are invalid fusion targets, etc. Finally, all valid fusion targets are output through screening.

[0144] It can be understood that the multi-source data fusion system provided by the present invention corresponds to the multi-source data fusion method provided in the aforementioned embodiments. The relevant technical features of the multi-source data fusion system can refer to the relevant technical features of the multi-source data fusion method, which will not be repeated here.

[0145] See also Figure 6 , Figure 6 Schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 6 As shown, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, the following steps are implemented: aligning the timestamps of the measurement data to be fused with the existing valid fusion target; performing weighted summation of the Mahalanobis distance of the position and the Euclidean distance of the velocity of the measurement data and the aligned valid fusion target to obtain a weighted distance, and calculating the association threshold range and association probability of the measurement data and the aligned valid fusion target based on the weighted distance; judging whether the measurement data is associated and matched to any valid fusion target to generate a matching pair based on the association threshold range and the association probability; for each matching pair, using the measurement data to update the filtering algorithm in the valid fusion target; for unmatched measurement data, using the unmatched measurement data as the initialization condition of the filtering algorithm to create a new fusion target.

[0146] See also Figure 7 , Figure 7 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 7As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, it implements the following steps: aligning the timestamps of the measurement data to be fused with the existing valid fusion target; obtaining a weighted distance by weighted summation of the Mahalanobis distance of the position and the Euclidean distance of the velocity of the measurement data and the aligned valid fusion target, and calculating the association threshold range and association probability of the measurement data and the aligned valid fusion target based on the weighted distance; judging whether the measurement data is associated and matched to any valid fusion target to generate a matching pair based on the association threshold range and the association probability; for each matching pair, using the measurement data to update the filtering algorithm in the valid fusion target; for unmatched measurement data, using the unmatched measurement data as the initialization condition of the filtering algorithm to create a new fusion target.

[0147] The embodiments of the present invention provide a multi-source data fusion method, system, electronic device and storage medium. When the frequency of sensor data transmission is limited, the sudden turning of the vehicle causes all perceived targets within the sensor field of view to have a large lateral velocity, resulting in the joint probability data association algorithm based on the elliptical threshold being unable to associate and match the correct fusion target and measurement ratio pairing. This effectively solves the problem of continuous ID jumps caused by the traditional joint probability association algorithm under the above problem. The improved algorithm can ensure that the output of the fusion target is stable when the vehicle speed suddenly changes while the association accuracy remains unchanged in ordinary scenes. In addition, the introduction of weighted distance does not increase the overall computational complexity of the algorithm. Therefore, the algorithm proposed in this technical solution can still maintain the efficiency and real-time performance of operation.

[0148] A comparative test of a traditional simple joint probabilistic data association algorithm and the improved algorithm proposed in this technical solution was conducted using a simulation scenario involving a vehicle making continuous and sudden lane changes. The experiments demonstrated that the traditional simple joint probabilistic data association algorithm exhibited multiple ID jumps during sudden turns, while the improved simple joint probabilistic data association algorithm proposed in this technical solution effectively integrated the target and maintained continuous target updates. Therefore, the algorithm proposed in this invention effectively addressed the issues of sensor data transmission frequency and ID jumps in the fused target caused by sudden turns.

[0149] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0150] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0154] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0155] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A multi-source data fusion method, characterized in that: The multi-source data includes at least two types of measurement data from different sensors, each of which includes position information and speed information. The fusion method includes: Step 1: align the timestamps of the measurement data to be fused with the timestamps of the existing valid fusion targets; Step 2: performing a weighted summation of the Mahalanobis distance of the position and the Euclidean distance of the velocity between the measurement data and the aligned effective fusion target to obtain a weighted distance, and calculating an association threshold range and an association probability between the measurement data and the aligned effective fusion target based on the weighted distance; Step 3: Determine whether the measurement data is associated and matched to any of the valid fusion targets based on the association threshold range and association probability to generate a matching pair; for each matching pair, use the measurement data to update the filtering algorithm in the valid fusion target; for unmatched measurement data, use the unmatched measurement data as the initialization condition of the filtering algorithm to create a new fusion target; The calculation formula of the weighted distance is: in, and The weight value can be adjusted. and They are Moment An effective fusion target position vector and velocity vector; and They are Moment The position vector and velocity vector of each measurement data; for Moment The position covariance of the effective fusion targets; The association probability between the measurement data and the aligned valid fusion target is calculated based on the weighted distance as follows: ； in, Indicates Moment The measurement data and The association probability of a valid fusion target; ； represents the sensor detection probability, M represents The total number of measurement data at the moment, is the Poisson distribution parameter.

2. The fusion method according to claim 1, characterized in that: The step 1 comprises: After obtaining the measurement data to be fused at time t, the position and velocity information of each of the effective fusion targets at time t is predicted using the motion state equation.

3. The fusion method according to claim 1, characterized in that: The association threshold range G between the measurement data and the aligned valid fusion target calculated based on the weighted distance is: 。 4. The fusion method according to claim 1, characterized in that: The step 3 comprises: The measurement data and the valid fusion target are classified into: a matching pair of measurement data and a valid fusion target, unmatched measurement data, and an unmatched valid fusion target according to a matching result between the measurement data and the valid fusion target.

5. The fusion method according to claim 1, characterized in that: After step 3, the following steps are also included: Perform target lifecycle management on all fusion targets, filter and output the valid fusion targets.

6. A multi-source data fusion system, characterized in that: The multi-source data includes at least two types of measurement data from different sensors, and the measurement data both include position information and speed information. The fusion system includes: an analysis module, an association matching module, and a filter update module; The parsing module is used to align the timestamps of the measurement data to be fused with the timestamps of the existing valid fusion targets; The association matching module is configured to obtain a weighted distance by performing a weighted summation on the Mahalanobis distance of the position and the Euclidean distance of the velocity between the measurement data and the aligned effective fusion target, and calculate an association threshold range and an association probability between the measurement data and the aligned effective fusion target based on the weighted distance; The filter update module is configured to determine whether the measurement data is associated and matched with any of the valid fusion targets to generate a matching pair based on the association threshold range and the association probability; for each matching pair, update the filtering algorithm in the valid fusion target using the measurement data; and for unmatched measurement data, create a new fusion target using the unmatched measurement data as an initialization condition for the filtering algorithm; The calculation formula of the weighted distance is: in, and The weight value can be adjusted. and They are Moment An effective fusion target position vector and velocity vector; and They are Moment The position vector and velocity vector of each measurement data; for Moment The position covariance of the effective fusion targets; The association probability between the measurement data and the aligned valid fusion target is calculated based on the weighted distance as follows: ； in, Indicates Moment The measurement data and The association probability of a valid fusion target; ； represents the sensor detection probability, M represents The total number of measurement data at the moment, is the Poisson distribution parameter.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the processor is configured to implement the steps of the multi-source data fusion method according to any one of claims 1 to 5 when executing a computer management program stored in the memory.

8. A computer-readable storage medium, characterized in that A computer management program is stored thereon, and when the computer management program is executed by the processor, the steps of the multi-source data fusion method according to any one of claims 1 to 5 are implemented.

Images