A multi-sensor data fusion method based on piecewise function

Abstract

The present invention relates to a multi-sensor data fusion method based on piecewise function, comprising the following steps: 1) acquiring multiple sensor data and statistically outputting target distribution; 2) analyzing the stability of multiple sensor data and outputting jump flags; 3) fitting the target distribution based on the piecewise function and outputting a weight curve; 4) fusing multiple sensor data based on the weight curve to obtain fused data. The multi-sensor data fusion method based on piecewise function described in the present invention fits the target distribution based on the piecewise function to obtain a weight curve, and then fuses multiple sensor data based on the weight curve to obtain fused data. Compared with the existing methods, the method has simple steps and requires lower computing power. In addition, the method also adjusts the weight curve by analyzing the stability of multiple sensor data, so that the fusion of multiple sensor data is more accurate and reliable.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a multi-sensor data fusion method based on piecewise functions. Background Art

[0002] With the rapid development of automobile intelligence, more and more cars are equipped with automatic driving systems. In order to achieve more accurate and safe automatic driving, real-time acquisition of the environment around the vehicle is an important basis for automatic driving control. At present, the automatic driving system uses a large number of sensors to obtain the environment around the vehicle, such as cameras, lidar and millimeter wave radar. The amount of calculation required to process and analyze the data obtained by multiple sensors is large, which affects the timeliness of the control output. In addition, the difference in data obtained by multiple sensors is likely to interfere with the control decision of the automatic driving, resulting in poor accuracy and safety of automatic driving control.

[0003] Therefore, the automatic control systems of cars on the market now all have the function of multi-sensor data fusion, which outputs the data obtained by multiple sensors after fusion processing, which not only reduces the amount of calculation required for data analysis and processing, but also makes the fused data more accurate and reliable, which can effectively improve the accuracy and safety of autonomous driving control. At present, the multi-sensor data fusion methods are mainly divided into traditional methods and machine learning methods. In the traditional method, data fusion through Kalman filter algorithm is more common, but Kalman filter algorithm is easily affected by model uncertainty and interference signals, which easily leads to reduced reliability of fused data; although machine learning method can realize multi-sensor data fusion more accurately, the fusion effect of machine learning method is greatly affected by sample size and scene coverage; Chinese patent CN201810765485.1 is a multi-sensor data weighted fusion method, which first filters the raw data obtained by the sensor, which weakens the authenticity of the data to a certain extent, and then fuses the data based on the filtered data and the corresponding weights to obtain fused data, and finally fuses the fused data again based on the probability statistics method, which is complex, affecting the efficiency of the system to a certain extent, resulting in poor timeliness of autonomous driving control. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a multi-sensor data fusion method based on piecewise function, so as to solve the problems of poor reliability and complicated steps in the current multi-sensor data fusion method, and achieve the effect of improving the reliability and timeliness of data fusion.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A multi-sensor data fusion method based on piecewise function comprises the following steps:

[0007] 1) Obtain data from multiple sensors and calculate the distribution of output targets;

[0008] 2) Analyze the stability of multiple sensor data and output the jump flag;

[0009] 3) Fit the target distribution based on the piecewise function and output the weight curve;

[0010] 4) Based on the weight curve, multiple sensor data are fused to obtain fused data.

[0011] Furthermore, the specific content of step 1) is as follows: target information in multiple sensor data is counted frame by frame to obtain target distribution, and the target information includes the distance between the target and the vehicle and the azimuth of the target relative to the vehicle.

[0012] Furthermore, the specific content of step 2) is as follows: based on the vehicle kinematic model, a stability analysis is performed on the target information in the multiple sensor data, and a corresponding jump threshold is set. When the target information jumps and the change value is greater than the jump threshold, a jump flag is output.

[0013] Further, step 3) includes the following sub-steps:

[0014] 31) Fitting the target distribution based on the piecewise function to obtain a weight curve;

[0015] 32) The weight curve is adjusted and output based on the jump flag. If the output jump flag is 1, the weight corresponding to the corresponding frame target information in the weight curve is multiplied by the adjustment coefficient.

[0016] Furthermore, the multiple sensor data in step 1) include camera data and radar data.

[0017] Further, step 4) includes the following sub-steps:

[0018] 41) Define the set of target positions in the camera data as P, define the set of target positions in the radar data as Q, and define the set of target positions in the fusion data as F. Then P, Q and F are expressed as follows:

[0019] P=[p1…p n ]

[0020] Q=[q1…q n ]

[0021] F=[f1…f n ]

[0022] Use pi It represents the coordinates of the target in the i-th frame camera data in the vehicle body coordinate system, and q i represents the coordinates of the target in the i-th frame of radar data in the vehicle body coordinate system, and f i represents the coordinates of the target in the i-th frame fusion data in the vehicle body coordinate system, i∈[1,n];

[0023] 42) Based on the weight curve, multiple sensor data are fused, and the fusion formula is as follows:

[0024] F=W1P+W2Q / (W1+W2)

[0025] Among them, W1 represents the weight corresponding to the camera data, and W2 represents the weight corresponding to the radar data.

[0026] Furthermore, the multi-sensor data fusion method based on piecewise function described in the present invention also includes step 5): quantitatively analyzing the fused data according to the true value obtained by the true value system to determine whether to output the fused data.

[0027] Furthermore, the specific contents of step 5) are as follows:

[0028] Calculate the sum of squares of the error between the fused data and the true value. The calculation formula is as follows:

[0029]

[0030] Among them, ε represents the sum of squares of the errors between the fused data and the true value, and f i is the set of fusion results F, r i The set of truth values ​​obtained for the truth system;

[0031] The judgment is made based on the preset threshold T of the sum of squared errors. When ε is less than T, the fused data is output for use by the autonomous driving system.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The multi-sensor data fusion method based on piecewise function described in the present invention obtains a weight curve by fitting the target distribution based on the piecewise function, and obtains fused data by fusing multiple sensor data based on the weight curve. Compared with the existing method, the method has simple steps and requires lower computing power. In addition, the method also adjusts the weight curve by analyzing the stability of multiple sensor data, so that the fusion of multiple sensor data is more accurate and reliable.

[0034] 2. The multi-sensor data fusion method based on piecewise function described in the present invention performs quantitative comparative analysis on the fused data after fusing multiple sensor data. When the fused data meets the preset standards, it is output for the automatic driving system to perform automatic driving control. The fused data output by this method has better reliability, which can effectively avoid automatic driving errors caused by abnormal output and improve the safety of the automatic driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The present invention is a flowchart of a multi-sensor data fusion method based on piecewise function according to an embodiment. DETAILED DESCRIPTION

[0036] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0037] Example:

[0038] See also Figure 1 , a multi-sensor data fusion method based on piecewise function, comprising the following steps:

[0039] 1) Acquire multiple sensor data and statistically output target distribution; the specific content is as follows: Count the target information in the multiple sensor data frame by frame to obtain the target distribution, the target information includes the distance between the target and the vehicle and the azimuth of the target relative to the vehicle; in this embodiment, the multiple sensor data include camera data and radar data.

[0040] 2) Analyze the stability of multiple sensor data and output the jump flag; the specific contents are as follows: Based on the vehicle kinematic model, the stability of the target information in the multiple sensor data is analyzed, and the corresponding jump threshold is set. When the target information jumps and the change value is greater than the jump threshold, the jump flag is output.

[0041] 3) Fitting the target distribution based on the piecewise function and outputting the weight curve; including the following sub-steps:

[0042] 31) Fitting the target distribution based on the piecewise function to obtain a weight curve;

[0043] 32) The weight curve is adjusted and output based on the jump flag. If the output jump flag is 1, the weight corresponding to the corresponding frame target information in the weight curve is multiplied by the adjustment coefficient; if the jump flag is not set, no adjustment is made.

[0044] 4) fusing multiple sensor data based on the weight curve to obtain fused data; including the following sub-steps:

[0045] 41) Define the set of target positions in the camera data as P, define the set of target positions in the radar data as Q, and define the set of target positions in the fusion data as F. Then P, Q and F are expressed as follows:

[0046] P=[p1…p n ]

[0047] Q=[q1…q n ]

[0048] F=[f1…f n ]

[0049] Use p i It represents the coordinates of the target in the i-th frame camera data in the vehicle body coordinate system, and q i represents the coordinates of the target in the i-th frame of radar data in the vehicle body coordinate system, and f i represents the coordinates of the target in the i-th frame fusion data in the vehicle body coordinate system, i∈[1,n];

[0050] 42) Based on the weight curve, multiple sensor data are fused, and the fusion formula is as follows:

[0051] F=W1P+W2Q / (W1+W2)

[0052] Among them, W1 represents the weight corresponding to the camera data, and W2 represents the weight corresponding to the radar data.

[0053] The present invention discloses a multi-sensor data fusion method based on piecewise function, which obtains a weight curve by fitting the target distribution based on the piecewise function, and obtains fused data by fusing multiple sensor data based on the weight curve. Compared with the existing method, the method has simple steps and requires lower computing power. In addition, the method also adjusts the weight curve by analyzing the stability of multiple sensor data, so that the fusion of multiple sensor data is more accurate and reliable.

[0054] Furthermore, the multi-sensor data fusion method based on piecewise function described in the present invention further includes step 5): quantitatively analyzing the fused data according to the true value obtained by the true value system to determine whether to output the fused data; in this embodiment, the true value system is an RTK measurement system using Real Time Kinematic real-time dynamic differential positioning technology;

[0055] The specific contents are as follows:

[0056] Calculate the sum of squares of the error between the fused data and the true value. The calculation formula is as follows:

[0057]

[0058] Among them, ε represents the sum of squares of the errors between the fused data and the true value, and f i is the set of fusion results F, r i The set of truth values ​​obtained for the truth system;

[0059] The judgment is made based on the preset threshold T of the sum of squared errors. When ε is less than T, the fused data is output for use by the autonomous driving system.

[0060] In this way, after the fusion of multiple sensor data, the fused data is quantitatively compared and analyzed. When the fused data meets the preset standards, it is output for the autonomous driving system to perform autonomous driving control. This makes the fused data output by this method more reliable, effectively avoids autonomous driving errors due to abnormal output, and improves the safety of the autonomous driving system.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A multi-sensor data fusion method based on piecewise function, characterized in that: The steps include: 1) Obtain data from multiple sensors and calculate the distribution of output targets; 2) Analyze the stability of multiple sensor data and output the jump flag; 3) Fit the target distribution based on the piecewise function and output the weight curve; 4) Fusing multiple sensor data based on the weight curve to obtain fused data; The specific content of step 1) is as follows: Counting the target information in the multiple sensor data frame by frame to obtain the target distribution, the target information includes the distance between the target and the vehicle and the azimuth of the target relative to the vehicle; The specific content of step 2) is as follows: based on the vehicle kinematics model, the target information in the multiple sensor data is analyzed for stability, and the corresponding jump threshold is set. When the target information jumps and the change value is greater than the jump threshold, the jump flag is output; Step 3) includes the following sub-steps: 31) Fitting the target distribution based on the piecewise function to obtain a weight curve; 32) The weight curve is adjusted and output based on the jump flag. If the output jump flag is 1, the weight corresponding to the corresponding frame target information in the weight curve is multiplied by the adjustment coefficient.

2. The multi-sensor data fusion method based on piecewise function according to claim 1, characterized in that: The multiple sensor data in step 1) include camera data and radar data.

3. The multi-sensor data fusion method based on piecewise function according to claim 2, characterized in that: Step 4) includes the following sub-steps: 41) Define the set of target positions in the camera data as P, define the set of target positions in the radar data as Q, and define the set of target positions in the fusion data as F. Then P, Q and F are expressed as follows: P=[p1…p n ] Q=[q1…q n ] F=[f1…f n ] Use p i It represents the coordinates of the target in the i-th frame camera data in the vehicle body coordinate system, and q i represents the coordinates of the target in the i-th frame of radar data in the vehicle body coordinate system, and f i represents the coordinates of the target in the i-th frame fusion data in the vehicle body coordinate system, i∈[1,n]; 42) Based on the weight curve, multiple sensor data are fused, and the fusion formula is as follows: F=W1P+W2Q / (W1+W2) Among them, W1 represents the weight corresponding to the camera data, and W2 represents the weight corresponding to the radar data.

4. The multi-sensor data fusion method based on piecewise function according to claim 3, characterized in that: The method also includes step 5): performing quantitative analysis on the fused data according to the true value obtained by the true value system to determine whether to output the fused data.

5. The multi-sensor data fusion method based on piecewise function according to claim 4, characterized in that: The specific contents of step 5) are as follows: Calculate the sum of squares of the error between the fused data and the true value. The calculation formula is as follows: Among them, ε represents the sum of squares of the errors between the fused data and the true value, and f i is the set of fusion results F, r i The set of truth values ​​obtained for the truth system; The judgment is made based on the preset threshold T of the sum of squared errors. When ε is less than T, the fused data is output for use by the autonomous driving system.

Images