Multi-sensor fusion parameter calibration method and system based on continuous integrated deployment framework

Abstract

The invention discloses a multi-sensor fusion parameter calibration method and system based on a continuous integration deployment framework, and the method comprises the steps: S1, collecting test data through a plurality of sensors, and transmitting the data to a cloud platform; s2, the cloud platform preprocesses the collected data, comprehensively processes the data of the multiple sensors through a fusion module, and calculates calibration parameters among the multiple sensors; s3, triggering an automatic test and parameter optimization process through a CI / CD assembly line tool, and adjusting parameters in the perception model; s4, the developer corrects the optimized parameters through a feedback module of the cloud platform, and the system further adjusts the calibration parameters through an automation mechanism; and S5, deploying the optimized calibration parameters into a sensor fusion system of the vehicle, and testing and updating through real-time data. According to the method, the calibration time is remarkably shortened through an automatic process and continuous optimization, the method adapts to different driving scenes and sensor configurations, and the adaptability and robustness are enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile sensor fusion, and in particular relates to a multi-sensor fusion parameter calibration method and system based on a continuous integration deployment framework. Background Art

[0002] In advanced driver assistance systems (ADAS), multi-sensor fusion algorithms provide accurate and reliable environmental perception and decision support by combining data from multiple sensors (such as 4D radar, lidar, etc.). The parameter calibration within the sensor fusion algorithm (such as the K matrix and R matrix in the Kalman filter) is crucial to the algorithm performance. However, the existing technology has the following shortcomings: Manual parameter adjustment method: time-consuming, dependent on engineer experience, difficult to adapt to dynamic changes, and lacks global optimality.

[0003] Adaptive parameter adjustment method: high complexity, stability issues, dependence on initial parameters, and high computing resource requirements.

[0004] Optimization algorithm parameter adjustment method: high computational cost, slow convergence, sensitive to initial values, and difficult to handle non-convex problems. Data-driven parameter adjustment method: strong data dependence, limited generalization ability, high model complexity, and poor interpretability. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-sensor fusion parameter calibration method and system based on a continuous integration deployment framework to solve the above technical problems.

[0006] In order to solve the above technical problems, the present invention adopts the following solutions: The multi-sensor fusion parameter calibration method based on the continuous integration deployment framework includes the following steps: Step S1: Automated data collection: Collect test data by installing multiple sensors on the test vehicle and transmit the data to the cloud platform. The data collected by the sensors is uploaded to the cloud platform in real time via wireless communication (such as Wi-Fi or 5G).

[0007] Step S2: Data processing and fusion: The cloud platform first pre-processes the collected data, and then uses the fusion module to comprehensively process the data of multiple sensors and calculate the calibration parameters between multiple sensors.

[0008] Step S3: Parameter calibration and adjustment: Trigger the automated testing and parameter optimization process through the CI / CD pipeline tool to adjust the parameters in the perception model; during the optimization process, the cloud platform will automatically adjust the calibration model according to the different test data input, and provide real-time feedback on the adjustment results.

[0009] Step S4: Feedback and correction: After the model adjustment is completed, the developer corrects the optimized parameters through the feedback module of the cloud platform, and the system further adjusts the calibration parameters through the automation mechanism. If there is a deviation after the parameter optimization, the developer can propose correction suggestions in the feedback module of the cloud platform, and the system further adjusts the calibration parameters through the automation mechanism.

[0010] Step S5: Result output and deployment: The optimized calibration parameters are deployed to the vehicle's sensor fusion system and tested and updated with real-time data. After the optimized calibration parameters pass the test, the system deploys the final results to the vehicle's sensor fusion system. After the calibration results are deployed, the perception model continues to provide real-time feedback and is continuously adjusted and optimized through the cloud platform to ensure the adaptability and robustness of the calibration process.

[0011] Further optimization, in step S1, the sensors set on the test vehicle include but are not limited to millimeter wave radar, 4D radar, lidar, camera, ultrasonic sensor, inertial measurement unit, GPS, vehicle network sensor and magnetometer.

[0012] Further optimization, in step S2, data preprocessing includes data cleaning and enhancement; wherein, data cleaning is used to remove noise and invalid data in the data collected by the sensor to improve data quality and reduce the impact of noise on the fusion result. The cleaned data is stored in a standard format (such as JSON or CSV).

[0013] By performing operations including but not limited to flipping, scaling, and splicing on the collected data samples, new data samples can be generated to enhance the diversity of data and improve the generalization ability of the model. In multi-sensor fusion, data enhancement can be used to improve the model's adaptability to different scenarios and enhance the robustness of the system.

[0014] Further optimization, in step S2, the fusion module includes but is not limited to Kalman filter, extended Kalman filter and unscented Kalman filter.

[0015] Among them, Kalman filtering is used to estimate the state of linear dynamic systems. It minimizes the estimation error by combining the sensor measurements and the predicted values ​​of the system model. In multi-sensor fusion, Kalman filtering is often used to fuse data from different sensors (such as 4D radar and lidar) to improve the estimation accuracy of target position and velocity. Extended Kalman filtering is applicable to nonlinear systems. When processing nonlinear sensor data (such as visual sensors and radar data), EKF can be used to fuse data from different sensors to improve the adaptability and accuracy of the system. Unscented Kalman filtering approximates the statistical characteristics of nonlinear systems by selecting a set of sample points (Sigma points), thereby avoiding errors in the linearization process. UKF performs well in processing complex nonlinear sensor data, and is particularly suitable for high-precision multi-sensor fusion scenarios.

[0016] Further optimization, in step S3, the model parameter optimization algorithm includes but is not limited to gradient descent method, genetic algorithm and particle swarm algorithm.

[0017] Among them, the gradient descent method is an iterative optimization algorithm that gradually adjusts the parameters to approach the optimal value by calculating the gradient of the performance index to the parameters. In multi-sensor fusion, the gradient descent method can be used to optimize the parameters of the Kalman filter (such as the Q matrix and the R matrix) to minimize the estimation error. Genetic algorithm is an optimization algorithm based on natural selection, which searches for the optimal solution through crossover, mutation and selection operations. It is suitable for the optimization of high-dimensional parameter space, especially in complex multi-sensor fusion scenarios, and can find a better parameter configuration. Particle swarm optimization is an optimization algorithm based on swarm intelligence, which searches for the optimal solution by simulating the foraging behavior of bird flocks. In multi-sensor fusion, it can be used to optimize sensor parameters and improve the overall performance of the system.

[0018] For further optimization, in step S4, a feedback loop is triggered by the CI / CD pipeline tool, and the corrected model parameters are input into the system again for verification. This process ensures that the calibration parameters of each version are verified and optimized through continuous integration.

[0019] A multi-sensor fusion parameter calibration system based on a continuous integration deployment framework, comprising: A data acquisition module is used to automatically collect test data through multiple sensors installed on the test vehicle and transmit the data to the cloud platform; a result output and deployment module; The cloud platform completes the multi-sensor fusion parameter calibration based on the collected data by using the continuous integration and deployment of the CI / CD framework set on the cloud platform; the cloud platform is deployed with a data processing and fusion module, a data processing and fusion module, a parameter calibration and adjustment module, a feedback and correction module, and a result output and deployment module; Among them, the data processing and fusion module pre-processes the collected data, including data cleaning, formatting and fusion processing, and calculates the calibration parameters between multiple sensors; The parameter calibration and adjustment module is used to trigger the automated testing and parameter optimization process through CI / CD tools to adjust and verify the parameters in the perception model; Feedback and correction module, which is used by developers to correct the optimized parameters through the feedback module of the cloud platform, and the system further adjusts the calibration parameters through an automated mechanism; The result output and deployment module is used to deploy the optimized calibration parameters to the vehicle's sensor fusion system and test and update them through real-time data.

[0020] For further optimization, the feedback and correction module corrects the optimized parameters through the feedback mechanism of the cloud platform and performs multiple rounds of optimization through an automated mechanism.

[0021] Compared with the prior art, the present invention has the following beneficial effects: 1. The method of the present invention significantly shortens the calibration time and avoids manual errors through automated processes and continuous optimization, thereby greatly improving the calibration efficiency and accuracy. The optimized calibration parameters are deployed to the vehicle's sensor fusion system, and automated and continuous optimization is achieved through testing and updating with real-time data, thereby achieving automated and continuous optimization of the calibration process.

[0022] 2. The method of the present invention is adaptable to different driving scenarios and sensor configurations, and enhances adaptability and robustness.

[0023] 3. The method described in the present invention ensures the consistency of calibration results through standardized CI / CD process and version control.

[0024] 4. The method described in the present invention supports multi-sensor collaborative calibration, comprehensively considers the relationship between each sensor, and improves the overall performance of the sensor fusion system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of the multi-sensor fusion parameter calibration method based on the continuous integration deployment framework described in the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the embodiments, but it should not be understood that the above subject matter scope of the present invention is limited to the following embodiments. Embodiment 1:

[0027] like Figure 1 As shown, a multi-sensor fusion parameter calibration method based on a continuous integration deployment framework includes the following steps: Step S1: Automated data collection: Collect test data by installing multiple sensors on the test vehicle and transmit the data to the cloud platform. The data collected by the sensors is uploaded to the cloud platform in real time via wireless communication (such as Wi-Fi or 5G). The sensors installed on the test vehicle include but are not limited to millimeter wave radar, 4D radar, lidar, camera, ultrasonic sensor, inertial measurement unit, GPS, vehicle network sensor and magnetometer.

[0028] Step S2: Data processing and fusion: The cloud platform first pre-processes the collected data, and then uses the fusion module to comprehensively process the data of multiple sensors and calculate the calibration parameters between multiple sensors.

[0029] Data preprocessing includes data cleaning and enhancement; data cleaning is used to remove noise and invalid data from the data collected by the sensor to improve data quality and reduce the impact of noise on the fusion results. The cleaned data is stored in a standard format (such as JSON or CSV). New data samples are generated by performing operations including but not limited to flipping, scaling, and splicing on the collected data samples to enhance data diversity and improve the generalization ability of the model. In multi-sensor fusion, data enhancement can be used to improve the model's adaptability to different scenarios and enhance the robustness of the system.

[0030] The fusion module includes but is not limited to Kalman filter, extended Kalman filter and unscented Kalman filter. Among them, Kalman filter is used to estimate the state of linear dynamic system. It minimizes the estimation error by combining the sensor measurement value and the predicted value of the system model. In multi-sensor fusion, Kalman filter is often used to fuse data from different sensors (such as 4D radar and lidar) to improve the estimation accuracy of target position and velocity. Extended Kalman filter is suitable for nonlinear systems. When processing nonlinear sensor data (such as visual sensor and radar data), EKF can be used to fuse data from different sensors to improve the adaptability and accuracy of the system. Unscented Kalman filter approximates the statistical characteristics of nonlinear system by selecting a set of sample points (Sigma points), thereby avoiding errors in the linearization process. UKF performs well in processing complex nonlinear sensor data, especially for high-precision multi-sensor fusion scenarios.

[0031] Step S3: Parameter calibration and adjustment: Trigger the automated testing and parameter optimization process through the CI / CD pipeline tool to adjust the parameters in the perception model; during the optimization process, the cloud platform will automatically adjust the calibration model according to the different test data input, and provide real-time feedback on the adjustment results.

[0032] Model parameter optimization algorithms include but are not limited to gradient descent, genetic algorithm, and particle swarm optimization. Among them, gradient descent is an iterative optimization algorithm that gradually adjusts the parameters to approach the optimal value by calculating the gradient of the performance index to the parameter. In multi-sensor fusion, gradient descent can be used to optimize the parameters of the Kalman filter (such as the Q matrix and the R matrix) to minimize the estimation error. Genetic algorithm is an optimization algorithm based on natural selection, which searches for the optimal solution through crossover, mutation, and selection operations. It is suitable for the optimization of high-dimensional parameter space, especially in complex multi-sensor fusion scenarios, and can find a better parameter configuration. Particle swarm optimization is an optimization algorithm based on swarm intelligence, which searches for the optimal solution by simulating the foraging behavior of bird flocks. In multi-sensor fusion, it can be used to optimize sensor parameters and improve the overall performance of the system.

[0033] Step S4: Feedback and correction: After the model adjustment is completed, the developer corrects the optimized parameters through the feedback module of the cloud platform, and the system further adjusts the calibration parameters through the automation mechanism. If there is a deviation after the parameter optimization, the developer can propose correction suggestions in the feedback module of the cloud platform, and the system further adjusts the calibration parameters through the automation mechanism.

[0034] The feedback loop is triggered by the CI / CD pipeline tool, and the corrected model parameters are re-entered into the system for verification. This process ensures that the calibration parameters of each version are verified and optimized through continuous integration.

[0035] Step S5: Result output and deployment: The optimized calibration parameters are deployed to the vehicle's sensor fusion system and tested and updated with real-time data. After the optimized calibration parameters pass the test, the system deploys the final results to the vehicle's sensor fusion system. After the calibration results are deployed, the perception model continues to provide real-time feedback and is continuously adjusted and optimized through the cloud platform to ensure the adaptability and robustness of the calibration process.

[0036] Application example: Multi-sensor fusion parameter calibration in autonomous vehicles.

[0037] 1. Experimental Background In autonomous vehicles, 4D millimeter wave radar and lidar are two key sensors that provide real-time perception information of the vehicle's surroundings. In order to improve the reliability and accuracy of the system, it is necessary to accurately calibrate the parameters of the fusion algorithm of these two sensors. This experiment aims to achieve automated, efficient and adaptable parameter calibration through a method based on the CI / CD framework.

[0038] 2. Sensor configuration: 4D millimeter-wave radar: used to detect the distance, speed, azimuth and pitch angle of the target.

[0039] LiDAR: Used to generate high-precision 3D point cloud data as a true reference.

[0040] Test vehicle: Equipped with the above sensors, driving on highways and urban roads.

[0041] Data collection: 10 hours of sensor data were collected in the highway and urban road scenes, including 4D millimeter wave radar perception data and lidar point cloud data.

[0042] Cloud platform configuration: Use AWS cloud platform for data storage and processing. Use Jenkins as a CI / CD tool to achieve automated testing and parameter optimization.

[0043] 3. Data Collection and Processing 1. Data Collection Highway scenario: The vehicle traveled at a speed of 80-120 km / h, and 10 hours of 4D millimeter wave radar and lidar data were collected.

[0044] Urban road scene: The vehicle traveled at a speed of 30-60 km / h and collected 10 hours of 4D millimeter wave radar and lidar data.

[0045] 2. Data preprocessing Data cleaning: Remove noise points in lidar data (such as anomalies caused by weather conditions).

[0046] Time synchronization: Time synchronization of 4D radar and lidar data to eliminate delays between different sensors.

[0047] Data formatting: Convert the cleaned data into a unified JSON format for easy subsequent processing.

[0048] 4. Data Fusion and Parameter Calibration 1. Data fusion: The Kalman filter algorithm is used to fuse the data of 4D millimeter wave radar and lidar into a unified coordinate system. During the fusion process, the rotation matrix and translation vector between the sensors are calculated.

[0049] 2. Parameter calibration Initial parameter setting: Based on historical data and experience, set the initial parameters of the Kalman filter (such as the Q matrix and R matrix).

[0050] Automated optimization: Trigger the automated testing process through Jenkins and use the maximum likelihood estimation (MLE) to dynamically adjust the Q matrix and R matrix.

[0051] Optimization algorithm: Genetic algorithm is used to optimize parameters to minimize the perception error.

[0052] 5. Feedback and Correction 1. Feedback mechanism: Developers evaluate the optimized parameters through the feedback module of the cloud platform. If deviations are found in the calibration results (such as insufficient target detection accuracy), developers manually adjust the parameters and trigger a new round of optimization through Jenkins.

[0053] 2. Correction process: In the highway scenario, it was found that the initial calibration parameters would lead to a decrease in target detection accuracy in some cases. The developer manually adjusted the noise covariance of the Q matrix and restarted the optimization process. After three rounds of optimization, the target detection accuracy was significantly improved.

[0054] 6. Result Output and Deployment 1. Result verification: The optimized calibration parameters are verified in highway and urban road scenarios respectively. The mean square error (MSE) is used as the performance indicator to evaluate the target detection accuracy.

[0055] 2. Deployment and real-time update: The optimized calibration parameters are finally deployed to the vehicle’s sensor fusion system. The system continuously adjusts the optimization parameters through real-time data feedback to ensure the adaptability and robustness of the system.

[0056] 7. Data Analysis 1. Performance indicators For highway scenarios: MSE before optimization: 0.25, MSE after optimization: 0.08; detection accuracy improved: 68%.

[0057] For urban road scenarios: MSE before optimization: 0.30, MSE after optimization: 0.10; detection accuracy improved: 67%.

[0058] The optimized calibration parameters improved the target detection accuracy by more than 67% (for example, the MSE in the highway scenario was reduced from 0.25 to 0.08), significantly reducing false positives and negatives.

[0059] 2. Optimize efficiency Optimization time: Each round of optimization takes an average of 2 hours.

[0060] Optimization rounds: An average of 3 rounds of optimization is enough to achieve a satisfactory calibration effect.

[0061] Total optimization time: 6 hours.

[0062] Compared with the traditional manual parameter adjustment method, automated optimization significantly shortens the calibration time (from several days to 6 hours) and improves the system development efficiency. Embodiment 2:

[0063] A multi-sensor fusion parameter calibration method based on a continuous integration deployment framework includes: A data acquisition module is used to automatically collect test data through multiple sensors installed on the test vehicle and transmit the data to the cloud platform; a result output and deployment module; The cloud platform completes the multi-sensor fusion parameter calibration based on the collected data by using the continuous integration and deployment of the CI / CD framework set on the cloud platform; the cloud platform is deployed with a data processing and fusion module, a data processing and fusion module, a parameter calibration and adjustment module, a feedback and correction module, and a result output and deployment module; Among them, the data processing and fusion module pre-processes the collected data, including data cleaning, formatting and fusion processing, and calculates the calibration parameters between multiple sensors; The parameter calibration and adjustment module is used to trigger the automated testing and parameter optimization process through CI / CD tools to adjust and verify the parameters in the perception model; Feedback and correction module, which is used by developers to correct the optimized parameters through the feedback module of the cloud platform, and the system further adjusts the calibration parameters through an automated mechanism; The result output and deployment module is used to deploy the optimized calibration parameters to the vehicle's sensor fusion system and test and update them through real-time data.

[0064] For further optimization, the feedback and correction module corrects the optimized parameters through the feedback mechanism of the cloud platform and performs multiple rounds of optimization through an automated mechanism.

[0065] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A multi-sensor fusion parameter calibration method and system based on a continuous integration deployment framework, characterized in that: The following steps are involved: Step S1: Automated data collection: Collect test data by installing multiple sensors on the test vehicle and transmit the data to the cloud platform; Step S2: Data processing and fusion: The cloud platform first pre-processes the collected data, and then uses the fusion module to comprehensively process the data of multiple sensors and calculate the calibration parameters between multiple sensors; Step S3: Parameter calibration and adjustment: The CI / CD pipeline tool is used to trigger the automated testing and parameter optimization process to adjust the parameters in the perception model. During the optimization process, the cloud platform will automatically adjust the calibration model according to the input test data and provide real-time feedback on the adjustment results. Step S4: Feedback and correction: After the model adjustment is completed, the developer corrects the optimized parameters through the feedback module of the cloud platform, and the system further adjusts the calibration parameters through an automated mechanism; Step S5: Result output and deployment: The optimized calibration parameters are deployed to the vehicle's sensor fusion system and tested and updated with real-time data.

2. The multi-sensor fusion parameter calibration method based on the continuous integration deployment framework according to claim 1 is characterized in that: In step S1, the sensors installed on the test vehicle include but are not limited to millimeter wave radar, 4D radar, laser radar, camera, ultrasonic sensor, inertial measurement unit, GPS, vehicle network sensor and magnetometer.

3. The multi-sensor fusion parameter calibration method based on the CI / CD framework according to claim 2 is characterized in that: In step S2, data preprocessing includes data cleaning and enhancement; wherein, noise and invalid data in the data collected by the sensor are removed by data cleaning; new data samples are generated by performing operations including but not limited to flipping, scaling and splicing on the collected data samples to enhance data diversity.

4. The multi-sensor fusion parameter calibration method based on the continuous integration deployment framework according to claim 3 is characterized in that: In step S2, the fusion module includes but is not limited to Kalman filtering, extended Kalman filtering and unscented Kalman filtering.

5. The multi-sensor fusion parameter calibration method based on the continuous integration deployment framework according to claim 4 is characterized in that: In step S3, the model parameter optimization algorithm includes but is not limited to gradient descent method, genetic algorithm and particle swarm algorithm.

6. The multi-sensor fusion parameter calibration method based on the continuous integration deployment framework according to claim 5 is characterized in that: In step S4, a feedback loop is triggered by the CI / CD pipeline tool, and the corrected model parameters are input into the system again for verification. This process ensures that the calibration parameters of each version are verified and optimized through continuous integration.

7. A multi-sensor fusion parameter calibration system based on a continuous integration deployment framework, characterized in that: include: A data acquisition module is used to automatically collect test data through multiple sensors installed on the test vehicle and transmit the data to the cloud platform; a result output and deployment module; The cloud platform completes the multi-sensor fusion parameter calibration based on the collected data by using the continuous integration and deployment of the CI / CD framework set on the cloud platform; the cloud platform is deployed with a data processing and fusion module, a data processing and fusion module, a parameter calibration and adjustment module, a feedback and correction module, and a result output and deployment module; Among them, the data processing and fusion module pre-processes the collected data, including data cleaning, formatting and fusion processing, and calculates the calibration parameters between multiple sensors; The parameter calibration and adjustment module is used to trigger the automated testing and parameter optimization process through CI / CD tools to adjust and verify the parameters in the perception model; Feedback and correction module, which is used by developers to correct the optimized parameters through the feedback module of the cloud platform, and the system further adjusts the calibration parameters through an automated mechanism; The result output and deployment module is used to deploy the optimized calibration parameters to the vehicle's sensor fusion system and test and update them through real-time data.

8. The multi-sensor fusion parameter calibration system based on continuous integration deployment framework according to claim 7, characterized in that: The feedback and correction module corrects the optimized parameters through the feedback mechanism of the cloud platform and performs multiple rounds of optimization through an automated mechanism.

Images