Automatic driving control method and system based on multi-source data aggregation

Abstract

The invention discloses an automatic driving control method and system based on multi-source data aggregation, and particularly relates to the technical field of automatic driving. The method comprises the following steps: acquiring sensor data, extracting an instantaneous relative frequency drift feature and a sensor data fusion drift error feature, evaluating the accuracy of sensor data time synchronization based on an adaptive weighted fusion algorithm, and dynamically adjusting the confidence coefficient weight of each piece of sensor data. The recurrent neural network is used for predicting the future movement track of the obstacle on the data with high time synchronization accuracy, in addition, through a confidence feedback mechanism, the sensor time synchronization accuracy and the environment perception data quality are combined, the data fusion error is dynamically corrected, the control precision and the driving stability are improved, and the driving stability is improved. According to the method, the problem of data mismatching caused by time synchronization errors can be effectively reduced, the sensing accuracy, trajectory prediction capability and safety of an automatic driving system are improved, and the vehicle has more stable and intelligent decision-making capability in a complex traffic environment.

Description

Technical Field

[0001] The present invention relates to the technical field of impact drills, and specifically relates to an automatic driving control method and system based on multi-source data aggregation. Background Art

[0002] Automatic driving control based on multi-source data aggregation means that an automatic driving system comprehensively perceives and analyzes the environment by integrating data from multiple sensors (such as lidar, millimeter-wave radar, cameras, GPS, IMU, etc.), combined with high-precision maps and vehicle networking information. This data fusion technology can enhance the vehicle's understanding of complex traffic scenarios, thereby enabling more accurate path planning, dynamic obstacle avoidance, driving strategy optimization and other control decisions.

[0003] Compared with a single sensor, the control method using multi-source data aggregation can effectively make up for the limitations of various sensors. For example, cameras are greatly affected by light, while lidar is more sensitive to bad weather. Through the complementary fusion of multi-sensor information, the system can improve the accuracy and reliability of data, enabling autonomous driving vehicles to maintain stable perception and decision-making capabilities in different scenarios, thereby enhancing driving safety and efficiency.

[0004] The prior art has the following deficiencies: Data from multiple sensors usually have different sampling frequencies and data processing delays. For example, the data frame rate of cameras is usually high, while the scanning period of lidar is long, and the echo processing time of millimeter-wave radar is also different. This time synchronization error may cause problems such as historical data mismatching with current data during data fusion. For example, in the case of high-speed driving or emergency obstacle avoidance, if the camera detects an obstacle but the historical data of the lidar fails to be updated in time, the system may misjudge the position of the obstacle, and then make incorrect braking or steering decisions, resulting in serious accidents. Summary of the Invention

[0005] The purpose of the present invention is to provide an automatic driving control method and system based on multi-source data aggregation to solve the deficiencies in the background art.

[0006] To achieve the above purpose, the present invention provides the following technical solutions: An automatic driving control method based on multi-source data aggregation, comprising the following steps: S1: Obtain data from multiple heterogeneous sensors, including lidar, millimeter-wave radar, cameras, GPS, and inertial measurement units; S2: Align the timestamps of each sensor data, perform time synchronization compensation based on the time interpolation algorithm, and respectively extract the instantaneous relative frequency drift characteristics and sensor data fusion drift error characteristics in each sensor data; S3: Based on the adaptive weighted fusion algorithm, combine the extracted instantaneous relative frequency drift features and the sensor data fusion drift error features to judge the accuracy of the sensor data time synchronization, dynamically adjust the confidence weights of each sensor data, and optimize the environmental perception data; S4: For the accurately time-synchronized data, based on the fused perception data, use a recurrent neural network to predict the future motion trajectory of the obstacle; S5: Based on the accuracy of the sensor data time synchronization and the current environmental perception data, use the confidence feedback mechanism to dynamically correct the data fusion error, and improve the control accuracy and driving stability.

[0007] Preferably, in S2, after analyzing the extracted instantaneous relative frequency drift features, generate an instantaneous relative frequency drift index. The acquisition method of the instantaneous relative frequency drift index is as follows: Let the actual sampling frequency time series of different sensors be: ; where represents the actual sampling frequency of a certain sensor at time , n is the number of sampling points, and the theoretical sampling frequency is set as ; Decompose the non-stationary signal into a series of intrinsic mode functions IMF: ; where K is the number of decomposed IMFs, is the kth intrinsic mode function, r(t) is the residual term, select the IMF component containing instantaneous frequency information, denoted as , perform Hilbert transform on the selected IMF. The Hilbert transform is used to calculate the analytic signal of the signal: ; Construct the analytic signal: ; where j is the imaginary unit, and the instantaneous frequency is calculated from the phase angle θ(t) of the analytic signal: ; The instantaneous frequency calculation formula: ; Calculate the instantaneous relative frequency drift index IRFDI, and the expression is: .

[0008] Preferably, in S2, after analyzing the extracted sensor data fusion drift error features, generate a data fusion drift error index. The acquisition method of the data fusion drift error index is as follows: Set the reference sensor data time series and the sensor data time series to be aligned ; Calculate the optimal time alignment path W and the cumulative time deviation D(W) between the two time series: ; where and represent the timestamps of the data points respectively. After normalization, the data fusion drift error index is obtained: ; where, is the data fusion drift error index, and m is the total number of data in the time series of sensor data to be aligned.

[0009] Preferably, in S3, based on the adaptive weighted fusion algorithm, combining the extracted instantaneous relative frequency drift feature and the sensor data fusion drift error feature, the accuracy of the sensor data time synchronization is judged, specifically: normalizing the instantaneous relative frequency drift index and the data fusion drift error index so that they are both in the range of [0, 1], and obtaining the accuracy index of the sensor data time synchronization through weighted average calculation of the normalized instantaneous relative frequency drift index and the data fusion drift error index.

[0010] Preferably, compare the obtained accuracy index of the time synchronization of each sensor data with the accuracy reference threshold preset according to historical data. If the accuracy index of the time synchronization of the sensor data is greater than or equal to the preset accuracy reference threshold, it indicates that the accuracy of the time synchronization of the sensor data is high, and mark it as accurate time synchronization data. At this time, increase the confidence weight of the sensor data and optimize the environmental perception data; if the accuracy index of the time synchronization of the sensor data is less than the preset accuracy reference threshold, it indicates that the accuracy of the time synchronization of the sensor data is low, and mark it as inaccurate time synchronization data. At this time, reduce the confidence weight of the sensor data.

[0011] Preferably, in S4, for the accurate time synchronization data, based on the fused perception data, a recurrent neural network is used to predict the future motion trajectory of the obstacle, specifically: Let the historical motion state of the obstacle be: ; where: is the position coordinate of the obstacle at time t, is the speed of the obstacle, is the acceleration of the obstacle, is the motion direction angle; The prediction task is based on the state sequence in the past T time steps, and estimate the trajectory in the future H time steps: ; where H is the prediction range; The RNN inputs the historical trajectory data in a sliding window manner: ; where: is the current hidden state, which contains historical motion information. W, U, b are the weight matrix and bias term of the RNN, which need to be optimized through training, and f is the activation function; Use the RNN to calculate the future state: ; where: g is the activation function of the output layer, and V and c are the parameters of the output layer. In an autoregressive manner, the predicted trajectory points are used as the input for the next step: ; By successive recursion, a complete future trajectory is obtained.

[0012] Preferably, in S5, based on the accuracy of sensor data time synchronization and the current environmental perception data, a confidence feedback mechanism is used to dynamically correct the data fusion error, specifically as follows: The current environmental perception data EDQ and the accuracy index TSAI of sensor data time synchronization are used as the input items of fuzzy logic, and the correction strategy FEC of data fusion error is used as the output item of fuzzy logic; The input variables TSAI and EDQ are fuzzified, that is, they are converted into fuzzy sets; A fuzzy rule base is established to determine the data fusion error correction strategy in different situations; Mamdani fuzzy inference is used for calculation. The membership degrees of the inputs TSAI and EDQ in different fuzzy sets are calculated. According to the rule base, all matching rules are found, and the rule strength is calculated. The weighted average method is used to calculate the fuzzy value of the fuzzy output FEC; The fuzzy set is converted into a specific correction value, and the obtained FEC is used to dynamically adjust the confidence weight of sensor data to optimize the data fusion error.

[0013] Preferably, if FEC exceeds the pre-set normal error threshold range, the weight of low-confidence sensor data is reduced, and the contributions of lidar and millimeter-wave radar are increased; if FEC is within the pre-set normal error threshold range, it indicates that the sensor data fusion error is low, and the current confidence assignment is maintained to ensure stable environmental perception.

[0014] The present invention also provides an autonomous driving control system based on multi-source data aggregation, including a data acquisition module, a feature extraction module, a confidence adjustment module, a trajectory prediction module, and an error correction and control optimization module; Data acquisition module: Obtain data from multiple heterogeneous sensors, including lidar, millimeter-wave radar, camera, GPS, and inertial measurement unit; Feature extraction module: Align the timestamps of each sensor data, perform time synchronization compensation based on the time interpolation algorithm, and extract the instantaneous relative frequency drift feature and the sensor data fusion drift error feature in each sensor data respectively; Confidence adjustment module: Based on the adaptive weighted fusion algorithm, combined with the extracted instantaneous relative frequency drift feature and the sensor data fusion drift error feature, judge the accuracy of sensor data time synchronization, dynamically adjust the confidence weight of each sensor data, and optimize the environmental perception data; Trajectory Prediction Module: For accurate time-synchronized data, based on the fused perception data, a recurrent neural network is used to predict the future motion trajectory of obstacles. Error Correction and Control Optimization Module: Based on the accuracy of sensor data time synchronization and the current environmental perception data, a confidence feedback mechanism is used to dynamically correct the data fusion error, improving control accuracy and driving stability.

[0015] In the above technical solution, the technical effects and advantages provided by the present invention are as follows: 1. The present invention obtains data from lidar, millimeter-wave radar, cameras, GPS, and inertial measurement units, and uses timestamp alignment and time interpolation algorithms for time synchronization compensation, effectively reducing the risk of mismatch between historical data and current data. Combining the instantaneous relative frequency drift characteristics and the sensor data fusion drift error characteristics, the accuracy of time synchronization is evaluated based on the adaptive weighted fusion algorithm, and the confidence weights of each sensor are dynamically adjusted, thereby optimizing the data fusion quality. For data with high time synchronization accuracy, the present invention uses a recurrent neural network (RNN) to predict the future motion trajectory of obstacles, improving the system's understanding ability of dynamic targets and ensuring more accurate obstacle avoidance and path planning decisions can be made under high-speed driving or complex environments.

[0016] 2. The present invention uses a confidence feedback mechanism to dynamically correct the data fusion error to improve the control accuracy and driving stability of the autonomous driving system. Through fuzzy logic reasoning, taking the accuracy index of sensor data time synchronization and the quality of current environmental perception data as inputs, combining with the fuzzy rule base to calculate the optimal data fusion error correction strategy, and dynamically adjusting the weights of each sensor data. If the fusion error exceeds the preset range, the influence of low-confidence sensor data is reduced, and the contribution of stable sensors such as lidar and millimeter-wave radar is increased; if the fusion error is within the normal range, the current confidence assignment is maintained to ensure the stability of environmental perception. Overall, the present invention significantly improves the data fusion accuracy, trajectory prediction accuracy, and control stability of the autonomous driving system, making it more secure and reliable in complex road environments. Description of the Drawings

[0017] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0018] Figure 1 It is a flowchart of the method of the present invention.

[0019] Figure 2 It is a system module diagram of the present invention. Detailed implementation manners

[0020] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] Example 1. Please refer to Figure 1 As shown, a method for autonomous driving control based on multi-source data aggregation in this embodiment includes the following steps: S1: Obtain data from multiple heterogeneous sensors, including lidar, millimeter-wave radar, camera, GPS, and inertial measurement unit; S2: Align the timestamps of the data of each sensor, perform time synchronization compensation based on the time interpolation algorithm, and respectively extract the instantaneous relative frequency drift feature and the sensor data fusion drift error feature in the data of each sensor; S3: Based on the adaptive weighted fusion algorithm, combine the extracted instantaneous relative frequency drift feature and the sensor data fusion drift error feature, judge the accuracy of the time synchronization of the sensor data, dynamically adjust the confidence weights of the data of each sensor, and optimize the environmental perception data; S4: For the accurately time-synchronized data, based on the fused perception data, use a recurrent neural network to predict the future motion trajectory of the obstacle; S5: Based on the accuracy of the time synchronization of the sensor data and the current environmental perception data, use the confidence feedback mechanism to dynamically correct the data fusion error, and improve the control accuracy and driving stability.

[0022] Obtaining data from multiple heterogeneous sensors is a key step for an autonomous driving system to achieve high-precision environmental perception. These sensors each have different working principles and data characteristics. By fusing this data, the autonomous driving system can more comprehensively understand the surrounding environment and improve the perception accuracy and safety.

[0023] Lidar is mainly used to construct a high-precision three-dimensional point cloud map. By emitting laser beams and measuring their return times, the relative distances and shapes of objects are calculated. Lidar can work stably under different lighting conditions and has strong analytical ability for the spatial structure of targets. However, its data acquisition rate is low, and it may be interfered by raindrops, snowflakes or fog in bad weather conditions, affecting the measurement accuracy.

[0024] Millimeter-wave radar is mainly used to detect the speed, distance, and movement direction of objects. It relies on electromagnetic wave reflection for ranging and has strong anti-interference ability, maintaining good performance even in rainy and foggy weather. Compared with lidar, millimeter-wave radar has lower data dimensions and is difficult to accurately identify the shape of objects, but it performs excellently in dynamic target detection and is suitable for detecting pedestrians, vehicles, and other moving obstacles.

[0025] The camera is an important sensor for the autonomous driving system to obtain visual information, capable of providing rich image data to support tasks such as target recognition, traffic sign reading, and lane line detection. It can analyze scene information through deep learning algorithms to achieve an understanding of complex traffic environments. However, the working performance of the camera is greatly affected by environmental illumination. For example, in strong light, shadows, or nighttime conditions, its recognition ability may decline. In addition, two-dimensional images lack direct depth information and usually need to be supplemented with other sensor data.

[0026] GPS (Global Positioning System) is used to obtain the geographical location information of the vehicle and is usually combined with high-precision maps for global path planning and navigation. High-precision GPS (RTK-GPS) can provide centimeter-level positioning accuracy, but in tunnels, areas with dense high-rise buildings, or signal-blocked environments, the positioning accuracy may decline. Therefore, GPS data usually needs to be fused with an Inertial Measurement Unit (IMU) to improve the continuity and stability of positioning.

[0027] The Inertial Measurement Unit (IMU) measures the vehicle's attitude, angular velocity, and acceleration through gyroscopes and accelerometers. When GPS signals are limited, the IMU can be used for short-term autonomous positioning and navigation. IMU data usually has cumulative errors and needs to be corrected with information such as GPS and wheel odometers to ensure accuracy for long-term use.

[0028] Generally speaking, the autonomous driving system needs to obtain data from multiple heterogeneous sensors and perform time synchronization, coordinate alignment, and data fusion to improve the accuracy of environmental perception. The complementarity between sensors enables the system to maintain stable perception ability under different working conditions and provides reliable data support for autonomous driving decisions.

[0029] In step S2, the autonomous driving system needs to align the timestamps of each sensor's data and perform time synchronization compensation based on the time interpolation algorithm to ensure the accuracy of data fusion. In addition, it is also necessary to extract the instantaneous relative frequency drift characteristics and sensor data fusion drift error characteristics in each sensor's data to optimize the system's correction ability for time synchronization errors.

[0030] First, timestamp alignment is the basis of data synchronization. The data acquisition times of different sensors may vary. For example, the scanning period of lidar is usually 100 ms, the echo processing time of millimeter-wave radar is about 50 ms, and the frame rate of cameras can reach 30 Hz (i.e., 33.3 ms). The timestamps of these sensors need to be calibrated based on a unified clock source (such as GPS timing or the unified clock of the ECU). To reduce timestamp errors, the constant offset correction method or the time synchronization strategy based on Kalman filtering can be adopted to ensure that the data of all sensors are aligned under the same time reference.

[0031] Secondly, time interpolation algorithms are used for time synchronization compensation to address the issue of asynchronous sensor data sampling times. Common methods include linear interpolation, spline interpolation, and interpolation based on motion models. For example, when fusing camera and millimeter-wave radar data at a certain moment, if the timestamp of the latest data point of the millimeter-wave radar is slightly earlier than that of the camera data, the linear interpolation method can be used to estimate the echo data at the current moment based on two adjacent millimeter-wave radar data points to match the timestamp of the camera frame. In addition, in scenarios of high-speed vehicle driving or complex environments, interpolation based on motion models (such as the uniform acceleration motion model or Bezier curve interpolation) can improve the accuracy of time compensation and reduce interpolation errors.

[0032] Next, it is necessary to extract the instantaneous relative frequency drift feature. The instantaneous relative frequency drift refers to the deviation ratio of the sampling frequencies of different sensors within a short time window. Since the internal clocks of sensors may be affected by factors such as temperature, voltage, or vibration, the sampling frequency may experience slight drifts, leading to data mismatches after long-term operation. Methods for extracting this feature include calculating the actual number of sampling points of each sensor within a fixed time window and comparing it with the theoretical number of sampling points to calculate the instantaneous frequency offset. This feature can be used to detect and correct sensor clock offsets to reduce the cumulative effect of time synchronization errors.

[0033] Finally, it is necessary to extract the feature of the drift error in sensor data fusion. Due to different data processing delays of different sensors, time drift may occur during data fusion. For example, the data processing time of lidar is relatively long, while the data of cameras can be obtained faster, so there may be a time deviation when fusing the two. Methods for extracting this feature include recording the time when the data processing of each sensor is completed and calculating its deviation from the unified reference clock. Through statistical analysis of these drift errors, the data fusion strategy can be optimized. For example, the dynamic time adjustment algorithm can be adopted to ensure that the data of different sensors are fused within the optimal time window.

[0034] After analyzing the extracted instantaneous relative frequency drift feature, an instantaneous relative frequency drift index is generated. The method for obtaining the instantaneous relative frequency drift index is as follows: Let the actual sampling frequency time series of different sensors be: ; where represents the actual sampling frequency of a certain sensor at time , and n is the number of sampling points. The theoretical sampling frequency is set to . Decompose the non-stationary signal into a series of intrinsic mode functions IMF: ; where K is the number of decomposed IMFs, is the k-th intrinsic mode function, r(t) is the residual term, select the IMF components containing the main instantaneous frequency information (usually the first k IMFs), denoted as , perform Hilbert transform on the selected IMF, and the Hilbert transform is used to calculate the analytic signal of the signal: ; construct the analytic signal: ; where j is the imaginary unit, and the instantaneous frequency is calculated from the phase angle θ(t) of the analytic signal: ; the instantaneous frequency calculation formula: ; calculate the instantaneous relative frequency drift index IRFDI, and the expression is: .

[0035] After analyzing the extracted sensor data fusion drift error characteristics, generate a data fusion drift error index. The method for obtaining the data fusion drift error index is: Set the reference sensor data time series and the sensor data time series to be aligned ; calculate the optimal time alignment path W and the cumulative time deviation D(W) between the two time series: ; where and represent the timestamps of the data points respectively. After normalization, the data fusion drift error index is obtained: ; in the formula, is the data fusion drift error index, and m is the total number of data in the sensor data time series to be aligned.

[0036] In S3, based on the adaptive weighted fusion algorithm, combine the extracted instantaneous relative frequency drift characteristics and sensor data fusion drift error characteristics to judge the accuracy of sensor data time synchronization. Specifically: normalize the instantaneous relative frequency drift index and the data fusion drift error index so that they are both in the range of [0,1], and calculate the accuracy index of sensor data time synchronization through weighted average of the normalized instantaneous relative frequency drift index and the data fusion drift error index.

[0037] For example, the present invention can calculate the accuracy index of sensor data time synchronization using the following formula, and the calculation expression is: ; where is the accuracy index of sensor data time synchronization, IRFDI is the instantaneous relative frequency drift index, is the data fusion drift error index, is the weight coefficient of the instantaneous relative frequency drift index and the data fusion drift error index (which can be optimized according to experimental experience or machine learning), and 0.

[0038] Compare the obtained accuracy index of each sensor data time synchronization with the accuracy reference threshold preset according to historical data. If the accuracy index of the sensor data time synchronization is greater than or equal to the preset accuracy reference threshold, it indicates that the accuracy of the sensor data time synchronization is high. Mark it as accurate time synchronization data, and at this time, increase the confidence weight of the sensor data to optimize the environmental perception data; if the accuracy index of the sensor data time synchronization is less than the preset accuracy reference threshold, it indicates that the accuracy of the sensor data time synchronization is low. Mark it as inaccurate time synchronization data, and at this time, reduce the confidence weight of the sensor data.

[0039] S4: For the accurate time synchronization data, based on the fused perception data, use a recurrent neural network to predict the future movement trajectory of the obstacle.

[0040] The goal of obstacle trajectory prediction is to infer its possible future movement path based on historical movement data. Let the historical movement state of the obstacle be: ; where: is the position coordinate of the obstacle at time t, is the speed of the obstacle, is the acceleration of the obstacle, is the movement direction angle.

[0041] The prediction task is to estimate the trajectory of the next H time steps based on the state sequence of the past T time steps: ; where H is the prediction range, such as the trajectory point 1 second later or 3 seconds later.

[0042] Ordinary RNN may have the problem of gradient disappearance. Therefore, the long short-term memory network (LSTM) or gated recurrent unit (GRU) is usually used for trajectory prediction. These two structures can capture long-term time dependencies more effectively and improve the prediction accuracy.

[0043] Training an RNN requires a large amount of trajectory data with timestamps. The data sources include: Sensor data on real roads, such as the movement trajectories of vehicles, pedestrians, or non-motor vehicles detected by lidar, cameras, and millimeter-wave radars.

[0044] Historical trajectory information of dynamic targets in high-precision maps.

[0045] Trajectory data generated in a simulation environment.

[0046] The input features include: the historical position, speed, acceleration, and direction angle of the target. The target category (pedestrians, cars, cyclists, etc.), and different categories may have different movement patterns. Traffic environment information, such as road constraints, lane line information, signal light status, etc.

[0047] The RNN inputs historical trajectory data in a sliding window manner: ; where: is the current hidden state, containing historical movement information. W, U, and b are the weight matrices and bias terms of the RNN, which need to be optimized through training. f is the activation function, such as tanh or ReLU.

[0048] Use the RNN to calculate the future state: ; where: g is the activation function of the output layer, such as a linear transformation. V and c are the parameters of the output layer. Adopt an autoregressive method, and use the predicted trajectory points as the input for the next step: ; Recursively deduce in turn to obtain the complete future trajectory.

[0049] To improve the prediction accuracy, the following optimization strategies can be adopted: Attention Mechanism: Enhance the attention to key time steps to improve prediction accuracy. Trajectory prediction based on Transformer: Compared with traditional RNNs, Transformer can better model long-term dependencies and is suitable for complex traffic environments. Multi-Modal Fusion: Combine high-precision maps, lane information, and traffic light status to improve the ability to understand the environment. Uncertainty Estimation: Use Bayesian deep learning or Gaussian Mixture Model (GMM) to predict the probability distribution of trajectories instead of a single trajectory point to improve the robustness of the prediction.

[0050] S5: Based on the accuracy of sensor data time synchronization and the current environmental perception data, use a confidence feedback mechanism to dynamically correct the data fusion error and improve the control accuracy and driving stability.

[0051] The accuracy index TSAI of the current environmental perception data EDQ and sensor data time synchronization is used as the input item of fuzzy logic, and the correction strategy FEC of data fusion error is used as the output item of fuzzy logic; First, the input variables TSAI and EDQ are fuzzified, that is, they are converted into fuzzy sets. The fuzzy set of TSAI (accuracy index of sensor data time synchronization): Low, Medium, High; The fuzzy set of EDQ (environmental perception data quality): Poor, Normal, Good; The fuzzy set of FEC (data fusion error correction strategy): Strong Correction, Moderate Correction, Mild Correction, No Correction; The membership function of each fuzzy variable can be defined using a trigonometric function or a Gaussian function. For example, the fuzzification of TSAI is as follows: ; ; Similar membership functions can also be used to define EDQ and FEC.

[0052] A fuzzy rule base is established to determine the data fusion error correction strategy in different situations. The following are some examples of fuzzy rules: When TSAI is low and EDQ is poor, the system needs to perform Strong Correction.

[0053] For example, in rainy weather, the camera perception is blocked, and at the same time, the sensor time synchronization error is large. At this time, the confidence of unreliable data should be greatly reduced, and the contribution of stable sensors such as millimeter-wave radar should be increased.

[0054] When TSAI is low but EDQ is normal or good, Moderate Correction is required.

[0055] For example, the camera data is relatively clear, but the sensor time synchronization error is still large. At this time, the data fusion error can be moderately corrected to reduce the impact of time synchronization problems.

[0056] When TSAI is at a medium level and EDQ is poor, Moderate Correction is still required.

[0057] For example, the sensor time synchronization is relatively stable, but environmental factors cause interference to the lidar (such as thick fog). At this time, the weights of the data of each sensor should be moderately adjusted to reduce the impact of the lidar.

[0058] When TSAI is at a medium level and EDQ is average or good, the system can perform a Mild Correction.

[0059] For example, when the overall data quality of the autonomous driving system is good and there are only minor time synchronization errors, only a small amount of correction is needed to optimize the fusion accuracy.

[0060] When TSAI is high and EDQ is poor, the system still needs to perform a Mild Correction.

[0061] For example, when the time synchronization is very accurate, but the sensor data quality is greatly affected by the environment, the data can be slightly adjusted to improve the overall perception quality.

[0062] When TSAI is high and EDQ is average or good, the system requires No Correction.

[0063] This indicates that the data fusion error is low, the time synchronization accuracy of the sensor data is high, and the quality of the environmental perception data is good. The current fusion strategy can be maintained without additional adjustment.

[0064] Mamdani fuzzy inference is used for calculation. Calculate the membership degrees of the input TSAI and EDQ in different fuzzy sets. For example: If TSAI = 0.4, it has a certain membership degree between "Low" and "Medium". If EDQ = 0.6, it has a certain membership degree between "Normal" and "Good". According to the rule base, find all matching rules and calculate the rule strength (i.e., the minimum membership degree of the fuzzy set). Use the max-min composition method or the weighted average method to calculate the fuzzy value of the fuzzy output FEC.

[0065] Convert the fuzzy set to a specific correction value. For example, use the Centroid Method to calculate FEC: ; where: is the strength (weight) of the i-th FEC rule. is the correction strategy value corresponding to the rule (e.g., "Strong Correction" = 1.0, "Moderate Correction" = 0.7, "MildCorrection" = 0.3, "No Correction" = 0.0). The finally obtained FEC will be used to dynamically adjust the confidence weights of the sensor data to optimize the data fusion error.

[0066] If the FEC exceeds the pre-set normal error threshold range, reduce the weight of the low-confidence sensor data, for example, reduce the influence of GPS or interfered camera data, and increase the contribution of lidar and millimeter-wave radar. If the FEC is within the pre-set normal error threshold range, it indicates that the sensor data fusion error is low, maintain the current confidence assignment, and ensure stable environmental perception.

[0067] In this embodiment, first, heterogeneous sensor data from lidar, millimeter-wave radar, camera, GPS, and inertial measurement unit are acquired and timestamp-aligned, and time synchronization compensation is performed through a time interpolation algorithm. At the same time, the instantaneous relative frequency drift feature and the sensor data fusion drift error feature are extracted. Based on the adaptive weighted fusion algorithm, combined with the extracted time synchronization error features, the accuracy of sensor data time synchronization is judged, and the confidence weights of each sensor data are dynamically adjusted to optimize the environmental perception data. On this basis, for data with high time synchronization accuracy, a recurrent neural network is used to predict the future motion trajectory of obstacles, thereby enhancing the perception ability of the autonomous driving system for dynamic targets. In addition, through the confidence feedback mechanism, combining the accuracy of sensor data time synchronization and the quality of the current environmental perception data, the data fusion error is dynamically corrected, further improving the control accuracy and driving stability, and ensuring the reliability and safety of the autonomous driving system in complex environments.

[0068] Embodiment 2, please refer to Figure 2 As shown, the autonomous driving control system based on multi-source data aggregation in this embodiment includes a data acquisition module, a feature extraction module, a confidence adjustment module, a trajectory prediction module, and an error correction and control optimization module; Data acquisition module: Acquire data from multiple heterogeneous sensors, including lidar, millimeter-wave radar, camera, GPS, and inertial measurement unit; Feature extraction module: Align the timestamps of each sensor data, perform time synchronization compensation based on the time interpolation algorithm, and extract the instantaneous relative frequency drift feature and the sensor data fusion drift error feature in each sensor data respectively; Confidence adjustment module: Based on the adaptive weighted fusion algorithm, combined with the extracted instantaneous relative frequency drift feature and the sensor data fusion drift error feature, judge the accuracy of sensor data time synchronization, dynamically adjust the confidence weights of each sensor data, and optimize the environmental perception data; Trajectory prediction module: For data with accurate time synchronization, based on the fused perception data, use a recurrent neural network to predict the future motion trajectory of obstacles; Error correction and control optimization module: Based on the accuracy of sensor data time synchronization and the current environmental perception data, a confidence feedback mechanism is used to dynamically correct the data fusion error, improving the control accuracy and driving stability.

[0069] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0070] It should be understood that the term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship. The specific meaning can be understood by referring to the context before and after.

[0071] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0072] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application.

Claims

1. An autonomous driving control method based on multi-source data aggregation, characterized in that: It includes the following steps: S1: Obtain data from multiple heterogeneous sensors, including lidar, millimeter-wave radar, cameras, GPS, and inertial measurement units; S2: Align the timestamps of the data of each sensor, perform time synchronization compensation based on the time interpolation algorithm, and extract the instantaneous relative frequency drift characteristics and sensor data fusion drift error characteristics in the data of each sensor respectively; S3: Based on the adaptive weighted fusion algorithm, combine the extracted instantaneous relative frequency drift characteristics and sensor data fusion drift error characteristics to judge the accuracy of the time synchronization of the sensor data, dynamically adjust the confidence weights of the data of each sensor, and optimize the environmental perception data; S4: For the accurately time-synchronized data, based on the fused perception data, use a recurrent neural network to predict the future motion trajectory of obstacles; S5: Based on the accuracy of the time synchronization of the sensor data and the current environmental perception data, use the confidence feedback mechanism to dynamically correct the data fusion error and improve the control accuracy and driving stability.

2. The method for autonomous driving control based on multi-source data aggregation according to claim 1, wherein: In S2, after analyzing the extracted instantaneous relative frequency drift characteristics, an instantaneous relative frequency drift index is generated. The method for obtaining the instantaneous relative frequency drift index is: Let the actual sampling frequency time series of different sensors be: ; where represents the actual sampling frequency of a certain sensor at time , n is the number of sampling points, and the theoretical sampling frequency is set as ; Decompose the non-stationary signal into a series of intrinsic mode functions IMF: ; where K is the number of decomposed IMFs, is the k-th intrinsic mode function, r(t) is the residual term, select the IMF component containing instantaneous frequency information, denoted as , perform Hilbert transform on the selected IMF, and the Hilbert transform is used to calculate the analytic signal of the signal: ; Construct the analytic signal: ; where j is the imaginary unit, and the instantaneous frequency is calculated from the phase angle θ(t) of the analytic signal: ; The instantaneous frequency calculation formula: ; Calculate the instantaneous relative frequency drift index IRFDI, and the expression is: .

3. The automatic driving control method based on multi-source data aggregation according to claim 2, wherein: In S2, after analyzing the extracted sensor data fusion drift error characteristics, a data fusion drift error index is generated. The method for obtaining the data fusion drift error index is: Set the reference sensor data time series and the sensor data time series to be aligned ; Calculate the optimal time alignment path W and the cumulative time deviation D(W) between two time series: ; where and represent the timestamps of the data points respectively, and after normalization, the data fusion drift error index is obtained: ; in the formula, is the data fusion drift error index, and m is the total number of data in the sensor data time series to be aligned.

4. The automatic driving control method based on multi-source data aggregation according to claim 3, wherein: In S3, based on the adaptive weighted fusion algorithm, combine the extracted instantaneous relative frequency drift characteristics and sensor data fusion drift error characteristics to judge the accuracy of the time synchronization of the sensor data. Specifically: normalize the instantaneous relative frequency drift index and the data fusion drift error index so that they are both within [0,1], and calculate the accuracy index of the time synchronization of the sensor data through weighted averaging of the normalized instantaneous relative frequency drift index and the data fusion drift error index.

5. The automatic driving control method based on multi-source data aggregation according to claim 4, characterized in that: Compare the obtained accuracy index of the time synchronization of the sensor data with the accuracy reference threshold preset according to historical data. If the accuracy index of the time synchronization of the sensor data is greater than or equal to the preset accuracy reference threshold, it indicates that the accuracy of the time synchronization of the sensor data is high, mark it as accurately time-synchronized data, and at this time increase the confidence weight of the sensor data and optimize the environmental perception data; if the accuracy index of the time synchronization of the sensor data is less than the preset accuracy reference threshold, it indicates that the accuracy of the time synchronization of the sensor data is low, mark it as inaccurately time-synchronized data, and at this time reduce the confidence weight of the sensor data.

6. The automatic driving control method based on multi-source data aggregation according to claim 1, characterized in that: In S4, for the accurately time-synchronized data, based on the fused perception data, use a recurrent neural network to predict the future motion trajectory of obstacles. Specifically: Let the historical motion state of the obstacle be: ; where: is the position coordinate of the obstacle at time t, is the speed of the obstacle, is the acceleration of the obstacle, is the motion direction angle; The prediction task is to estimate the trajectory for the next H time steps based on the sequence of states over the past T time steps , where H is the prediction horizon ; The RNN inputs historical trajectory data in a sliding window manner: ; where: is the current hidden state, containing historical motion information, W, U, and b are the weight matrices and bias terms of the RNN, which need to be optimized through training, and f is the activation function; Calculating the future state using RNN: where: g is the activation function of the output layer, V and c are the parameters of the output layer. In an autoregressive manner, the predicted trajectory points are used as the input for the next step: By successive recursion, a complete future trajectory is obtained.

7. A method for autonomous driving control based on multi-source data aggregation according to claim 1, characterized in that: In S5, based on the accuracy of the time synchronization of the sensor data and the current environmental perception data, use the confidence feedback mechanism to dynamically correct the data fusion error. Specifically: Use the current environmental perception data EDQ and the accuracy index TSAI of the time synchronization of the sensor data as the input items of fuzzy logic, and use the correction strategy FEC of the data fusion error as the output item of fuzzy logic; Fuzzify the input variables TSAI and EDQ, that is, convert them into fuzzy sets; Establish a fuzzy rule base to determine the data fusion error correction strategy under different conditions; Use Mamdani fuzzy inference for calculation, calculate the membership degrees of the input TSAI and EDQ in different fuzzy sets, find all matching rules according to the rule base, calculate the rule strengths, and use the weighted average method to calculate the fuzzy value of the fuzzy output FEC; Convert the fuzzy set into a specific correction value, and the obtained FEC will be used to dynamically adjust the confidence weights of sensor data to optimize the data fusion error.

8. The automatic driving control method based on multi-source data aggregation according to claim 7, characterized in that: If the FEC exceeds the pre-set normal error threshold range, reduce the weights of the low-confidence sensor data and increase the contributions of lidar and millimeter-wave radar; if the FEC is within the pre-set normal error threshold range, it means that the sensor data fusion error is low, maintain the current confidence assignment, and ensure stable environmental perception.

9. An autonomous driving control system based on multi-source data aggregation, which is used to implement an autonomous driving control method based on multi-source data aggregation according to any one of claims 1-8, characterized in that: It includes a data acquisition module, a feature extraction module, a confidence adjustment module, a trajectory prediction module, and an error correction and control optimization module; Data acquisition module: Obtain data from multiple heterogeneous sensors, including lidar, millimeter-wave radar, camera, GPS, and inertial measurement unit; Feature extraction module: Align the time stamps of the data of each sensor, perform time synchronization compensation based on the time interpolation algorithm, and extract the instantaneous relative frequency drift feature and the sensor data fusion drift error feature in the data of each sensor respectively; Confidence adjustment module: Based on the adaptive weighted fusion algorithm, combine the extracted instantaneous relative frequency drift feature and the sensor data fusion drift error feature, judge the accuracy of the sensor data time synchronization, dynamically adjust the confidence weights of the data of each sensor, and optimize the environmental perception data; Trajectory prediction module: For the accurately time-synchronized data, based on the fused perception data, use a recurrent neural network to predict the future motion trajectory of obstacles; Error correction and control optimization module: Based on the accuracy of the sensor data time synchronization and the current environmental perception data, use the confidence feedback mechanism to dynamically correct the data fusion error and improve the control accuracy and driving stability.

Images