Unmanned vehicle sensor detection data fusion processing system

Abstract

The invention relates to the technical field of unmanned driving, and discloses an unmanned vehicle sensor detection data fusion processing system which comprises an environment sensing module, a rule generation module, a dynamic execution module and an online optimization module. The environment sensing module collects four-dimensional environment parameters, point cloud data of a laser radar, image data of a camera and distance data of an ultrasonic sensor in real time through multi-sensor integration; the rule generation module generates an IF-THEN fuzzy rule base through a random forest rule engine based on data of an actual driving scene; the dynamic execution module is used for matching a fuzzy rule base according to four-dimensional environmental parameters and adjusting the weight of multi-sensor data fusion; and the online optimization module continuously iteratively updates the fuzzy rule through a genetic algorithm according to the performance feedback of the sensor. According to the invention, through cooperation of the multi-modal sensor and self-evolution of the fuzzy rule base, the environment perception capability and decision reliability of the unmanned vehicle in a complex scene are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to a sensor detection data fusion processing system for an unmanned vehicle. Background Art

[0002] In the early stages of autonomous driving technology, vehicles relied primarily on simple sensors to obtain limited environmental information. These early sensors were scarce and limited in performance, enabling only basic distance detection and rough environmental recognition. The vehicle's driving decision-making process was extremely simplistic, making it difficult to cope with complex and ever-changing road scenarios. Autonomous driving was limited to specific, simple environments, unable to meet the diverse demands of real-world traffic scenarios.

[0003] With the continuous evolution of technology, multi-sensor fusion technology has emerged and has gradually become a key path to improving environmental perception capabilities. However, existing multi-sensor fusion systems are generally flawed. Most fail to fully consider the combined impact of complex and changing environmental factors and the vehicle's real-time driving status on sensor performance. Existing systems struggle to achieve optimal sensor weight allocation in various scenarios, resulting in environmental perception accuracy and reliability that fail to meet the stringent requirements of practical autonomous driving applications.

[0004] Chinese invention patent CN119148145A discloses a multi-sensor fusion system for autonomous driving in inclement weather. Its core innovation lies in the construction of an adaptive multimodal perception architecture. The system first completes sensor calibration and spatiotemporal alignment in good weather. When weather conditions exceed the perception threshold of a single sensor, the multi-sensor fusion mechanism is triggered. However, this invention only handles inclement weather through a pre-set algorithm, resulting in low-dimensional environmental modeling and a lack of closed-loop optimization mechanisms.

[0005] Chinese invention patent CN113111905B achieves accurate obstacle detection by fusing data from multi-line lidar and ultrasonic sensors. This includes ground removal, point cloud segmentation and clustering, and voxelized grid filtering for lidar data, as well as time series processing, multi-weight filtering, and scatter point operations for ultrasonic data. This invention triggers multi-sensor fusion only when the single-sensor perception calculation threshold is exceeded, lacking a dynamic adjustment mechanism.

[0006] Therefore, the present invention proposes a sensor detection data fusion processing system for unmanned vehicles. Summary of the Invention

[0007] The present invention provides a sensor detection data fusion and processing system for unmanned vehicles to solve the problems of multi-sensor data fusion and environmental perception accuracy faced by unmanned driving in multi-modal complex environments.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions: The present invention provides a sensor detection data fusion processing system for an unmanned vehicle, comprising: Environmental perception module, rule generation module, dynamic execution module and online optimization module; The environmental perception module is used to collect weather information, light intensity, road surface conditions, and vehicle speed in real time to generate four-dimensional environmental parameters. By integrating lidar, cameras, and ultrasonic sensors, it collects perception data of the vehicle's surrounding environment, providing a data source for subsequent data fusion processing to improve the vehicle's perception and decision-making capabilities in complex environments. The rule generation module generates fuzzy rules in the form of IF-THEN based on data of actual driving scenarios through a random forest rule engine, and the fuzzy rules are stored in a fuzzy rule library; The dynamic execution module is used to match the fuzzy rule base according to the four-dimensional environmental parameters, adjust the weight of the laser radar, camera and ultrasonic sensor data fusion, and realize the smooth transition of the weight in the event of sudden changes in the environment through the synergy of the mathematical constraints of the smooth transition of the weight and the conflict resolution strategy; The online optimization module is used to iteratively update the fuzzy rule base through a genetic algorithm according to sensor performance evaluation.

[0009] The beneficial effects brought about by the technical solution provided by the present invention include at least: The present invention integrates lidar, cameras and ultrasonic sensors, combining the advantages of multiple sensors to significantly enhance the vehicle's perception of the surrounding environment. This fusion method can effectively compensate for the limitations of a single sensor in complex environments.

[0010] The present invention dynamically adjusts sensor weights according to real-time environmental conditions and vehicle driving status through the system, ensuring that the advantages of each sensor are fully utilized in different scenarios. This dynamic adjustment mechanism can adapt to various complex environmental conditions and improve the accuracy and reliability of environmental perception.

[0011] The present invention introduces mathematical constraints for smooth transition of weights, defines a weight change rate formula, sets upper and lower limits for the change rate, and ensures that the sum of the weights is 1. This ensures the stability of sensor weights during the switching process and avoids the problem of unstable detection results caused by sudden changes in weights.

[0012] The present invention effectively handles rule conflicts by adopting a fitness priority strategy, a coverage supplement strategy and a comprehensive trade-off strategy, ensuring that the system selects the optimal rule to guide sensor data fusion and vehicle decision-making.

[0013] The present invention iterates the fuzzy rule base every 10 minutes through a genetic algorithm and dynamically optimizes the fitness function to continuously reduce the false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 is a schematic diagram of the system architecture provided by an embodiment of the present invention; Figure 2 Schematic diagram of IMU vibration spectrum analysis provided by an embodiment of the present invention; Figure 3 A schematic diagram of the fuzzy rule generation process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following, in combination with the accompanying drawings and preferred embodiments, describes in detail the specific implementation method, structure, characteristics and effects of an unmanned vehicle sensor detection data fusion processing system proposed in accordance with the present invention.

[0017] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0018] The following examples are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0019] The following describes in detail a specific solution of the unmanned vehicle sensor detection data fusion processing system provided by the present invention with reference to the accompanying drawings.

[0020] This embodiment provides a sensor detection data fusion and processing system for an unmanned vehicle, including: Environmental perception module, rule generation module, dynamic execution module and online optimization module; The environmental perception module collects weather information, light intensity, road surface conditions, and vehicle speed in real time to generate four-dimensional environmental parameters. By integrating lidar, cameras, and ultrasonic sensors, it collects perception data of the vehicle's surroundings, providing a data source for subsequent data fusion processing to improve the vehicle's perception and decision-making capabilities in complex environments. The rule generation module uses the random forest rule engine to generate IF-THEN fuzzy rules based on data from actual driving scenarios. The fuzzy rules are stored in the fuzzy rule library. The dynamic execution module is used to match the fuzzy rule base based on the four-dimensional environmental parameters and adjust the weights of the lidar, camera, and ultrasonic sensor data fusion. Through the synergy of mathematical constraints for smooth weight transition and conflict resolution strategies, the weights can be smoothly transitioned when the environment suddenly changes. The online optimization module is used to iteratively update the fuzzy rule base through genetic algorithm according to the sensor performance evaluation.

[0021] Please refer to Figure 1 It is a schematic diagram of the system architecture provided by an embodiment of the present invention.

[0022] For the unmanned vehicle sensor detection data fusion processing system in this embodiment, its modules should include an environment perception module, a rule generation module, a dynamic execution module, and an online optimization module; 1. Environmental Perception Module The environmental perception module includes: meteorological sensors, light sensors, IMU, vehicle speed sensors, lidar, cameras and ultrasonic sensors; Meteorological sensors are used to collect weather condition information and digitally encode the weather condition information, including: sunny day is 0, rainy day is 1, foggy day is 2, and snowy day is 3; The light sensor is used to measure the light intensity value and quantify the light intensity value into lux units. When the light intensity value range is 0-50, it is judged as dark. When the light intensity value range is 50-1000, it is judged as low light. When the light intensity value range is When the light intensity value range is greater than When , it is judged as strong light; The IMU is used to measure the road surface condition and judge the road surface condition by the road slip coefficient. When the road slip coefficient ranges from 0 to 0.3, it is judged as dry; when the road slip coefficient ranges from 0.3 to 0.6, it is judged as slightly slippery; when the road slip coefficient ranges from 0.6 to 0.8, it is judged as severely slippery; when the road slip coefficient range is greater than 0.8, it is judged as extremely dangerous; The vehicle speed sensor is used to directly obtain the vehicle speed value in km / h. When the vehicle speed range is 0-30 km / h, it is judged as low speed; when the vehicle speed range is 30-80 km / h, it is judged as medium speed; when the vehicle speed range is greater than 80 km / h, it is judged as high speed; LiDAR is used to collect point cloud data of the vehicle's surrounding environment at a frequency of 20Hz; The camera is used to collect image data of the vehicle's surrounding environment at a frequency of 30fps; The ultrasonic sensor is used to collect distance data of the vehicle's surrounding environment. The frequency is 40Hz and the maximum detection distance is 5 meters. Frequency refers to the number of times per second the sensor collects data.

[0023] Please refer to Figure 2 Schematic diagram of IMU vibration spectrum analysis provided by an embodiment of the present invention.

[0024] The road slip coefficient is based on IMU vibration spectrum analysis. The specific steps for calculating the road slip coefficient include: S1, collects Z-axis acceleration data of the IMU. During the vehicle's driving process, the Z-axis acceleration data of the IMU installed on the vehicle is collected in real time. This is the original data source for subsequent analysis; S2, preprocessing the collected data, including denoising and calibration. Since the collected raw data may contain noise and sensor errors, data preprocessing is required; S3, perform Fourier transform on the pre-processed data to convert the time domain signal into a frequency domain signal. Fourier transform is a key step in spectrum analysis. Through this transformation, the time domain acceleration signal is converted into a frequency domain signal, so that the distribution of different frequency components in the signal can be analyzed; S4, extracting the energy E1 in the 5-15 Hz frequency band, which corresponds to the characteristic resonance frequency between the tire and the road surface; S5, extracts the total energy E2 in the 5-50 Hz frequency band, which covers the main vibration frequency band of the vehicle suspension system; S6, calculating an energy ratio E1 / E2, where the energy ratio E1 / E2 is used to reflect the slipperiness of the road; S7, limits the ratio to the range of [0,1] to obtain the slippery coefficient. When the road slippery coefficient range is 0-0.3, it is judged as dry. When the road slippery coefficient range is 0.3-0.6, it is judged as slightly slippery. When the road slippery coefficient range is 0.6-0.8, it is judged as severely slippery. When the road slippery coefficient range is greater than 0.8, it is judged as extremely dangerous.

[0025] It should be noted that a meteorological sensor is a device used to measure and monitor meteorological parameters. It can convert physical quantities in the meteorological environment into measurable electrical signals or other forms of signals for data collection, analysis and processing.

[0026] A light sensor is a device that detects light intensity. It measures the light level in an environment by converting light signals into electrical signals.

[0027] IMU is the abbreviation of Inertial Measurement Unit, which is a sensor device used to measure the acceleration and angular velocity of an object.

[0028] A vehicle speed sensor is a device used to measure the speed of a vehicle. It can convert the vehicle's speed into a measurable electrical signal or other form of signal, thereby providing real-time speed information to the vehicle's control system, driving assistance system, and driver.

[0029] LiDAR is a remote sensing technology that uses lasers for distance measurement and target detection. It calculates the distance and position information of the target object by emitting a laser beam and measuring the time or phase change when the laser beam is reflected from the target object.

[0030] An ultrasonic sensor is a device that uses ultrasonic waves to measure distance, detect objects, and measure speed. It transmits and receives ultrasonic signals and obtains information about the target object based on the signal's propagation time and reflection characteristics.

[0031] Z-axis data measurement is simple and stable. Tire vertical vibration is most sensitive to changes in road conditions and can directly reflect road roughness and slipperiness. Therefore, the IMU's Z-axis acceleration data is used to sense road conditions. The road slip coefficient obtained based on IMU vibration spectrum analysis often has non-stationary characteristics, so wavelet transform is used for denoising. Wavelet transform has unique advantages in processing non-stationary signals. It can analyze signals at different scales, effectively removing noise while retaining important signal features.

[0032] The IMU data is transformed using the Fast Fourier Transform algorithm. Fast Fourier Transform is an efficient algorithm for discrete Fourier transform. It reduces the computational complexity of discrete Fourier transform from Reduce to , which greatly improves the computational efficiency. The specific steps include: S1, collects Z-axis acceleration data from IMU, with a sampling frequency of =500Hz, continuously collect M=1024 data points and store them in array a[m], where Indicates the number of times the signal is sampled per second, in Hertz. M represents the number of continuous data samples used for Fourier transform. In order to obtain higher spectral resolution, a larger number of data points needs to be selected. This system selects the number of data points M=1024. a[m] represents the Z-axis acceleration data sequence of the collected IMU, which is a discrete time series. m is the discrete time index, and the value range is m=0,1,...,M-1. S2, apply the Hanning window b[m] to the collected data a[m], and obtain the formula for the windowed data c[m]: ; Where m=0,1,...,M-1, b[m] represents the Hanning window function sequence, which is a discrete sequence of length M, and c[m] represents the windowed data sequence. The function of the Hanning window function is to reduce spectrum leakage. The formula of the Hanning window function is: ; Among them, π is a mathematical constant. Through the windowing operation, the signal can be made smoother in the time domain, thereby reducing spectrum leakage in the frequency domain.

[0033] S3, use numpy.fft.fft function to perform fast Fourier transform calculation on the windowed data c[m] to obtain frequency domain data ,in represents the frequency domain data sequence obtained after fast Fourier transform calculation, k is the frequency index, and the value range is k=0,1,...,M-1, where M represents the number of data points; S4, calculate the amplitude spectrum of frequency domain data , and convert it into a unilateral spectrum, retaining only the positive frequency part. The amplitude of the unilateral spectrum needs to be multiplied by 2 and divided by the number of data points M to obtain the correct amplitude, where Represents frequency domain data The amplitude spectrum indicates the amplitude of each frequency component. It is plural. is its modulus value.

[0034] In the calculation of the road slipperiness coefficient, signals in the 5-15Hz frequency band primarily reflect small-scale unevenness on the road surface, such as small stones, cracks, or slight potholes. These high-frequency vibrations are typically associated with subtle changes in the road surface and can provide detailed information on the slipperiness of the road. Signals in the 5-50Hz frequency band encompass unevenness ranging from shorter to longer wavelengths and can reflect the overall condition of the road surface. Therefore, the 5-15Hz and 5-50Hz frequency bands are selected for IMU data processing. The system can effectively capture both small-scale and large-scale unevenness on the road surface, thereby providing comprehensive information on the slipperiness of the road.

[0035] 2. Rule Generation Module Please refer to Figure 3 It is a schematic diagram of the fuzzy rule generation process provided by an embodiment of the present invention.

[0036] The rule generation module includes: random forest rule engine and fuzzy rule base; The random forest rule engine generates fuzzy rules based on a data-driven approach and stores them in a fuzzy rule base. The specific steps for generating fuzzy rules include: S1, training a multi-output random forest model using four-dimensional environment parameters; S2, extracting primary rules in the form of IF-THEN by traversing the decision tree nodes; S3, convert the continuous variable splitting threshold into a fuzzy language description; S4, calculate the confidence based on the accuracy of the rule in the validation set; The form of fuzzy rules is: IF weather=W AND light=L AND road=R AND speed=S; THEN w_lidar=α, w_cam=β, w_us=γ; Where W∈{0: sunny, 1: rainy, 2: foggy, 3: snowy} is the digital code of weather conditions, L∈[0,120000] is the ambient light intensity, R∈[0,1] is the road slipperiness coefficient, S∈[0,120] is the vehicle speed, w_lidar=α is the lidar weight, w_cam=β is the camera weight, and w_us=γ is the ultrasonic sensor weight. The weights α, β, and γ correspond to w_lidar, w_cam, and w_us, respectively. The fuzzy rule is added with a confidence parameter h∈[0,1] to modify the weight. The confidence parameter is calculated through ten-fold cross validation. The calculation formula for the modified weight is: ; in, Correct the weights for the lidar, Correction weights for the camera, Corrected weights for ultrasonic sensors.

[0037] It should be noted that the Random Forest Rule Engine is a machine learning model based on the Random Forest algorithm, which is used to generate and optimize fuzzy rules. The Random Forest improves the stability and accuracy of the model by constructing multiple decision trees and combining the prediction results of each tree.

[0038] The fuzzy rule base is a core component in the fuzzy logic control system. It stores a series of fuzzy rules that are used to describe the fuzzy relationship between the system input and output. The main function of the fuzzy rule base is to express complex system behaviors or decision-making processes in the form of a set of fuzzy rules.

[0039] The four-dimensional environmental parameter data used to train the multi-output random forest model comes from aggregated data from a large number of actual driving scenarios. By deploying unmanned vehicles in different regions, seasons, and time periods, and equipping the vehicles with meteorological sensors, light sensors, IMUs, and speed sensors, the four-dimensional environmental parameter data and the corresponding lidar, camera, and ultrasonic sensor weight information are continuously collected. This method can obtain real and comprehensive environmental parameter data that reflects various situations in actual driving.

[0040] The collected four-dimensional environmental parameter data were preprocessed. For weather condition information, the mode was used to fill missing values. For continuous variables including light intensity, road slipperiness coefficient and vehicle speed, the mean filling method was used to handle missing values.

[0041] The Z-score method is used to detect and process outliers. For each feature, the mean and standard deviation of the feature are calculated. The Z-score formula for each data point is: ; Where d is the data point, μ is the mean, and σ is the standard deviation. Values with a Z-score greater than 3 are considered outliers and are replaced by the median of the feature.

[0042] The Min-Max normalization method is used to process continuous variables including light intensity, road slip coefficient and vehicle speed. For each continuous variable, the minimum value of the variable is calculated. and maximum value , use the Min-Max normalization formula to scale the data to the range [0,1]: ; Among them, A is the original data, and are the minimum and maximum values of the feature, respectively.

[0043] To ensure a balance between model performance and efficiency, the following parameter configuration is used for training the random forest model: The number of decision trees was set to 200 and was optimized in the range of 100–500 by grid search; The maximum depth is limited to 10 to prevent overfitting; The split standard regression task uses mean squared error, and the classification task uses Gini impurity; The minimum number of samples in a leaf node is set to 5 to avoid overfitting.

[0044] For decision tree traversal, we traverse each decision tree in the random forest, extract the primary rules in the form of IF-THEN, and obtain all decision trees from the random forest model.

[0045] In the random forest model, the path from the root node to the leaf node of each decision tree can be converted into an IF-THEN rule. The rule extraction uses a depth-first search algorithm to traverse each decision tree, starting from the root node and recursively accessing the splitting conditions of each non-leaf node; the splitting conditions are combined into rule premises in sequence; when reaching a leaf node, the predicted value of the node is recorded.

[0046] To convert the splitting threshold of a continuous variable into a natural language description, it is necessary to predefine the fuzzy language variables and their corresponding numerical ranges. The fuzzy language mapping tables are shown in Tables 1 to 4:

[0047]

[0048]

[0049]

[0050] 3. Dynamic Execution Module The dynamic execution module adjusts the weight of the laser radar, camera and ultrasonic sensor data fusion according to the fuzzy rules output by the rule generation module. The dynamic execution module includes: a weight distribution controller, a data preprocessing unit and a weighted fusion processor; The weight distribution controller is used to match the corresponding fuzzy rules in the fuzzy rule library according to the four-dimensional environmental parameters, calculate and output the corrected weight value of each sensor; The data preprocessing unit is used to preprocess the data from the lidar, camera, and ultrasonic sensor. The data preprocessing unit synchronizes the timestamps of the sensor data through linear interpolation and unifies the sensor data into the vehicle coordinate system through spatial coordinate conversion. The weighted fusion processor is used to perform weighted fusion on the lidar point cloud data, the camera image data, and the ultrasonic sensor distance data to generate the final environmental perception result. The specific steps for the weighted fusion processor to achieve data fusion include: S1, normalize the distance data output by each sensor, and the normalization process adopts Min-Max standardization; S2, calculates the weighted average distance based on dynamic weights The formula is: = + + ; in, represents the target distance after fusion, represents the target distance measured by the lidar, Indicates the target distance calculated by the camera using the parallax method. Indicates the target distance measured by the ultrasonic sensor, and They are the correction weights of lidar, camera and ultrasonic sensor, satisfying + + =1; S3, output fusion results; The dynamic execution module introduces mathematical constraints for smooth weight transition to ensure the stability of sensor weights during the switching process. The specific steps include: S1, define the weight change rate formula as: ; in, represents the weight change rate of sensor l, represents the weight of sensor l at the current time t, represents the weight of sensor l at the next moment t+1, The weight change rate formula is used to constrain the change speed of sensor weights during the switching process to avoid unstable vehicle decision-making due to drastic changes in weights. S2, Settings The upper and lower limits of To determine the maximum weight change rate based on the actual application scenario and sensor performance, set =0.1 / s, ensuring that a single weight adjustment does not exceed ±10%; S3, ensures that the sum of the weights of each sensor is always 1, and , l represents the sensor number, n is the number of sensors; S4, during the weight adjustment process, the above constraints are met with the help of iterative algorithms to achieve a smooth transition of weights.

[0051] It should be noted that the point cloud data of the lidar is a set of discrete points in three-dimensional space generated by emitting a laser beam and receiving its reflected signal. Each point accurately records the spatial coordinates and reflection characteristics of the object surface.

[0052] The image data of the camera is two-dimensional or three-dimensional visual information captured by optical sensors and is used to describe the visual characteristics of the vehicle's surroundings.

[0053] The distance data of the ultrasonic sensor is the relative distance information of the object measured by the principle of sound wave flight time, which is active ranging data.

[0054] For the weight distribution controller, the fuzzy rules in the fuzzy rule base are matched according to the four-dimensional environmental parameters. The weight distribution controller uses weighted Euclidean distance as the similarity metric. For each input four-dimensional environmental parameter, its similarity with all the rules in the fuzzy rule base is calculated, and the rule with the highest similarity is selected as the matching rule.

[0055] For lidar data, Gaussian filtering is used to remove noise. For camera data, histogram equalization is used to enhance image contrast. For ultrasonic sensor data, median filtering is used to remove outliers.

[0056] Different sensors have different sampling frequencies. To make their data correspond in time, timestamp synchronization is required. The specific steps for timestamp synchronization using linear interpolation are as follows: S1, determine the reference timestamp, select the data timestamp of a sensor as the reference, usually the sensor with the highest sampling frequency; S2, find adjacent timestamps. For each data point of other sensors, find its two adjacent timestamps in the timestamp sequence of the reference sensor. and , making ≤ ≤ ,in is the timestamp of the sensor data point to be synchronized; S3, linear interpolation assumes that the sensor to be synchronized is at time There is a data value , the reference sensor at time and There are data values and , you can use the linear interpolation formula to calculate the time The estimated value of the reference sensor The formula is: ; Different sensors are installed at different locations on the vehicle, and their coordinate systems are also different. Spatial coordinate system conversion is required to achieve spatial alignment of the data. The specific steps include: S1, determine the sensor coordinate system, clarify the coordinate system of each sensor, including the lidar coordinate system, camera coordinate system and ultrasonic sensor coordinate system; S2, obtain conversion parameters, obtain conversion parameters between different coordinate systems through calibration, including rotation matrix r and translation vector j; S3, coordinate transformation, uses transformation parameters to transform the point cloud data of the lidar, the image data of the camera, and the distance data of the ultrasonic sensor into the same coordinate system.

[0057] Assumption Point The coordinates in the sensor coordinate system are ( , , ), the coordinates in the target coordinate system are =( , , ), the coordinate transformation formula is: = +j; Weight smooth transition is a stability guarantee mechanism when dynamically adjusting weights in multi-sensor fusion systems. Its core is to prevent perception result jumps caused by weight mutations through mathematical constraints.

[0058] 4. Online Optimization Module The online optimization module includes sensor performance evaluation and a genetic algorithm optimizer. The sensor performance evaluation is used to monitor sensor performance feedback in real time. When there is a deviation between the fused data and the actual road conditions, the genetic algorithm optimizer iteratively updates the fuzzy rule base. The specific steps include: S1, uses chromosome encoding to encode fuzzy rules into gene individuals. The conditional part of the gene includes discrete encoding of weather conditions and continuous encoding of light intensity, road slip coefficient and vehicle speed. The action part of the gene includes continuous encoding of the weights of lidar, camera and ultrasonic sensors. The sensor weights meet + + = 1 normalization constraint; S2, the fitness evaluation function formula is: ; Among them, Fitness represents fitness, Coverage represents rule coverage, Error represents average weighted error, and Performance represents real-time performance score; ; Detection represents the detection rate, FalseAlarm represents the false alarm rate, and Latency represents the delay coefficient. The detection rate and false alarm rate are calculated based on the ultrasonic sensor calibration results. S3, performs genetic operations, uses the tournament selection operator to select excellent individuals from the population, with a tournament size of K=3, uses the two-point crossover operator for gene recombination, and the crossover probability q=0.9. The crossover sites avoid weight normalization constraints, and uses the directed mutation operator to perform discrete value mutation and continuous value mutation within the range of ±0.2 on the conditional part genes and the action part genes. After the operation, the weight renormalization process is automatically performed; S4 implements dynamic update control, performing regular optimization iterations every 10 minutes, with a maximum iteration number of 50. When three consecutive perception errors or sensor failures of the same type are detected, an emergency optimization process is triggered. The emergency optimization process prioritizes adjusting and optimizing the relevant fuzzy rules to quickly resolve the performance degradation caused by the current perception error or sensor failure. The newly generated fuzzy rules must pass simulation verification and then be grayed out on 10% of the vehicle nodes. After confirming that the performance of the fuzzy rules has improved by ≥5% compared to the previous version and there are no new failures, a full update will be performed. A performance improvement of ≥5% refers to a relative improvement in the average precision of target detection. S5, when initializing the population, retain the top 50 existing rules in terms of fitness, and the newly generated fuzzy rules must meet the following conditions: ≥0.4, when driving at high speed ≤0.3 and in dark environment ≥0.2; S6: When the conditions of two rules are fully matched but the weight difference exceeds 20%, they are determined to be conflicting rules. Conflict resolution strategies are adopted for the conflicting rules. The conflict resolution strategies include: fitness priority strategy, coverage supplement strategy and comprehensive trade-off strategy. The fitness-first strategy first compares the fitness of the two conflicting rules on the validation set and retains the rule with higher fitness; Coverage supplement strategy: if the fitness difference between two conflicting rules is less than 5%, their coverage will be further compared and the rule with larger coverage will be retained; Comprehensive trade-off strategy: if the fitness difference is less than 5% and the coverage is the same, the comprehensive score of the two rules is calculated, and the rule with the higher comprehensive score is retained. The comprehensive score formula is: Score=Fitness×ω+Coverage×(1-ω) Among them, Fitness is the fitness, Coverage is the coverage, and ω is the fitness weight coefficient.

[0059] It should be noted that in genetic algorithms, chromosome encoding refers to a method of expressing the solution to the problem to be optimized as a gene sequence to facilitate genetic operations such as selection, crossover, and mutation.

[0060] Conditional genes are gene segments in chromosomes that encode the prerequisites for IF-THEN rules.

[0061] The action part gene is the gene fragment in the chromosome that encodes the conclusion part of the IF-THEN rule, which directly determines the weight distribution of each sensor in data fusion.

[0062] Rule coverage indicates the proportion of scenarios that the rules in the fuzzy rule base can match under the current environmental conditions, reflecting the comprehensiveness and adaptability of the rule base.

[0063] The average weight error is used to measure the deviation between the weight assignment of rules in the fuzzy rule base and the actual optimal weight.

[0064] The detection rate is used to measure the system's ability to detect targets, reflecting the proportion of targets that the system successfully detects when all targets actually exist.

[0065] The false alarm rate is used to measure the proportion of targets that the system incorrectly detects when there is no target, reflecting the frequency with which the system incorrectly reports the existence of a target when there is no actual target.

[0066] The delay coefficient is used to measure the time delay of system processing and response. The delay coefficient reflects the time required from sensor data collection to the system making a decision and executing the corresponding action.

[0067] The tournament selection operator is a commonly used selection operation in genetic algorithms, which is used to select excellent individuals from a population to generate the next generation of population.

[0068] The crossover probability determines the probability that the genetic information of two parent individuals is exchanged in the crossover operation.

[0069] The directional mutation operator is a special mutation operation used to make directional changes to the genes of individuals in genetic algorithms. Genetic algorithm is a search and optimization technology based on the principles of natural selection and genetics. It simulates the biological evolution process and gradually improves the quality of candidate solutions by simulating the genetic mechanism of organisms, thereby finding the optimal solution or approximate optimal solution to the problem.

[0070] Verified by grid search and simulation tests, when the weight ratio of rule coverage, weight error, and real-time performance score in the fitness function is 0.2:0.5:0.3, the system achieves the optimal balance between detection accuracy and stability.

[0071] If the system meets any of the following conditions three times in a row within 60 seconds, it is considered a perception error requiring urgent optimization: The detection rate is less than 80%; The false alarm rate is higher than 10%; The distance measurement difference between the main sensors is greater than 2 meters and lasts for more than 1 second.

[0072] When the number of valid point clouds of the lidar is ≤50% of the historical average and lasts for ≥10 seconds, the weight is reduced to 0.2.

[0073] When the average image clarity of the camera is <0.7 and lasts for ≥5 seconds, switch to the backup camera.

[0074] When the standard deviation of 10 consecutive ultrasonic measurements in a static environment is greater than 0.5 meters, it is marked as low confidence.

[0075] Through sensitivity analysis in the comprehensive trade-off strategy, ω=0.6 is selected, which makes the overall performance of the fuzzy rule base optimal.

[0076] Regarding the unmanned vehicle sensor detection data fusion processing system in this embodiment.

[0077] Furthermore, it should be noted that the present invention may be provided as a method, apparatus, or computer program product. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code.

[0078] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0080] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The terms "comprises," "includes," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0081] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be noted that although the preferred embodiment of the present invention has been described, it is clear that those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered as within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A sensor detection data fusion processing system for unmanned vehicles, characterized in that: include: Environmental perception module, rule generation module, dynamic execution module and online optimization module; The environmental perception module is used to collect weather information, light intensity, road surface conditions, and vehicle speed in real time to generate four-dimensional environmental parameters. By integrating lidar, cameras, and ultrasonic sensors, it collects perception data of the vehicle's surrounding environment, providing a data source for subsequent data fusion processing to improve the vehicle's perception and decision-making capabilities in complex environments. The rule generation module generates fuzzy rules in the form of IF-THEN based on data of actual driving scenarios through a random forest rule engine, and the fuzzy rules are stored in a fuzzy rule library; The dynamic execution module is used to match the fuzzy rule base according to the four-dimensional environmental parameters, adjust the weight of the laser radar, camera and ultrasonic sensor data fusion, and realize the smooth transition of the weight in the event of sudden changes in the environment through the synergy of the mathematical constraints of the smooth transition of the weight and the conflict resolution strategy; The online optimization module is used to iteratively update the fuzzy rule base through a genetic algorithm according to sensor performance evaluation.

2. The unmanned vehicle sensor detection data fusion processing system according to claim 1, characterized in that: The environment perception module includes: Weather sensors, light sensors, IMUs, vehicle speed sensors, lidars, cameras, and ultrasonic sensors; The meteorological sensor is used to collect weather condition information, and the weather condition information is digitally encoded to include: 0 for sunny day, 1 for rainy day, 2 for foggy day and 3 for snowy day; The light sensor is used to measure the light intensity value, quantify the light intensity value into lux units, and when the light intensity value ranges from 0 to 50, it is determined to be dark; when the light intensity value ranges from 50 to 1000, it is determined to be low light; when the light intensity value ranges from When the light intensity value range is greater than When , it is judged as strong light; The IMU is used to measure the road surface condition and determine the road surface condition by a road slip coefficient. When the road slip coefficient ranges from 0 to 0.3, the road is determined to be dry; when the road slip coefficient ranges from 0.3 to 0.6, the road is determined to be slightly slippery; when the road slip coefficient ranges from 0.6 to 0.8, the road is determined to be severely slippery; and when the road slip coefficient range is greater than 0.8, the road is determined to be extremely dangerous. The vehicle speed sensor is used to directly obtain a vehicle speed value in km / h. When the vehicle speed range is 0-30 km / h, it is determined to be a low speed; when the vehicle speed range is 30-80 km / h, it is determined to be a medium speed; when the vehicle speed range is greater than 80 km / h, it is determined to be a high speed; The laser radar is used to collect point cloud data of the vehicle's surrounding environment at a frequency of 20 Hz; The camera is used to collect image data of the vehicle's surrounding environment at a frequency of 30fps; The ultrasonic sensor is used to collect distance data of the vehicle's surrounding environment, with a frequency of 40Hz and a maximum detection distance of 5 meters; The frequency refers to the number of times the sensor collects data per second.

3. The unmanned vehicle sensor detection data fusion processing system according to claim 2, characterized in that: The road slippery coefficient is calculated based on IMU vibration spectrum analysis, and the specific steps of calculating the road slippery coefficient include: S1, collects Z-axis acceleration data of IMU; S2, preprocessing the collected data, including denoising and calibration; S3, performing Fourier transform on the preprocessed data to convert the time domain signal into a frequency domain signal; S4, extracting energy E1 within a frequency band of 5-15 Hz, where the frequency band of 5-15 Hz corresponds to a characteristic resonance frequency between the tire and the road surface; S5, extracting the total energy E2 within the frequency band of 5-50 Hz, where 5-50 Hz covers the main vibration frequency band of the vehicle suspension system; S6, calculating an energy ratio E1 / E2, where the energy ratio E1 / E2 is used to reflect the slipperiness of the road; S7, limiting the ratio to the range of [0,1] to obtain the road slip coefficient.

4. The unmanned vehicle sensor detection data fusion processing system according to claim 1, characterized in that: The rule generation module includes: Random forest rule engine and fuzzy rule base; The random forest rule engine generates fuzzy rules based on a data-driven approach and stores them in a fuzzy rule library. The specific steps of generating the fuzzy rules include: S1, training a multi-output random forest model using four-dimensional environment parameters; S2, extracting primary rules in the form of IF-THEN by traversing the decision tree nodes; S3, convert the continuous variable splitting threshold into a fuzzy language description; S4, calculate the confidence based on the accuracy of the rule in the validation set; The fuzzy rules are in the form of: IF weather=W AND light=L AND road=R AND speed=S; THEN w_lidar=α, w_cam=β, w_us=γ; Where W∈{0: sunny, 1: rainy, 2: foggy, 3: snowy} is the digital code of weather conditions, L∈[0,120000] is the ambient light intensity, R∈[0,1] is the road slipperiness coefficient, S∈[0,120] is the vehicle speed, w_lidar=α is the lidar weight, w_cam=β is the camera weight, and w_us=γ is the ultrasonic sensor weight. The weights α, β, and γ correspond to w_lidar, w_cam, and w_us, respectively. The fuzzy rule is added with a confidence parameter h∈[0,1] to modify the weight. The confidence parameter is calculated by ten-fold cross validation. The calculation formula of the modified weight is: ; in, Correct the weights for the lidar, Correction weights for the camera, Corrected weights for ultrasonic sensors.

5. The unmanned vehicle sensor detection data fusion processing system according to claim 1, characterized in that: The dynamic execution module adjusts the weight of the laser radar, camera and ultrasonic sensor data fusion according to the fuzzy rules output by the rule generation module, and the dynamic execution module includes: Weight distribution controller, data preprocessing unit and weighted fusion processor; The weight distribution controller is used to match the corresponding fuzzy rules in the fuzzy rule base according to the four-dimensional environmental parameters, calculate and output the corrected weight value of each sensor; The data preprocessing unit is used to preprocess the data of the laser radar, camera, and ultrasonic sensor. The data preprocessing unit synchronizes the timestamps of the sensor data by linear interpolation and unifies the sensor data into the vehicle coordinate system by spatial coordinate conversion. The weighted fusion processor is used to perform weighted fusion on the point cloud data of the lidar, the image data of the camera, and the distance data of the ultrasonic sensor to generate a final environmental perception result. The specific steps of the weighted fusion processor to achieve data fusion include: S1, normalize the distance data output by each sensor, and the normalization process adopts Min-Max standardization; S2, calculates the weighted average distance based on dynamic weights The formula is: ; in, represents the target distance after fusion, represents the target distance measured by the lidar, Indicates the target distance calculated by the camera using the parallax method. Indicates the target distance measured by the ultrasonic sensor, 、 and They are the correction weights of lidar, camera and ultrasonic sensor, satisfying ; S3, output fusion results.

6. The unmanned vehicle sensor detection data fusion processing system according to claim 5, characterized in that: The dynamic execution module introduces mathematical constraints for smooth weight transition to ensure the stability of sensor weights during the switching process. The specific steps include: S1, define the weight change rate formula as: ； in, represents the weight change rate of sensor l, represents the weight of sensor l at the current time t, represents the weight of sensor l at the next moment t+1, The weight change rate formula is used to constrain the change speed of sensor weights during the switching process to avoid unstable vehicle decision-making due to drastic changes in weights. S2, Settings The upper and lower limits of To determine the maximum weight change rate based on the actual application scenario and sensor performance, set , ensure that the single weight adjustment does not exceed ±10%; S3, ensures that the sum of the weights of each sensor is always 1, , l represents the sensor number, n is the number of sensors; S4, during the weight adjustment process, the above constraints are met with the help of iterative algorithms to achieve a smooth transition of weights.

7. The unmanned vehicle sensor detection data fusion processing system according to claim 1, characterized in that: The online optimization module includes: sensor performance evaluation and genetic algorithm optimizer; The sensor performance evaluation is used to monitor sensor performance feedback in real time. When there is a deviation between the fused data and the actual road conditions, the fuzzy rule base is iteratively updated through the genetic algorithm optimizer. The specific steps include: S1, uses chromosome encoding to encode fuzzy rules into gene individuals. The conditional part of the gene includes discrete encoding of weather conditions and continuous encoding of light intensity, road slip coefficient and vehicle speed. The action part of the gene includes continuous encoding of the weights of the lidar, camera and ultrasonic sensor. The sensor weights meet Normalization constraints; S2, the fitness evaluation function formula is: ； in, represents fitness, represents the rule coverage, represents the average weighted error, represents the real-time performance score; ； in, represents the detection rate, represents the false alarm rate, It represents the delay coefficient. The detection rate and false alarm rate are calculated based on the ultrasonic sensor calibration results. S3, performing genetic operations, using a tournament selection operator to select excellent individuals from the population, with a tournament size of K=3, using a two-point crossover operator for gene recombination, with a crossover probability q=0.9, and avoiding weight normalization constraints at the crossover sites. Using a directed mutation operator, the conditional part genes and the action part genes are subjected to discrete value mutation and continuous value mutation within the range of ±0.2, and weight renormalization is automatically performed after the operation; S4: Implement dynamic update control, executing regular optimization iterations every 10 minutes, with a maximum iteration number of 50. When three consecutive perception errors or sensor failures of the same type are detected, an emergency optimization process is triggered. This emergency optimization process prioritizes adjusting and optimizing the relevant fuzzy rules to quickly resolve the performance degradation caused by the current perception error or sensor failure. Newly generated fuzzy rules must pass simulation verification and then be released in grayscale on 10% of vehicle nodes. A full update is then performed after confirming that the performance of the fuzzy rules has improved by ≥5% compared to the previous version and that no new failures have occurred. The performance improvement of ≥5% refers to a relative improvement in the mean average precision of target detection. S5, when initializing the population, retain the top 50 existing rules in terms of fitness. The newly generated fuzzy rules must meet the following requirements: , when driving at high speed and dark environments ; S6, when the conditions of two rules are completely matched but the weight difference exceeds 20%, they are determined to be conflicting rules, and a conflict resolution strategy is adopted for the conflicting rules, which includes: fitness priority strategy, coverage supplement strategy and comprehensive trade-off strategy; The fitness priority strategy first compares the fitness of two conflicting rules on the validation set and retains the rule with higher fitness; The coverage supplementation strategy is as follows: if the fitness difference between two conflicting rules is less than 5%, their coverage is further compared and the rule with the larger coverage is retained; The comprehensive trade-off strategy is to calculate the comprehensive scores of the two rules if the fitness difference is less than 5% and the coverage is the same, and retain the rule with the higher comprehensive score. The comprehensive score formula is: ； in, For fitness, For coverage, is the fitness weight coefficient.

Images