Multi-sensor data fusion method for unmanned pre-controller

Abstract

The invention relates to the technical field of sensing data fusion, in particular to a multi-sensor data fusion method for an unmanned pre-controller. The method comprises the following steps: acquiring environmental obstacle position data, environmental object distance data and environmental object motion state data through unmanned driving environment multiple sensors, performing time sequence synchronization and standardization processing, and performing feature space vector conversion at the same time; carrying out multi-sensor particle filtering fusion analysis and multi-sensor uncertainty inference adjustment analysis to obtain an environment multi-sensor uncertainty adjustment coefficient; and performing multi-sensor uncertainty fusion optimization and environment space modeling processing according to the environment multi-sensor uncertainty adjustment coefficient, performing environment state driving control prediction analysis, and generating an unmanned driving environment state change driving control strategy so as to execute a corresponding unmanned driving environment change pre-control function. According to the invention, rapid fusion analysis and utilization of unmanned multi-sensor data can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of sensor data fusion, and in particular to a multi-sensor data fusion method for an unmanned driving pre-controller. Background Art

[0002] In the unmanned driving pre-controller, the design and implementation of the pre-controller are crucial to ensure the safety and driving performance of the vehicle. The pre-controller needs to process data from different sensors in real time to generate accurate control instructions. Current research directions include developing more advanced data fusion technologies to improve the pre-controller's processing capabilities for multi-sensor data, by collecting raw data from different sensors and pre-processing them to ensure data quality and consistency, and using advanced fusion algorithms such as multi-level fusion networks or adaptive fusion models to comprehensively analyze the processed multi-sensor data to extract valuable information, and input the fusion results into the pre-controller to generate accurate control instructions, thereby improving the response speed and decision accuracy of the unmanned driving pre-controller. However, in the existing technology, multi-sensor data fusion methods usually focus on using a single fusion strategy to process sensor data. Although these methods can provide effective data integration in some application scenarios, their performance is often limited in the face of complex situations such as dynamic environments, sensor failures, and data inconsistencies. Summary of the invention

[0003] Based on this, it is necessary for the present invention to provide a multi-sensor data fusion method for an unmanned driving pre-controller to solve at least one of the above technical problems.

[0004] To achieve the above object, a multi-sensor data fusion method for an unmanned driving pre-controller comprises the following steps:

[0005] Step S1: Obtaining unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data through unmanned driving environment multi-sensors, and performing time series synchronization and standardization processing on the unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data to obtain environmental obstacle position standard data, environmental object distance standard data, and environmental object motion state standard data;

[0006] Step S2: performing feature space vector conversion on the standard data of the environmental obstacle position, the standard data of the environmental object distance and the standard data of the environmental object motion state to obtain the spatial feature vector of the environmental obstacle position, the spatial feature vector of the environmental object distance and the spatial feature vector of the environmental object motion state; performing multi-sensor particle filter fusion analysis on the spatial feature vector of the environmental obstacle position, the spatial feature vector of the environmental object distance and the spatial feature vector of the environmental object motion state to obtain the high-dimensional fusion feature representation data of the environmental multi-sensor;

[0007] Step S3: performing multi-sensor uncertainty inference adjustment analysis on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector to obtain the environmental multi-sensor uncertainty adjustment coefficient; performing multi-sensor uncertainty fusion optimization on the environmental multi-sensor high-dimensional fusion feature representation data according to the environmental multi-sensor uncertainty adjustment coefficient to obtain the environmental multi-sensor uncertainty optimization fusion feature data;

[0008] Step S4: Perform environmental space modeling processing on the environmental multi-sensor uncertain optimization fusion feature data to generate an unmanned driving multi-sensor environmental data space model; based on the unmanned driving multi-sensor environmental data space model, use the model predictive control method to perform environmental state driving control prediction analysis on the unmanned driving pre-controller, and generate an unmanned driving environmental state change driving control strategy to execute the corresponding unmanned driving environmental change pre-control function.

[0009] Further, step S1 includes the following steps:

[0010] Step S11: performing multi-sensor real-time monitoring of the unmanned driving environment space area through the unmanned driving environment multi-sensor to obtain the unmanned driving environment space multi-sensor monitoring data, wherein the unmanned driving environment space multi-sensor monitoring data includes environmental obstacle object detection image data monitored in real time by a high-resolution camera, environmental obstacle object spatial distribution data monitored in real time by a laser radar, and environmental obstacle object motion data monitored in real time by an inertial measurement unit;

[0011] Step S12: performing obstacle image spatial positioning mapping on the environmental obstacle object detection image data to obtain the initial position data of the unmanned driving environment space obstacle; performing spatial position error correction on the initial position data of the unmanned driving environment space obstacle based on the environmental obstacle object spatial distribution data to obtain the unmanned driving environment obstacle position data;

[0012] Step S13: measuring the distance between obstacles in the unmanned driving environment space area and the unmanned driving equipment based on the obstacle position data of the unmanned driving environment to obtain the distance data of the unmanned driving environment objects;

[0013] Step S14: performing object motion dynamic trajectory analysis on the environmental obstacle object motion data to obtain the environmental obstacle object motion dynamic trajectory; performing obstacle operation state analysis on the obstacles in the unmanned driving environment space area based on the environmental obstacle object motion dynamic trajectory to obtain the unmanned driving environment object motion state data;

[0014] Step S15: Perform time series synchronization processing on the unmanned driving environment obstacle position data, the unmanned driving environment object distance data and the unmanned driving environment object motion state data to obtain the environmental obstacle position data, the environmental object distance data and the environmental object motion state data in the same time series dimension; perform standardization processing on the environmental obstacle position data, the environmental object distance data and the environmental object motion state data in the same time series dimension to obtain the environmental obstacle position standard data, the environmental object distance standard data and the environmental object motion state standard data.

[0015] Furthermore, the step S12 of correcting the spatial position error of the initial position data of the spatial obstacles in the unmanned driving environment based on the spatial distribution data of the environmental obstacles includes the following steps:

[0016] The spatial distribution data of environmental obstacle objects and the initial position data of the spatial obstacles in the unmanned driving environment are mapped and transformed in the same spatial coordinate system to obtain the actual spatial distribution points of the environmental obstacle objects and the initial points of the spatial obstacles in the unmanned driving environment in the same spatial coordinate system;

[0017] Based on the actual distribution points of environmental obstacles in the same spatial coordinate system, the initial points of the corresponding unmanned driving environment space obstacles are detected for spatial deviation, and the initial position deviation points of the unmanned driving environment obstacles are obtained;

[0018] Based on the actual spatial distribution points of environmental obstacles in the same spatial coordinate system, the initial position deviation points of the unmanned driving environment obstacles are matched with the position deviation compensation calculation to obtain the environmental obstacle position deviation compensation value of each initial position deviation point;

[0019] According to the environmental obstacle position deviation compensation value of each initial position deviation point, the spatial position error of the corresponding unmanned driving environment obstacle initial position data in the unmanned driving environment spatial obstacle is corrected to obtain the unmanned driving environment obstacle position data.

[0020] Further, step S2 includes the following steps:

[0021] Step S21: Analyze the spatial distribution characteristics of the standard data of the environmental obstacle positions to obtain spatial distribution characteristic data of the environmental obstacle positions;

[0022] Step S22: performing distance pattern feature analysis on the environmental object distance standard data to obtain environmental object distance pattern feature data;

[0023] Step S23: analyzing the dynamic behavior characteristics of the environmental object motion state standard data to obtain the dynamic behavior characteristic data of the environmental object motion;

[0024] Step S24: performing feature space vector conversion on the environmental obstacle position spatial distribution feature data, the environmental object distance pattern feature data, and the environmental object motion dynamic behavior feature data to obtain the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector;

[0025] Step S25: Perform multi-sensor particle filter fusion analysis on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector and the environmental object motion state spatial feature vector to obtain environmental multi-sensor high-dimensional fusion feature representation data.

[0026] Further, step S25 includes the following steps:

[0027] Step S251: performing time series state division processing on the environment obstacle position spatial feature vector, the environment object distance spatial feature vector and the environment object motion state spatial feature vector to obtain the obstacle position, object distance and object motion state spatial feature vector in the initial state and the obstacle position, object distance and object motion state spatial feature vector in the dynamically changing state;

[0028] Step S252: performing state representation particle conversion on the obstacle position, object distance and object motion state space feature vector in the initial state to obtain an environment space feature vector state representation particle group;

[0029] Step S253: performing future state change prediction analysis on the obstacle position, object distance and object motion state spatial feature vector in a dynamically changing state to obtain future change prediction data of the obstacle position, object distance and object motion state;

[0030] Step S254: performing state update weighted calculation on the corresponding environment state representation particles in the environment space feature vector state representation particle group according to the obstacle position, object distance and object motion state future change prediction data, and obtaining the state change update weight of each environment state representation particle in each state update period; performing state update resampling on the corresponding obstacle position, object distance and object motion state space feature vector in a dynamically changing state based on the state change update weight of each environment state representation particle in each state update period, and obtaining the obstacle position state update vector, object distance state update vector and object motion state update vector of each environment state representation particle;

[0031] Step S255: Perform high-dimensional feature fusion mapping on the obstacle position state update vector, object distance state update vector and object motion state update vector of each environment state representation particle to obtain environment multi-sensor high-dimensional fusion feature representation data.

[0032] Further, step S3 includes the following steps:

[0033] Step S31: Perform Bayesian prior distribution modeling on the spatial feature vector of the environmental obstacle position to generate a Bayesian prior distribution mathematical model of the obstacle position feature;

[0034] Step S32: performing a posteriori uncertainty inference analysis on the environmental object distance space feature vector and the environmental object motion state space feature vector based on the obstacle position feature Bayesian prior distribution mathematical model to obtain the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data;

[0035] Step S33: performing dynamic variation measurement analysis based on the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data to obtain the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value;

[0036] Step S34: using a multi-sensor uncertainty adjustment calculation formula to perform uncertainty adjustment calculation on the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value to obtain an environmental multi-sensor uncertainty adjustment coefficient;

[0037] Step S35: performing multi-sensor uncertainty fusion optimization on the environmental multi-sensor high-dimensional fusion feature representation data according to the environmental multi-sensor uncertainty adjustment coefficient to obtain environmental multi-sensor uncertainty optimized fusion feature data.

[0038] Further, step S33 includes the following steps:

[0039] Perform uncertainty time series dynamic change analysis on the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data to obtain obstacle position-distance uncertainty time series dynamic change data and obstacle position-motion state uncertainty time series dynamic change data;

[0040] Draw a time series change curve graph for the obstacle position-distance uncertainty time series dynamic change data and the obstacle position-motion state uncertainty time series dynamic change data to generate an obstacle position-distance uncertainty time series change curve graph and an obstacle position-motion state uncertainty time series change curve graph;

[0041] Performing statistical analysis on the curve amplitude of the obstacle position-distance uncertainty time series change curve and the obstacle position-motion state uncertainty time series change curve to obtain the obstacle position-distance uncertainty time series change fluctuation amplitude and the obstacle position-motion state uncertainty time series change fluctuation amplitude;

[0042] Based on the fluctuation amplitude of the obstacle position-distance uncertainty time series change and the fluctuation amplitude of the obstacle position-motion state uncertainty time series change, a dynamic variation measurement analysis is performed on the obstacle position-distance uncertainty time series change curve and the obstacle position-motion state uncertainty time series change curve to obtain the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value.

[0043] Further, the dynamic variation measurement analysis of the obstacle position-distance uncertainty time series change curve graph and the obstacle position-motion state uncertainty time series change curve graph based on the obstacle position-distance uncertainty time series change fluctuation amplitude and the obstacle position-motion state uncertainty time series change fluctuation amplitude includes the following steps:

[0044] Based on the fluctuation amplitude of the obstacle position-distance uncertainty time series change, the obstacle position-distance uncertainty time series change curve is divided into distance time series change intervals to obtain the obstacle position-distance uncertainty change stable time series interval and the obstacle position-distance uncertainty change fluctuation time series interval;

[0045] According to the obstacle position-distance uncertainty change stable time series interval and the obstacle position-distance uncertainty change fluctuating time series interval, the obstacle position-distance uncertainty time series change curve is analyzed for distance dynamic variation measurement to obtain the obstacle position-distance dynamic uncertainty variation measurement value;

[0046] Based on the fluctuation amplitude of the obstacle position-motion state uncertainty time series change, the obstacle position-motion state uncertainty time series change curve is divided into the motion state time series change interval to obtain the obstacle position-motion state uncertainty change stable time series interval and the obstacle position-motion state uncertainty change fluctuation time series interval;

[0047] According to the stable time series interval of obstacle position-motion state uncertainty change and the fluctuating time series interval of obstacle position-motion state uncertainty change, the obstacle position-motion state uncertainty time series change curve is analyzed for dynamic variation measurement of motion state, and the dynamic uncertainty variation measurement value of obstacle position-motion state is obtained.

[0048] Furthermore, the multi-sensor uncertainty adjustment calculation formula in step S34 is specifically:

[0049]

[0050] Where U t is the environmental multi-sensor uncertainty adjustment coefficient, n is the number of sensors in the unmanned driving environment, i is the sensor item measurement parameter, T is the total duration of the observation time window, t is the integral time variable parameter, and w i is the uncertainty weight coefficient of the i-th sensor, d i (t) is the obstacle distance measurement value of the i-th sensor at time t, α i is the obstacle position-distance dynamic uncertainty variation measure of the i-th sensor, m i (t) is the measured value of the motion state of the obstacle object by the i-th sensor at time t, β i is the obstacle position-motion state dynamic uncertainty variation measure of the i-th sensor, and η is the correction parameter of the environmental multi-sensor uncertainty adjustment coefficient.

[0051] Further, step S4 includes the following steps:

[0052] Step S41: performing environment fusion feature point cloud sampling on the environment multi-sensor uncertain optimization fusion feature data to obtain an environment multi-sensor fusion feature point cloud set;

[0053] Step S42: performing environmental space modeling processing according to the environmental multi-sensor fusion feature point cloud set to generate an unmanned driving multi-sensor environmental data space model;

[0054] Step S43: using a model predictive control method based on the unmanned driving multi-sensor environment data space model to predict the environmental state change response control of the unmanned driving pre-controller, so as to generate an unmanned driving environmental state prediction change response control instruction;

[0055] Step S44: Generate an unmanned driving environment state change driving control strategy by responding to the unmanned driving environment state prediction change response control instruction to the unmanned driving pre-controller and performing environment state change driving control strategy analysis to execute the corresponding unmanned driving environment change pre-control function.

[0056] Beneficial effects of the present invention:

[0057] Compared with the prior art, the multi-sensor data fusion method for unmanned driving pre-controller proposed in the present invention has the beneficial effect of using the corresponding unmanned driving environment multi-sensor (including high-resolution camera, laser radar and inertial measurement unit) to perform multi-sensor real-time monitoring of the unmanned driving environment space area, so that the sensor equipment can fully obtain the multi-source sensor data in the unmanned driving environment. In this way, by obtaining the initial position of the obstacle in the unmanned driving environment, it not only includes the application of image processing algorithm to ensure that the spatial coordinates of each obstacle in the image can be accurately identified, but also involves the combined use of the spatial distribution data of environmental obstacle objects. By correcting the spatial position error of the initial position data, the real position of the obstacle can be more accurately determined. By measuring the distance between the obstacle in the unmanned driving environment space area and the unmanned driving device based on the obstacle position data of the unmanned driving environment, the distance between the obstacle and the unmanned driving device can be measured, thereby improving the adaptability and intelligence of the entire unmanned driving pre-controller. By analyzing the dynamic trajectory of environmental obstacle motion data, it is possible to provide an in-depth understanding of obstacle behavior. By analyzing this data, it is possible to identify the motion mode of obstacles, such as dynamic features such as speed, acceleration, and motion direction. This analysis not only helps to determine the motion state of obstacles in real time, but also provides a basis for the unmanned driving pre-controller to predict the future position of obstacles. Based on the dynamic trajectory of the obstacle's motion, it is possible to effectively analyze the operating state of obstacles to assess their potential threats to unmanned driving equipment, thereby providing important information for the intelligent decision-making of the unmanned driving pre-controller. At the same time, the unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data are processed in time series synchronization to ensure that they are analyzed and processed in the same time series dimension. This process can be achieved through timestamps and synchronization algorithms, so that all data are integrated at the same time for further analysis. In addition, by standardizing the environmental obstacle position data, environmental object distance data, and environmental object motion status data in the same time dimension, not only the comparability between different data sources is improved, but also the data deviation caused by sensor characteristics and measurement errors is reduced, the reliability and consistency of the data are increased, and it can better cope with complex and dynamic environments.Secondly, by performing feature space vector conversion on the standard data of environmental obstacle positions, standard data of environmental object distances, and standard data of environmental object motion states, this feature space vector conversion helps to integrate different types of data into a unified representation, thereby facilitating subsequent analysis and processing. The feature space vectorization process enables data to be compared and integrated in the same feature space, thereby better reflecting the overall characteristics and complexity of the environment. This process not only improves the efficiency of data processing, but also improves the accuracy of data analysis, enabling the unmanned driving pre-controller to provide more sophisticated and accurate decision support based on multi-dimensional data. Multi-sensor particle filter fusion analysis is also performed on the spatial feature vectors of the environmental obstacle position, the spatial feature vectors of the environmental object distance, and the spatial feature vectors of the environmental object motion state. Particle filtering is an advanced data fusion method for processing nonlinear and non-Gaussian systems. By fusing multi-dimensional feature vectors, more comprehensive and accurate environmental feature representation data can be obtained. Particle filter fusion can combine the advantages of various sensor data, reduce the errors and noise in a single sensor data, and thus improve the reliability and accuracy of the final feature representation. This high-dimensional fusion feature representation data provides a comprehensive perspective on the environment, enabling the unmanned driving pre-controller to perform more accurate perception and decision-making in complex dynamic environments, thereby significantly improving the overall performance and safety of the pre-controller. Then, by performing multi-sensor uncertainty inference adjustment analysis on the spatial feature vectors of the environmental obstacle position, the spatial feature vectors of the environmental object distance, and the spatial feature vectors of the environmental object motion state, the uncertainty of the data of multiple sensors can be comprehensively considered, thereby optimizing the effect of uncertainty fusion. This adjustment process can eliminate the deviation and error of a single sensor, making the sensor data more accurate in the fusion process, ensuring a comprehensive and accurate understanding of environmental information. This adjustment mechanism not only improves the quality of data fusion, but also reduces the uncertainty caused by sensor errors, thereby enhancing the stability and robustness of the overall data fusion process. In addition, by optimizing the multi-sensor high-dimensional fusion features according to the environmental multi-sensor uncertainty adjustment coefficient, the quality and accuracy of the feature data can be improved, so that the complex information in the environment can be better understood and analyzed. The optimized multi-sensor fusion data not only enhances the ability to detect and identify obstacles, but also improves the reliability of the overall decision, thereby effectively integrating environmental data from different sensors, reducing data redundancy and inconsistency, making the final fusion feature data more accurate, and supporting more efficient environmental perception and processing.Finally, by modeling the environment space of the environmental multi-sensor uncertain optimization fusion feature data, a detailed and accurate environment data space model can be generated. This model not only contains the three-dimensional information of static objects in the environment (such as roads, buildings, etc.), but also can capture dynamically changing elements (such as moving pedestrians, vehicles, etc.). This spatial model provides rich environmental information for the unmanned driving pre-controller, enabling it to more accurately understand the surrounding environment and the distribution of obstacles. Through spatial modeling, more effective path planning, collision detection and environmental perception can be carried out, thereby improving the safety and reliability of the unmanned driving pre-controller, thereby optimizing the control algorithm and decision-making strategy of the unmanned vehicle. In addition, the driving control strategy of the unmanned driving pre-controller is analyzed based on the unmanned driving multi-sensor environmental data space model using the model predictive control method. This process ensures that the unmanned vehicle can make timely and effective responses to environmental changes. By applying control instructions to the vehicle's pre-control system, it can respond to various changes in the environment (such as the appearance of obstacles, changes in road conditions, etc.) in real time. At the same time, the analysis of driving control strategies for environmental state changes can help the unmanned driving pre-controller evaluate the effects of different control strategies and optimize the control strategies, thereby improving the response speed and decision-making ability of the unmanned driving pre-controller. This not only enhances the adaptability of the unmanned driving pre-controller in complex and dynamic environments, but also significantly improves the driving safety and comfort of the vehicle, thereby achieving a higher level of autonomous driving performance in actual driving. Through this method, the unmanned driving pre-controller can effectively integrate data from different sensors and reduce errors caused by data inconsistency and sensor failures, thereby improving the overall performance and safety of the unmanned driving pre-controller. The application of this technology will significantly enhance the operation capabilities of unmanned vehicles in complex environments and can enhance the obstacle avoidance control performance of the unmanned driving pre-controller. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0059] Figure 1 It is a schematic flow chart of the steps of the multi-sensor data fusion method for unmanned driving pre-controller of the present invention;

[0060] Figure 2 for Figure 1 Detailed step flow diagram of step S1;

[0061] Figure 3 for Figure 1 Detailed step flow chart of step S2 in FIG. DETAILED DESCRIPTION

[0062] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0063] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0064] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0065] To achieve this, please refer to Figures 1 to 3 The present invention provides a multi-sensor data fusion method for an unmanned driving pre-controller, the method comprising the following steps:

[0066] Step S1: Obtaining unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data through unmanned driving environment multi-sensors, and performing time series synchronization and standardization processing on the unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data to obtain environmental obstacle position standard data, environmental object distance standard data, and environmental object motion state standard data;

[0067] Step S2: performing feature space vector conversion on the standard data of the environmental obstacle position, the standard data of the environmental object distance and the standard data of the environmental object motion state to obtain the spatial feature vector of the environmental obstacle position, the spatial feature vector of the environmental object distance and the spatial feature vector of the environmental object motion state; performing multi-sensor particle filter fusion analysis on the spatial feature vector of the environmental obstacle position, the spatial feature vector of the environmental object distance and the spatial feature vector of the environmental object motion state to obtain the high-dimensional fusion feature representation data of the environmental multi-sensor;

[0068] Step S3: performing multi-sensor uncertainty inference adjustment analysis on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector to obtain the environmental multi-sensor uncertainty adjustment coefficient; performing multi-sensor uncertainty fusion optimization on the environmental multi-sensor high-dimensional fusion feature representation data according to the environmental multi-sensor uncertainty adjustment coefficient to obtain the environmental multi-sensor uncertainty optimization fusion feature data;

[0069] Step S4: Perform environmental space modeling processing on the environmental multi-sensor uncertain optimization fusion feature data to generate an unmanned driving multi-sensor environmental data space model; based on the unmanned driving multi-sensor environmental data space model, use the model predictive control method to perform environmental state driving control prediction analysis on the unmanned driving pre-controller, and generate an unmanned driving environmental state change driving control strategy to execute the corresponding unmanned driving environmental change pre-control function.

[0070] In the embodiment of the present invention, please refer to Figure 1 FIG. 1 is a schematic diagram of the steps of the multi-sensor data fusion method for an unmanned driving pre-controller of the present invention. In this example, the multi-sensor data fusion method for an unmanned driving pre-controller includes the following steps:

[0071] Step S1: Obtaining unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data through unmanned driving environment multi-sensors, and performing time series synchronization and standardization processing on the unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data to obtain environmental obstacle position standard data, environmental object distance standard data, and environmental object motion state standard data;

[0072] In an embodiment of the present invention, a multi-sensor such as a high-resolution camera, a laser radar and an inertial measurement unit (IMU) is used to monitor the space area of ​​the unmanned driving environment in real time, wherein the high-resolution camera is used to capture the image data of the environmental obstacles, the laser radar generates the three-dimensional spatial distribution data of the environmental obstacles by emitting a laser beam and receiving the reflected light, and the IMU sensor records the motion data of the object, including acceleration and angular velocity, and uses an image recognition algorithm to perform image space positioning mapping on the environmental obstacle object detection image previously monitored to convert the obstacle mark position in the image into the initial position in the spatial coordinate system, and uses the environmental obstacle object spatial distribution data provided by the laser radar to correct the spatial position error of the initial position of the unmanned driving environment obstacle. This process includes adjusting the initial position of the obstacle through coordinate transformation and data fusion algorithm to accurately match the spatial data provided by the laser radar, thereby obtaining the corrected unmanned driving environment obstacle position data. At the same time, the distance between the unmanned driving device and the obstacle is measured and calculated by combining the previously corrected unmanned driving environment obstacle position data. This process includes the application of the distance formula and the real-time update of the distance data to ensure the accuracy and real-time nature of the distance information between the obstacle and the device, thereby obtaining the unmanned driving environment object distance data. Secondly, the motion data of environmental obstacle objects previously obtained through real-time monitoring are analyzed by using kinematic models and trajectory prediction algorithms. According to the continuous position information, a motion trajectory diagram of the obstacle on the time axis is drawn to display the motion trajectory of the obstacle, and the speed, acceleration and motion mode of the corresponding obstacle are analyzed. The running state of the obstacle is classified using a dynamic model (such as uniform linear motion, accelerated motion, etc.), and the analysis results are marked as different states (such as stationary, slow movement, fast movement), so as to obtain the motion state data of objects in the unmanned driving environment. Then, by using a time series synchronization algorithm (such as an interpolation method or a synchronous data processing tool), the previously analyzed unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion status data are synchronized in time series to adjust the data acquisition frequency differences of different sensors so that all data points are aligned on the time axis and ensure that all data are analyzed in the same time dimension. The synchronized data is standardized by using data standardization technology (such as Z-score standardization) to convert values ​​of different data types into a unified standard range. This can eliminate the impact of different data sources and measurement units on the analysis results, and ultimately obtain standard data on environmental obstacle positions, standard data on environmental object distances, and standard data on environmental object motion status.

[0073] Step S2: performing feature space vector conversion on the standard data of the environmental obstacle position, the standard data of the environmental object distance and the standard data of the environmental object motion state to obtain the spatial feature vector of the environmental obstacle position, the spatial feature vector of the environmental object distance and the spatial feature vector of the environmental object motion state; performing multi-sensor particle filter fusion analysis on the spatial feature vector of the environmental obstacle position, the spatial feature vector of the environmental object distance and the spatial feature vector of the environmental object motion state to obtain the high-dimensional fusion feature representation data of the environmental multi-sensor;

[0074] In an embodiment of the present invention, feature analysis is performed on previously standardized environmental obstacle position standard data, environmental object distance standard data, and environmental object motion state standard data by using a feature analysis method to analyze and obtain corresponding position spatial distribution characteristics (including the density of obstacles, the shape of the distribution area, and their relative positional relationship), object distance pattern characteristics (including the average distance between objects, the variance of the distance distribution, and its spatial layout characteristics), and object motion dynamic behavior characteristics (including the characteristics of the motion trajectory, the speed change pattern, and the law of motion), and feature vector conversion is performed on the feature data previously obtained after feature analysis by using a data processing tool such as MATLAB or Python's Scikit-learn library to convert multiple feature data into a unified spatial feature vector, which includes the spatial characteristics of the environmental obstacle position, the spatial characteristics of the object distance, and the spatial characteristics of the object motion state, thereby obtaining the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector. Then, a particle filter fusion analysis is performed on the spatial feature vectors of the environmental obstacle position, the spatial feature vectors of the environmental object distance, and the spatial feature vectors of the environmental object motion state obtained after the vector conversion, so that the previously obtained feature vectors are input into the particle filter algorithm. The algorithm is used to process and fuse the data of multiple sensors, and use the particle filter to estimate and update the environmental state through a series of particles (representing possible states), and apply tools such as the particle filter function in MATLAB or the FilterPy library in Python to perform filtering and fusion operations, and finally fuse the high-dimensional fusion feature representation data of the environmental multi-sensor.

[0075] Step S3: performing multi-sensor uncertainty inference adjustment analysis on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector to obtain the environmental multi-sensor uncertainty adjustment coefficient; performing multi-sensor uncertainty fusion optimization on the environmental multi-sensor high-dimensional fusion feature representation data according to the environmental multi-sensor uncertainty adjustment coefficient to obtain the environmental multi-sensor uncertainty optimization fusion feature data;

[0076] In the embodiment of the present invention, the spatial feature vector of the environmental obstacle position obtained by the previous transformation is modeled by using a Bayesian network or a Gaussian mixture model to establish a Bayesian prior distribution model of the obstacle position. This model can be expressed as a multivariate probability distribution. The parameters of the model are estimated by the corresponding environmental obstacle position data, including the mean and covariance matrix, and its mathematical form is: Wherein μ is the mean vector of the obstacle position, Σ is the covariance matrix, and the model can reflect the position distribution characteristics of the obstacle in the environment, and is combined with the previously constructed position Bayesian prior distribution mathematical model and the previously converted environmental object distance space feature vector and environmental object motion state space feature vector, and performs posterior analysis by using the Bayesian inference algorithm. For example, by calculating the posterior distribution p(x|y), where x is the obstacle position, y is the measured distance and motion state characteristics, and applying the Bayesian theorem for inference, the calculation result is the posterior uncertainty inference data of the obstacle position-distance and the obstacle position-motion state, and at the same time, the dynamic variation measurement statistical analysis is performed on the posterior uncertainty inference data of the obstacle position-distance and the position-motion state. This involves calculating the variance and standard deviation of the two types of uncertainty data and evaluating their dynamic characteristics over time. For example, using statistical analysis tools (such as MATLAB or the SciPy library in Python) to calculate the variance or covariance matrix of these data can reveal how uncertainty changes over time or environmental conditions change, so as to determine the corresponding dynamic uncertainty variation measure to fuse the data of different sensors and calculate the comprehensive uncertainty coefficient. First, the dynamic uncertainty variation measure of obstacle position-distance and position-motion state is input into the adjustment calculation formula, which takes into account the measurement accuracy and data correlation of different sensors to adjust the comprehensive uncertainty, so as to calculate the corresponding environmental multi-sensor uncertainty adjustment coefficient. Then, the high-dimensional fusion feature representation data is optimized by combining the environmental multi-sensor uncertainty adjustment coefficient obtained by the previous quantitative calculation to collect high-dimensional feature data obtained from different sensors after initial fusion analysis, and these fused feature data are optimized by applying the multi-sensor uncertainty adjustment coefficient calculated previously. The fusion optimization process involves applying the uncertainty adjustment coefficient to the fusion algorithm to improve the accuracy and reliability of the fused data, and finally obtains the environmental multi-sensor uncertainty optimized fusion feature data.

[0077] Step S4: Perform environmental space modeling processing on the environmental multi-sensor uncertain optimization fusion feature data to generate an unmanned driving multi-sensor environmental data space model; based on the unmanned driving multi-sensor environmental data space model, use the model predictive control method to perform environmental state driving control prediction analysis on the unmanned driving pre-controller, and generate an unmanned driving environmental state change driving control strategy to execute the corresponding unmanned driving environmental change pre-control function.

[0078] In an embodiment of the present invention, the fused feature point cloud data is sampled by sampling the fused feature point cloud data of the environmental multi-sensor uncertain optimization fusion feature data previously obtained after fusion by a particle filter algorithm to select representative feature points, and the environmental space modeling is performed by using the environmental multi-sensor fusion feature point cloud set obtained by previous analysis, so as to divide the point cloud data into different environmental areas by applying point cloud segmentation technology, and fit the point cloud by geometric modeling algorithms, such as RANSAC (random sampling consensus algorithm) and least squares method, so as to construct and generate an unmanned driving multi-sensor environmental data space model. Then, by combining the previously constructed unmanned driving multi-sensor environmental data space model, the model predictive control (MPC) method is used to control and predict the response of the unmanned driving pre-controller to the environmental state change, so as to establish a dynamic model to describe the change of the environmental state, such as using a linear or nonlinear state space model, by inputting the current environmental state data into the prediction model, calculating the change of the future state of the environment, and these calculation results will respond to the unmanned driving pre-controller and generate corresponding environmental state prediction change control instructions, and by combining the previously generated environmental state prediction change control instructions to respond to the unmanned driving pre-controller, so as to analyze the driving control strategy of the unmanned driving pre-controller to the environmental state change according to the prediction results. Specifically, the driving strategy is formulated by combining the PID controller or LQR (linear quadratic regulator) algorithm in the control theory. The driving control strategy will adjust the operation of the unmanned driving pre-controller according to the change of the environmental state to achieve a smooth and safe response to environmental changes. This process involves real-time calculation and adjustment of control instructions to ensure that the unmanned driving pre-controller can maintain stable driving performance in a dynamic environment, and finally generate an unmanned driving environmental state change driving control strategy to execute the corresponding environmental change pre-control function, thereby ensuring that the unmanned driving pre-controller can effectively respond to changes in complex environments.

[0079] Further, step S1 includes the following steps:

[0080] Step S11: performing multi-sensor real-time monitoring of the unmanned driving environment space area through the unmanned driving environment multi-sensor to obtain the unmanned driving environment space multi-sensor monitoring data, wherein the unmanned driving environment space multi-sensor monitoring data includes environmental obstacle object detection image data monitored in real time by a high-resolution camera, environmental obstacle object spatial distribution data monitored in real time by a laser radar, and environmental obstacle object motion data monitored in real time by an inertial measurement unit;

[0081] Step S12: performing obstacle image spatial positioning mapping on the environmental obstacle object detection image data to obtain the initial position data of the unmanned driving environment space obstacle; performing spatial position error correction on the initial position data of the unmanned driving environment space obstacle based on the environmental obstacle object spatial distribution data to obtain the unmanned driving environment obstacle position data;

[0082] Step S13: measuring the distance between obstacles in the unmanned driving environment space area and the unmanned driving equipment based on the obstacle position data of the unmanned driving environment to obtain the distance data of the unmanned driving environment objects;

[0083] Step S14: performing object motion dynamic trajectory analysis on the environmental obstacle object motion data to obtain the environmental obstacle object motion dynamic trajectory; performing obstacle operation state analysis on the obstacles in the unmanned driving environment space area based on the environmental obstacle object motion dynamic trajectory to obtain the unmanned driving environment object motion state data;

[0084] Step S15: Perform time series synchronization processing on the unmanned driving environment obstacle position data, the unmanned driving environment object distance data and the unmanned driving environment object motion state data to obtain the environmental obstacle position data, the environmental object distance data and the environmental object motion state data in the same time series dimension; perform standardization processing on the environmental obstacle position data, the environmental object distance data and the environmental object motion state data in the same time series dimension to obtain the environmental obstacle position standard data, the environmental object distance standard data and the environmental object motion state standard data.

[0085] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Detailed step flow diagram of step S1 in FIG. 1 , in this embodiment, step S1 includes the following steps:

[0086] Step S11: performing multi-sensor real-time monitoring of the unmanned driving environment space area through the unmanned driving environment multi-sensor to obtain the unmanned driving environment space multi-sensor monitoring data, wherein the unmanned driving environment space multi-sensor monitoring data includes environmental obstacle object detection image data monitored in real time by a high-resolution camera, environmental obstacle object spatial distribution data monitored in real time by a laser radar, and environmental obstacle object motion data monitored in real time by an inertial measurement unit;

[0087] In an embodiment of the present invention, multiple sensors such as high-resolution cameras, lidars, and inertial measurement units (IMUs) are used to monitor the spatial area of ​​an unmanned driving environment in real time. The high-resolution camera is used to capture image data of environmental obstacles, and the outline and position of the obstacle object are detected through an image processing algorithm. The lidar generates three-dimensional spatial distribution data of the environmental obstacles by emitting a laser beam and receiving reflected light, and the IMU sensor records the motion data of the object, including acceleration and angular velocity. These three types of data are combined to form a comprehensive environmental monitoring data set, and finally the multi-sensor monitoring data of the unmanned driving environment space is obtained.

[0088] Step S12: performing obstacle image spatial positioning mapping on the environmental obstacle object detection image data to obtain the initial position data of the unmanned driving environment space obstacle; performing spatial position error correction on the initial position data of the unmanned driving environment space obstacle based on the environmental obstacle object spatial distribution data to obtain the unmanned driving environment obstacle position data;

[0089] In the embodiment of the present invention, the image recognition algorithm is used to perform image space positioning mapping on the environmental obstacle detection image data previously monitored, so as to convert the obstacle mark position in the image into the initial position in the spatial coordinate system, thereby obtaining the initial position data of the unmanned driving environment space obstacle. At the same time, the spatial position error of the unmanned driving environment space obstacle initial position data is corrected by using the environmental obstacle object spatial distribution data provided by the laser radar. This process includes adjusting the initial position of the obstacle through coordinate transformation and data fusion algorithm to accurately match the spatial data provided by the laser radar, and finally obtaining the corrected unmanned driving environment obstacle position data.

[0090] Step S13: measuring the distance between obstacles in the unmanned driving environment space area and the unmanned driving equipment based on the obstacle position data of the unmanned driving environment to obtain the distance data of the unmanned driving environment objects;

[0091] In an embodiment of the present invention, the distance between the unmanned driving device and the obstacle is measured and calculated by combining the previously corrected unmanned driving environment obstacle position data, and the obstacle position and device position data obtained by the sensor are used to obtain the distance between the obstacle and the unmanned driving device using a geometric calculation method (such as Euclidean distance calculation). This process includes the application of the distance formula and the real-time update of the distance data to ensure the accuracy and real-time nature of the distance information between the obstacle and the device, and ultimately the distance data of the unmanned driving environment objects is obtained.

[0092] Step S14: performing object motion dynamic trajectory analysis on the environmental obstacle object motion data to obtain the environmental obstacle object motion dynamic trajectory; performing obstacle operation state analysis on the obstacles in the unmanned driving environment space area based on the environmental obstacle object motion dynamic trajectory to obtain the unmanned driving environment object motion state data;

[0093] In the embodiment of the present invention, the motion trajectory of the environmental obstacle object motion data previously obtained through real-time monitoring is analyzed by using a kinematic model and a trajectory prediction algorithm, so as to draw a motion trajectory diagram of the obstacle on the time axis according to the continuous position information to display the motion trajectory of the obstacle, thereby obtaining the dynamic trajectory of the environmental obstacle object. The speed, acceleration and motion mode of the corresponding obstacle are analyzed by combining the dynamic trajectory of the environmental obstacle object motion obtained by the previous analysis, and the running state of the obstacle is classified by using a dynamic model (such as uniform linear motion, accelerated motion, etc.), and the analysis results are marked as different states (such as stationary, slow movement, fast movement), and finally the motion state data of the unmanned driving environment object is obtained.

[0094] Step S15: Perform time series synchronization processing on the unmanned driving environment obstacle position data, the unmanned driving environment object distance data and the unmanned driving environment object motion state data to obtain the environmental obstacle position data, the environmental object distance data and the environmental object motion state data in the same time series dimension; perform standardization processing on the environmental obstacle position data, the environmental object distance data and the environmental object motion state data in the same time series dimension to obtain the environmental obstacle position standard data, the environmental object distance standard data and the environmental object motion state standard data.

[0095] In an embodiment of the present invention, the previously analyzed unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data are synchronously processed by using a time series synchronization algorithm (such as an interpolation method or a synchronous data processing tool) to adjust the data acquisition frequency differences of different sensors so that all data points are aligned on the time axis, and all data are analyzed in the same time dimension, thereby obtaining environmental obstacle position data, environmental object distance data, and environmental object motion state data in the same time series dimension. At the same time, the synchronized data is standardized by using a data standardization technology (such as Z-score standardization) to convert values ​​of different data types into a unified standard range, so that the influence of different data sources and measurement units on the analysis results can be eliminated, and finally the environmental obstacle position standard data, environmental object distance standard data, and environmental object motion state standard data are obtained.

[0096] Furthermore, the step S12 of correcting the spatial position error of the initial position data of the spatial obstacles in the unmanned driving environment based on the spatial distribution data of the environmental obstacles includes the following steps:

[0097] The spatial distribution data of environmental obstacle objects and the initial position data of the spatial obstacles in the unmanned driving environment are mapped and transformed in the same spatial coordinate system to obtain the actual spatial distribution points of the environmental obstacle objects and the initial points of the spatial obstacles in the unmanned driving environment in the same spatial coordinate system;

[0098] In an embodiment of the present invention, by acquiring spatial distribution data of environmental obstacle objects, which data can be obtained by a laser radar (LiDAR), a camera or other sensors, in addition, the initial position data of obstacles in the unmanned driving environment are collected, which data come from the vehicle's sensors or preset map data, and a unified coordinate system is determined as a standard. This coordinate system can be a world coordinate system or a vehicle coordinate system. When selecting the coordinate system, it should be ensured that it can cover the entire environmental area and is suitable for the accuracy requirements of processing data. At the same time, by using a coordinate transformation algorithm (such as homogeneous coordinate transformation), both the environmental obstacle data and the unmanned driving obstacle initial position data are mapped to the selected unified coordinate system. This process can be achieved through matrix transformation, including the calculation of rotation matrices and displacement vectors to integrate the converted data into the same coordinate system. At this time, the actual spatial distribution points of environmental obstacle objects and the initial points of spatial obstacles in the unmanned driving environment are both in the unified coordinate system, and further analysis and processing can be performed to finally obtain the actual spatial distribution points of environmental obstacle objects and the initial points of spatial obstacles in the unmanned driving environment in the same spatial coordinate system.

[0099] Preferably, based on the actual distribution points of the environmental obstacle objects in the same spatial coordinate system, the initial points of the corresponding unmanned driving environment space obstacles are detected for spatial deviation, so as to obtain the initial position deviation points of the unmanned driving environment obstacles;

[0100] In the embodiment of the present invention, by comparing the actual distribution points of environmental obstacles in the same spatial coordinate system with the initial position data in the unmanned driving environment point by point, a distance calculation method can be used to measure the deviation between the actual obstacle position and the initial position, for example The previously quantified calculated deviation is compared and judged by using a preset deviation threshold. If the deviation is greater than or equal to the preset deviation threshold, the corresponding initial point position of the unmanned driving environment spatial obstacle is determined as the deviation point position. Otherwise, no processing is performed. Finally, the initial position deviation point position of the unmanned driving environment obstacle is detected.

[0101] Preferably, the initial position deviation points of the unmanned driving environment obstacles are matched with the position deviation compensation calculation based on the actual distribution points of the environment obstacle objects in the same spatial coordinate system, so as to obtain the environmental obstacle position deviation compensation value of each initial position deviation point;

[0102] In an embodiment of the present invention, each deviation point is matched with the actual obstacle position to calculate the distribution of the actual obstacle position in the error area, and the actual obstacle point most relevant to the deviation point is selected. The deviation is compensated using interpolation methods or regression analysis techniques. The calculation of the compensation value can be based on the nearest neighbor algorithm, weighted average method or other statistical methods, where the compensation formula can be compensation value = actual position - initial position, and finally the environmental obstacle position deviation compensation value of each initial position deviation point is obtained.

[0103] Preferably, according to the environmental obstacle position deviation compensation value of each initial position deviation point, the spatial position error correction is performed on the corresponding unmanned driving environment obstacle initial position data in the unmanned driving environment spatial obstacle to obtain the unmanned driving environment obstacle position data.

[0104] In an embodiment of the present invention, a position error correction calculation is performed on the corresponding unmanned driving environment obstacle initial position deviation point in the unmanned driving environment spatial obstacle initial position data by combining the environmental obstacle position deviation compensation value of each initial position deviation point obtained by previous quantitative calculation, so as to correct the initial position of each obstacle with deviation in the unmanned driving environment by using the corrected position = initial position + compensation value, and update the corrected position data to the corresponding obstacle position in the unmanned driving environment, so as to finally obtain the unmanned driving environment obstacle position data.

[0105] Further, step S2 includes the following steps:

[0106] Step S21: Analyze the spatial distribution characteristics of the standard data of the environmental obstacle positions to obtain spatial distribution characteristic data of the environmental obstacle positions;

[0107] Step S22: performing distance pattern feature analysis on the environmental object distance standard data to obtain environmental object distance pattern feature data;

[0108] Step S23: analyzing the dynamic behavior characteristics of the environmental object motion state standard data to obtain the dynamic behavior characteristic data of the environmental object motion;

[0109] Step S24: performing feature space vector conversion on the environmental obstacle position spatial distribution feature data, the environmental object distance pattern feature data, and the environmental object motion dynamic behavior feature data to obtain the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector;

[0110] Step S25: Perform multi-sensor particle filter fusion analysis on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector and the environmental object motion state spatial feature vector to obtain environmental multi-sensor high-dimensional fusion feature representation data.

[0111] As an embodiment of the present invention, refer to Figure 3 As shown, Figure 1 Detailed step flow diagram of step S2 in FIG. 1 , in this embodiment, step S2 includes the following steps:

[0112] Step S21: Analyze the spatial distribution characteristics of the standard data of the environmental obstacle positions to obtain spatial distribution characteristic data of the environmental obstacle positions;

[0113] In an embodiment of the present invention, a spatial analysis algorithm, such as a K-means clustering algorithm or DBSCAN, is used to perform a spatial distribution clustering analysis on the standard data of the environmental obstacle positions obtained previously through standardization. These algorithms divide the positions of obstacles in the environment into different areas or clusters according to the spatial distribution characteristics, thereby identifying the distribution pattern of the obstacles, and finally obtaining the spatial distribution characteristic data of the environmental obstacle positions by calculating the centroid of each cluster and the distribution of the points within the cluster. These data include the density of the obstacles, the shape of the distribution area, and their relative positional relationship.

[0114] Step S22: performing distance pattern feature analysis on the environmental object distance standard data to obtain environmental object distance pattern feature data;

[0115] In an embodiment of the present invention, by applying a distance pattern analysis algorithm, such as distance matrix calculation or correlation analysis, the distance pattern recognition and analysis is performed on the environmental object distance standard data previously obtained through standardization to identify and analyze the distance relationship between obstacle objects in the unmanned driving environment, and using statistical analysis methods, such as mean, variance and correlation coefficient, to calculate the average distance and distance change pattern between objects. In addition, the shortest path algorithm in graph theory can be used to analyze the distance structure between objects, and ultimately obtain environmental object distance pattern feature data, including the average distance between objects, the variance of the distance distribution and its spatial layout characteristics.

[0116] Step S23: analyzing the dynamic behavior characteristics of the environmental object motion state standard data to obtain the dynamic behavior characteristic data of the environmental object motion;

[0117] In an embodiment of the present invention, by using motion analysis methods, such as state space models (State Space Models) or Kalman filtering (Kalman Filtering), the motion behavior characteristics of the standard data of the motion state of environmental objects obtained after previous standardization are identified and analyzed to identify and analyze the motion dynamic behavior characteristics of obstacle objects in the unmanned driving environment. These methods are used to estimate the changing laws of the position, velocity and acceleration of the object, and analyze the motion pattern and dynamic behavior of the object through time series analysis or dynamic time warping (Dynamic Time Warping, DTW) algorithm, and finally obtain the motion dynamic behavior characteristic data of the environmental object, including the characteristics of the motion trajectory, the speed change pattern and the motion law.

[0118] Step S24: performing feature space vector conversion on the environmental obstacle position spatial distribution feature data, the environmental object distance pattern feature data, and the environmental object motion dynamic behavior feature data to obtain the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector;

[0119] In an embodiment of the present invention, a data processing tool such as MATLAB or Python's Scikit-learn library is used to convert the spatial distribution feature data of the environmental obstacle position, the environmental object distance pattern feature data, and the environmental object motion dynamic behavior feature data previously obtained after feature analysis into feature vectors, so as to convert multiple feature data into a unified spatial feature vector. These feature vectors include the spatial features of the environmental obstacle position, the spatial features of the object distance, and the spatial features of the object motion state, thereby providing a comprehensive description of the environment and the object state, and finally obtaining the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector.

[0120] Step S25: Perform multi-sensor particle filter fusion analysis on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector and the environmental object motion state spatial feature vector to obtain environmental multi-sensor high-dimensional fusion feature representation data.

[0121] In an embodiment of the present invention, a particle filtering fusion analysis is performed on the environmental obstacle position space feature vector, the environmental object distance space feature vector, and the environmental object motion state space feature vector obtained after vector conversion, so that the feature vector obtained in step S24 is input into the particle filtering algorithm. The algorithm is used to process and fuse data from multiple sensors, and uses a particle filter to estimate and update the environmental state through a series of particles (representing possible states), and applies tools such as the particle filter function in MATLAB or the FilterPy library in Python to perform filtering and fusion operations, and finally fuses the high-dimensional fusion feature representation data of the environmental multi-sensor.

[0122] Further, step S25 includes the following steps:

[0123] Step S251: performing time series state division processing on the environment obstacle position spatial feature vector, the environment object distance spatial feature vector and the environment object motion state spatial feature vector to obtain the obstacle position, object distance and object motion state spatial feature vector in the initial state and the obstacle position, object distance and object motion state spatial feature vector in the dynamically changing state;

[0124] In an embodiment of the present invention, a time series data processing technique (such as a sliding window method or time series segmentation) is used to perform time series state division processing on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector obtained after the previous conversion, so as to divide these feature vectors into state data in an initial state and state data in a dynamically changing state. For example, a time series analysis is performed on each feature vector by a sliding window method to divide the state characteristics of the initial state and subsequent time points. This division makes it possible to analyze the environmental characteristics in the initial and dynamically changing states respectively, and finally obtain the obstacle position, object distance and object motion state spatial feature vectors in the initial state and the obstacle position, object distance and object motion state spatial feature vectors in the dynamically changing state.

[0125] Step S252: performing state representation particle conversion on the obstacle position, object distance and object motion state space feature vector in the initial state to obtain an environment space feature vector state representation particle group;

[0126] In an embodiment of the present invention, the obstacle position, object distance and object motion state feature vectors in the initial state obtained by the division in step S251 are converted into particles. In the specific operation, the particle filtering technology is used to map each feature vector into a particle state, and each particle represents a possibility of the environmental state. By inputting the initial state feature vector into the particle filter, a group of particle groups of environmental space feature vectors are generated. These particle groups reflect the possible distribution of environmental characteristics in the initial state, and finally the environmental space feature vector state representation particle group is obtained by conversion.

[0127] Step S253: performing future state change prediction analysis on the obstacle position, object distance and object motion state spatial feature vector in a dynamically changing state to obtain future change prediction data of the obstacle position, object distance and object motion state;

[0128] In an embodiment of the present invention, a future state prediction model (such as a long short-term memory network LSTM or a Kalman filter) is used to predict the state change of the obstacle position, object distance and object motion state spatial feature vector in a dynamically changing state that was previously divided, so that the collected dynamic state data is input into the prediction model to predict the environmental state changes in a period of time in the future. For example, by using an LSTM network to train and predict time series data, the predicted data of the future obstacle position, object distance and object motion state are obtained. These predicted data provide an estimate of future environmental changes, and ultimately the predicted data of future changes in the obstacle position, object distance and object motion state are obtained.

[0129] Step S254: performing state update weighted calculation on the corresponding environment state representation particles in the environment space feature vector state representation particle group according to the obstacle position, object distance and object motion state future change prediction data, and obtaining the state change update weight of each environment state representation particle in each state update period; performing state update resampling on the corresponding obstacle position, object distance and object motion state space feature vector in a dynamically changing state based on the state change update weight of each environment state representation particle in each state update period, and obtaining the obstacle position state update vector, object distance state update vector and object motion state update vector of each environment state representation particle;

[0130] In an embodiment of the present invention, by combining the obstacle position, object distance and object motion state future change prediction data obtained by previous prediction analysis, the corresponding environment state representation particles in the environment space feature vector state representation particle group are weighted updated to calculate the weight of each particle in each update period according to the predicted future state change, and adjust the particle weight through weighted calculation so that the particle weight that is more in line with the future state is higher. Then, by combining the state change update weight of each environment state representation particle in each state update period obtained by previous calculation, these particles are resampled to obtain a new particle state vector, representing the updated obstacle position, object distance and object motion state. This resampling method usually uses a systematic resampling or a roulette resampling algorithm to ensure that the particle group can accurately reflect the prediction result, and finally obtains the obstacle position state update vector, object distance state update vector and object motion state update vector of each environment state representation particle.

[0131] Step S255: Perform high-dimensional feature fusion mapping on the obstacle position state update vector, object distance state update vector and object motion state update vector of each environment state representation particle to obtain environment multi-sensor high-dimensional fusion feature representation data.

[0132] In an embodiment of the present invention, a high-dimensional feature fusion technology (such as principal component analysis PCA or a feature fusion network in deep learning) is used to perform high-dimensional feature fusion mapping on the obstacle position state update vector, object distance state update vector, and object motion state update vector of each environmental state representation particle previously resampled, so as to map these vectors into comprehensive high-dimensional feature representation data. For example, the updated vector is input into a PCA model to perform dimensionality reduction and feature fusion to obtain fused high-dimensional feature representation data. These data integrate information from different sensors, thereby providing a comprehensive description of the environmental state, and are finally fused to obtain environmental multi-sensor high-dimensional fusion feature representation data.

[0133] Further, step S3 includes the following steps:

[0134] Step S31: Perform Bayesian prior distribution modeling on the spatial feature vector of the environmental obstacle position to generate a Bayesian prior distribution mathematical model of the obstacle position feature;

[0135] In the embodiment of the present invention, the spatial feature vector of the environmental obstacle position obtained by the previous transformation is modeled by using a Bayesian network or a Gaussian mixture model to establish a Bayesian prior distribution model of the obstacle position. This model can be expressed as a multivariate probability distribution. The parameters of the model are estimated by the corresponding environmental obstacle position data, including the mean and covariance matrix, and its mathematical form is: Where μ is the mean vector of the obstacle position, Σ is the covariance matrix. The model can reflect the position distribution characteristics of obstacles in the environment and provide a preliminary understanding of position uncertainty. Finally, a mathematical model of the Bayesian prior distribution of obstacle position characteristics is constructed.

[0136] Step S32: performing a posteriori uncertainty inference analysis on the environmental object distance space feature vector and the environmental object motion state space feature vector based on the obstacle position feature Bayesian prior distribution mathematical model to obtain the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data;

[0137] In an embodiment of the present invention, a previously constructed Bayesian prior distribution mathematical model of obstacle position characteristics is combined with the previously converted environmental object distance space feature vector and environmental object motion state space feature vector, and a posterior analysis is performed by utilizing a Bayesian inference algorithm. For example, by calculating the posterior distribution p(x|y), where x is the obstacle position and y is the measured distance and motion state characteristics, the Bayesian theorem is applied for inference, and the calculation result is the posterior uncertainty inference data of the obstacle position-distance and the obstacle position-motion state. This can be achieved by calculating the joint probability distribution and the posterior probability density function, thereby obtaining a specific uncertainty quantification result.

[0138] Step S33: performing dynamic variation measurement analysis based on the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data to obtain the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value;

[0139] In an embodiment of the present invention, a dynamic variation metric is statistically analyzed by performing a statistical analysis on the posterior uncertainty inference data of the obstacle position-distance and position-motion state, which involves calculating the variance and standard deviation of the two uncertainty data and evaluating their dynamic characteristics over time. For example, a statistical analysis tool (such as the SciPy library in MATLAB or Python) is used to calculate the variance or covariance matrix of these data. This can reveal how uncertainty changes over time or environmental conditions, thereby providing a basis for calculating the dynamic uncertainty variation metric value, and ultimately obtaining the obstacle position-distance dynamic uncertainty variation metric value and the obstacle position-motion state dynamic uncertainty variation metric value. These variation metric values ​​are used to further optimize the uncertainty management strategy.

[0140] Step S34: using a multi-sensor uncertainty adjustment calculation formula to perform uncertainty adjustment calculation on the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value to obtain an environmental multi-sensor uncertainty adjustment coefficient;

[0141] In an embodiment of the present invention, a suitable multi-sensor uncertainty adjustment calculation formula is formed by combining the quantity measurement parameters of sensors in an unmanned driving environment, the total duration of an observation time window, an integral time variable parameter, an uncertainty weight coefficient, an obstacle distance measurement value, an obstacle position-distance dynamic uncertainty variation measurement value, an obstacle motion state measurement value, an obstacle position-motion state dynamic uncertainty variation measurement value and related parameters to perform quantitative calculation of the uncertainty adjustment coefficient, so as to fuse data from different sensors and calculate a comprehensive uncertainty coefficient. First, the dynamic uncertainty variation measurement values ​​of the obstacle position-distance and position-motion state are input into the adjustment calculation formula. The adjustment calculation formula takes into account the measurement accuracy and data correlation of different sensors to adjust the comprehensive uncertainty, thereby calculating the corresponding uncertainty adjustment coefficient. This coefficient is used to further optimize the fusion of multi-sensor data, and finally obtain the environmental multi-sensor uncertainty adjustment coefficient.

[0142] Step S35: performing multi-sensor uncertainty fusion optimization on the environmental multi-sensor high-dimensional fusion feature representation data according to the environmental multi-sensor uncertainty adjustment coefficient to obtain environmental multi-sensor uncertainty optimized fusion feature data.

[0143] In an embodiment of the present invention, high-dimensional fusion feature representation data is optimized by combining the environmental multi-sensor uncertainty adjustment coefficient obtained by previous quantitative calculation to collect high-dimensional feature data obtained from different sensors after initial fusion analysis, and these fused feature data are optimized by applying the previously calculated multi-sensor uncertainty adjustment coefficient. The fusion optimization process involves applying the uncertainty adjustment coefficient to the fusion algorithm to improve the accuracy and reliability of the fused data, and finally obtaining environmental multi-sensor uncertainty optimized fusion feature data.

[0144] Further, step S33 includes the following steps:

[0145] Perform uncertainty time series dynamic change analysis on the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data to obtain obstacle position-distance uncertainty time series dynamic change data and obstacle position-motion state uncertainty time series dynamic change data;

[0146] In an embodiment of the present invention, a time series change method is used to perform a statistical analysis of the time series changes of the previously inferred obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data to obtain the Bayesian posterior distribution of the initial position, distance and motion state of the obstacle. These data are synthesized into a time series through data fusion technology, and the dynamic changes are smoothed using a Kalman filter. The obtained time series data is input into an uncertainty analysis model, such as a dynamic system analysis method based on Monte Carlo simulation, and the Bayesian posterior uncertainty distribution at each time point is plotted to complete the time series dynamic change analysis, and finally the obstacle position-distance uncertainty time series dynamic change data and the obstacle position-motion state uncertainty time series dynamic change data are obtained.

[0147] Preferably, a time series change curve graph is drawn for the obstacle position-distance uncertainty time series dynamic change data and the obstacle position-motion state uncertainty time series dynamic change data to generate an obstacle position-distance uncertainty time series change curve graph and an obstacle position-motion state uncertainty time series change curve graph;

[0148] In an embodiment of the present invention, a data visualization tool (such as Matplotlib or Excel) is used to draw a change curve graph of the obstacle position-distance uncertainty time series dynamic change data and the obstacle position-motion state uncertainty time series dynamic change data obtained after the previous time series analysis, so as to draw a corresponding time series change curve graph. Through these tools, the time axis is set as the x-axis and the uncertainty value is set as the y-axis, so as to intuitively display the dynamic uncertainty situation at different time points, and finally draw and generate the obstacle position-distance uncertainty time series change curve graph and the obstacle position-motion state uncertainty time series change curve graph.

[0149] Preferably, the obstacle position-distance uncertainty time series change curve diagram and the obstacle position-motion state uncertainty time series change curve diagram are statistically analyzed to obtain the obstacle position-distance uncertainty time series change fluctuation amplitude and the obstacle position-motion state uncertainty time series change fluctuation amplitude;

[0150] In an embodiment of the present invention, a statistical analysis of the variation amplitude of the previously generated obstacle position-distance uncertainty time series variation curve diagram and the obstacle position-motion state uncertainty time series variation curve diagram is performed by using corresponding statistical analysis software (such as MATLAB) to extract the curve amplitude at each time point, and calculate the standard deviation or root mean square value of the amplitude to quantify the fluctuation of the time series variation, thereby evaluating the degree of fluctuation of the analysis uncertainty, and finally obtaining the obstacle position-distance uncertainty time series variation fluctuation amplitude and the obstacle position-motion state uncertainty time series variation fluctuation amplitude.

[0151] Preferably, a dynamic variation measurement analysis is performed on the obstacle position-distance uncertainty time-series change curve graph and the obstacle position-motion state uncertainty time-series change curve graph based on the obstacle position-distance uncertainty time-series change fluctuation amplitude and the obstacle position-motion state uncertainty time-series change fluctuation amplitude to obtain the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value.

[0152] In an embodiment of the present invention, the corresponding obstacle position-distance uncertainty time series change curve diagram is divided into change intervals by combining the obstacle position-distance uncertainty time series change fluctuation amplitude obtained by previous analysis, so as to identify the fluctuation period in the frequency domain, and combine wavelet analysis and other technologies to distinguish the stable time series interval and the fluctuating time series interval in the curve diagram. In specific operations, a threshold can be set to distinguish the stable interval from the fluctuating interval. For example, if the fluctuation amplitude is less than the set threshold, the interval is considered to be stable; if it is greater than or equal to the time period range corresponding to the threshold, it is a fluctuating interval, and the mean and standard deviation of the data in each time series interval are calculated by combining the stable time series interval and the fluctuating time series interval obtained by previous division, and the ratio of the data mean in all the fluctuating time series intervals of the obstacle position-distance uncertainty time series change curve diagram to the data mean in the stable time series interval plus the ratio of the corresponding standard deviation is used as the corresponding dynamic uncertainty variation measure value, thereby obtaining the obstacle position-distance dynamic uncertainty variation measure value. Similarly, by performing a dynamic variation measurement analysis on the corresponding obstacle position-motion state uncertainty time series change curve according to the same processing process as above, the dynamic variation fluctuation of uncertainty is evaluated by fully considering the fluctuation trend and amplitude change of uncertainty, and finally the obstacle position-motion state dynamic uncertainty variation measurement value is obtained.

[0153] Further, the dynamic variation measurement analysis of the obstacle position-distance uncertainty time series change curve graph and the obstacle position-motion state uncertainty time series change curve graph based on the obstacle position-distance uncertainty time series change fluctuation amplitude and the obstacle position-motion state uncertainty time series change fluctuation amplitude includes the following steps:

[0154] Based on the fluctuation amplitude of the obstacle position-distance uncertainty time series change, the obstacle position-distance uncertainty time series change curve is divided into distance time series change intervals to obtain the obstacle position-distance uncertainty change stable time series interval and the obstacle position-distance uncertainty change fluctuation time series interval;

[0155] In an embodiment of the present invention, the corresponding obstacle position-distance uncertainty time series change curve diagram is divided into change intervals by combining the obstacle position-distance uncertainty time series change fluctuation amplitude obtained by previous analysis, so as to identify the fluctuation period from the frequency domain, and combine with wavelet analysis and other technologies to distinguish the stable time series interval and the fluctuating time series interval in the curve diagram. In specific operations, a threshold can be set to distinguish the stable interval from the fluctuating interval. For example, if the fluctuation amplitude is less than the set threshold, the interval is considered to be stable; if it is greater than or equal to the time period range corresponding to the threshold, it is a fluctuating interval. This analysis result will determine the change pattern of the obstacle position-distance uncertainty in each time series interval, and finally divide the obstacle position-distance uncertainty change stable time series interval and the obstacle position-distance uncertainty change fluctuating time series interval to obtain.

[0156] Preferably, a distance dynamic variation measurement analysis is performed on the obstacle position-distance uncertainty time series variation curve diagram according to the obstacle position-distance uncertainty variation stable time series interval and the obstacle position-distance uncertainty variation fluctuation time series interval to obtain the obstacle position-distance dynamic uncertainty variation measurement value;

[0157] In an embodiment of the present invention, by combining the previously divided stable time series intervals of obstacle position-distance uncertainty change and the fluctuating time series intervals of obstacle position-distance uncertainty change, the mean and standard deviation of the data in each time series interval are calculated respectively, and by using the ratio of the mean of the data in all fluctuating time series intervals of the obstacle position-distance uncertainty time series change curve to the mean of the data in the stable time series interval plus the ratio of the corresponding standard deviation as the dynamic uncertainty variation measure value, if the coefficient of variation is large, it indicates that the dynamic uncertainty of the corresponding time series change curve is high, otherwise it is low. This measure value helps to evaluate the stability of the change of obstacle position-distance uncertainty in the time series change curve, and finally obtains the obstacle position-distance dynamic uncertainty variation measure value.

[0158] Preferably, the obstacle position-motion state uncertainty time series change curve diagram is divided into motion state time series change intervals based on the obstacle position-motion state uncertainty time series change fluctuation amplitude, so as to obtain the obstacle position-motion state uncertainty change stable time series interval and the obstacle position-motion state uncertainty change fluctuation time series interval;

[0159] In an embodiment of the present invention, the corresponding obstacle position-motion state uncertainty time series change curve diagram is divided into change intervals by combining the obstacle position-motion state uncertainty time series change fluctuation amplitude obtained by previous analysis, so as to distinguish the stable time series interval and the fluctuating time series interval in the curve diagram. Specifically, the motion state data is segmented, and the standard deviation and mean of each segment of data are calculated to identify the fluctuation pattern. If the standard deviation is small within a certain interval, the interval is considered to be a stable motion state interval; if the standard deviation is large, it is a fluctuating interval. In this way, the stable time series interval and the fluctuating time series interval are divided, and finally the obstacle position-motion state uncertainty change stable time series interval and the obstacle position-motion state uncertainty change fluctuating time series interval are obtained.

[0160] Preferably, the obstacle position-motion state uncertainty time series change curve graph is subjected to motion state dynamic variation measurement analysis according to the obstacle position-motion state uncertainty change stable time series interval and the obstacle position-motion state uncertainty change fluctuating time series interval to obtain the obstacle position-motion state dynamic uncertainty variation measurement value.

[0161] In an embodiment of the present invention, similarly, by combining the previously divided stable time series intervals of obstacle position-motion state uncertainty change and the fluctuating time series intervals of obstacle position-motion state uncertainty change, the mean and standard deviation of the change data in each time series interval are calculated respectively, and by using the ratio of the data mean in all fluctuating time series intervals of the obstacle position-motion state uncertainty time series change curve to the data mean in the stable time series interval plus the ratio of the corresponding standard deviation as the dynamic uncertainty variation measure value. If the coefficient of variation is large, it means that the motion state uncertainty changes dramatically, and if the coefficient of variation is small, it means that the change is relatively stable. This measure value can be used to evaluate the variation of the motion state uncertainty during unmanned driving, and finally obtain the obstacle position-motion state dynamic uncertainty variation measure value.

[0162] Furthermore, the multi-sensor uncertainty adjustment calculation formula in step S34 is specifically:

[0163]

[0164] Where U tis the environmental multi-sensor uncertainty adjustment coefficient, n is the number of sensors in the unmanned driving environment, i is the sensor item measurement parameter, T is the total duration of the observation time window, t is the integral time variable parameter, and w i is the uncertainty weight coefficient of the i-th sensor, d i (t) is the obstacle distance measurement value of the i-th sensor at time t, α i is the obstacle position-distance dynamic uncertainty variation measure of the i-th sensor, m i (t) is the measured value of the motion state of the obstacle object by the i-th sensor at time t, β i is the obstacle position-motion state dynamic uncertainty variation measure of the i-th sensor, and η is the correction parameter of the environmental multi-sensor uncertainty adjustment coefficient.

[0165] The present invention obtains a multi-sensor uncertainty adjustment calculation formula by using a specific mathematical model and after verification, which is used to perform uncertainty adjustment calculation on the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value. The multi-sensor uncertainty adjustment calculation formula can more accurately estimate the uncertainty in the environment by integrating the dynamic uncertainty measurement of each sensor. This comprehensive adjustment helps to reduce the impact of a single sensor error on the overall data and enhance the reliability of the system. The formula also provides a way to weightedly process the data of multiple sensors and optimize the data fusion effect, which means that the data of different sensors can be adjusted according to their uncertainty, thereby improving the accuracy and consistency of the overall data fusion. The weight of each sensor in the data fusion can be dynamically adjusted by calculating the uncertainty adjustment coefficient, which allows automatic adjustment under different environmental conditions to ensure that the data fusion results are adapted to the actual situation. At the same time, the formula also takes into account the measurement values ​​and uncertainties of the sensors at different time points, thereby improving the performance in processing complex dynamic environments, which helps to improve the accuracy and stability of the unmanned driving process in real-time operation. In addition, by introducing correction parameters, the uncertainty adjustment coefficient is allowed to be further optimized and adjusted, which increases the flexibility of processing different sensor characteristics, so that different types of sensors can be better adapted. By accurately adjusting uncertainty, obstacles and motion states in the environment can be better perceived and understood, the accuracy and response speed of environmental perception are improved, and the decision-making ability of autonomous driving or intelligent systems is enhanced. In short, these effects help improve the performance of unmanned driving pre-controllers in multi-sensor environments, ensure the accuracy and reliability of data fusion, and thus improve the overall environmental perception and operation capabilities. Therefore, this formula fully considers the environmental multi-sensor uncertainty adjustment coefficient U t, the number of sensors in the unmanned driving environment n, the item measurement parameter i of the sensor, the total length of the observation time window T, the integral time variable parameter t, the uncertainty weight coefficient w of the i-th sensor i , the obstacle distance measurement value d of the i-th sensor at time t i (t), the obstacle position-distance dynamic uncertainty variation measure α of the i-th sensor i , the obstacle motion state measurement value m of the i-th sensor at time t i (t), the obstacle position-motion state dynamic uncertainty variation measure β of the i-th sensor i , the correction parameter η of the environmental multi-sensor uncertainty adjustment coefficient, according to the environmental multi-sensor uncertainty adjustment coefficient U t The correlation between the above parameters constitutes a functional relationship This formula can realize the uncertainty adjustment calculation process of the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value. At the same time, by introducing the correction parameter η of the environmental multi-sensor uncertainty adjustment coefficient, it can be adjusted according to the error situation occurring in the calculation process, thereby improving the accuracy and applicability of the multi-sensor uncertainty adjustment calculation formula.

[0166] Further, step S4 includes the following steps:

[0167] Step S41: performing environment fusion feature point cloud sampling on the environment multi-sensor uncertain optimization fusion feature data to obtain an environment multi-sensor fusion feature point cloud set;

[0168] In an embodiment of the present invention, the fused feature point cloud data is sampled by the environmental multi-sensor uncertain optimized fusion feature data previously obtained after fusion by the particle filter algorithm to select representative feature points. In this process, a downsampling technology based on a Voxel Grid filter can be used to reduce the complexity of the point cloud data and maintain the overall characteristics of the data, so that the environmental feature point cloud captured by different sensors can be obtained, and finally an environmental multi-sensor fusion feature point cloud set is obtained.

[0169] Step S42: performing environmental space modeling processing according to the environmental multi-sensor fusion feature point cloud set to generate an unmanned driving multi-sensor environmental data space model;

[0170] In an embodiment of the present invention, environmental space modeling is performed by using the environmental multi-sensor fusion feature point cloud set obtained by previous analysis, so as to divide the point cloud data into different environmental areas by applying point cloud segmentation technology, and fit the point cloud through geometric modeling algorithms, such as RANSAC (random sampling consensus algorithm) and least squares method, so as to generate a three-dimensional model of the environment. The model can be a grid model or a voxel model, which is selected according to specific application requirements. The generated model is used to describe important features such as obstacles and roads in the environment, and convert them into the spatial data structure required by the unmanned driving pre-controller, and finally construct a spatial model of unmanned driving multi-sensor environmental data.

[0171] Step S43: using a model predictive control method based on the unmanned driving multi-sensor environment data space model to predict the environmental state change response control of the unmanned driving pre-controller, so as to generate an unmanned driving environmental state prediction change response control instruction;

[0172] In an embodiment of the present invention, a model predictive control (MPC) method is used to perform control prediction of the environmental state change response of the unmanned driving pre-controller in combination with a previously constructed unmanned driving multi-sensor environmental data space model, so as to establish a dynamic model to describe the change of the environmental state, such as using a linear or nonlinear state space model, and by inputting the current environmental state data into the prediction model, the change of the future state of the environment is calculated. These calculation results will respond to the unmanned driving pre-controller and generate corresponding environmental state prediction change control instructions. The prediction model can use optimization algorithms such as gradient descent or genetic algorithm to solve the control problem, ensure that the generated control instructions can effectively respond to environmental changes, and finally generate unmanned driving environmental state prediction change response control instructions.

[0173] Step S44: Generate an unmanned driving environment state change driving control strategy by responding to the unmanned driving environment state prediction change response control instruction to the unmanned driving pre-controller and performing environment state change driving control strategy analysis to execute the corresponding unmanned driving environment change pre-control function.

[0174] In an embodiment of the present invention, a control instruction response based on a previously generated prediction change of the unmanned driving environment state is applied to the unmanned driving pre-controller, and a driving control strategy analysis of the unmanned driving pre-controller in response to the change of the environment state is performed on the unmanned driving pre-controller based on the prediction result. Specifically, a driving strategy is formulated based on a PID controller or LQR (linear quadratic regulator) algorithm in control theory. The driving control strategy adjusts the operation of the unmanned driving pre-controller according to the change of the environment state to achieve a smooth and safe response to the environment change. This process involves real-time calculation and adjustment of control instructions to ensure that the unmanned driving pre-controller can maintain stable driving performance in a dynamic environment, and finally generates an unmanned driving environment state change driving control strategy to execute the corresponding environment change pre-control function, thereby ensuring that the unmanned driving pre-controller can effectively respond to changes in complex environments.

[0175] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0176] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A multi-sensor data fusion method for an unmanned driving pre-controller, characterized in that: The following steps are involved: Step S1: Obtaining unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data through unmanned driving environment multi-sensors, and performing time series synchronization and standardization processing on the unmanned driving environment obstacle position data, unmanned driving environment object distance data, and unmanned driving environment object motion state data to obtain environmental obstacle position standard data, environmental object distance standard data, and environmental object motion state standard data; Step S2: performing feature space vector conversion on the standard data of the environmental obstacle position, the standard data of the environmental object distance and the standard data of the environmental object motion state to obtain the spatial feature vector of the environmental obstacle position, the spatial feature vector of the environmental object distance and the spatial feature vector of the environmental object motion state; performing multi-sensor particle filter fusion analysis on the spatial feature vector of the environmental obstacle position, the spatial feature vector of the environmental object distance and the spatial feature vector of the environmental object motion state to obtain the high-dimensional fusion feature representation data of the environmental multi-sensor; Step S3: performing multi-sensor uncertainty inference adjustment analysis on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector to obtain the environmental multi-sensor uncertainty adjustment coefficient; performing multi-sensor uncertainty fusion optimization on the environmental multi-sensor high-dimensional fusion feature representation data according to the environmental multi-sensor uncertainty adjustment coefficient to obtain the environmental multi-sensor uncertainty optimization fusion feature data; Step S4: Perform environmental space modeling processing on the environmental multi-sensor uncertain optimization fusion feature data to generate an unmanned driving multi-sensor environmental data space model; based on the unmanned driving multi-sensor environmental data space model, use the model predictive control method to perform environmental state driving control prediction analysis on the unmanned driving pre-controller, and generate an unmanned driving environmental state change driving control strategy to execute the corresponding unmanned driving environmental change pre-control function.

2. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: performing multi-sensor real-time monitoring of the unmanned driving environment space area through the unmanned driving environment multi-sensor to obtain the unmanned driving environment space multi-sensor monitoring data, wherein the unmanned driving environment space multi-sensor monitoring data includes environmental obstacle object detection image data monitored in real time by a high-resolution camera, environmental obstacle object spatial distribution data monitored in real time by a laser radar, and environmental obstacle object motion data monitored in real time by an inertial measurement unit; Step S12: performing obstacle image spatial positioning mapping on the environmental obstacle object detection image data to obtain the initial position data of the unmanned driving environment space obstacle; performing spatial position error correction on the initial position data of the unmanned driving environment space obstacle based on the environmental obstacle object spatial distribution data to obtain the unmanned driving environment obstacle position data; Step S13: measuring the distance between obstacles in the unmanned driving environment space area and the unmanned driving equipment based on the obstacle position data of the unmanned driving environment to obtain the distance data of the unmanned driving environment objects; Step S14: performing object motion dynamic trajectory analysis on the environmental obstacle object motion data to obtain the environmental obstacle object motion dynamic trajectory; performing obstacle operation state analysis on the obstacles in the unmanned driving environment space area based on the environmental obstacle object motion dynamic trajectory to obtain the unmanned driving environment object motion state data; Step S15: Perform time series synchronization processing on the unmanned driving environment obstacle position data, the unmanned driving environment object distance data and the unmanned driving environment object motion state data to obtain the environmental obstacle position data, the environmental object distance data and the environmental object motion state data in the same time series dimension; perform standardization processing on the environmental obstacle position data, the environmental object distance data and the environmental object motion state data in the same time series dimension to obtain the environmental obstacle position standard data, the environmental object distance standard data and the environmental object motion state standard data.

3. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 2 is characterized in that: The step S12 of correcting the spatial position error of the initial position data of the spatial obstacles in the unmanned driving environment based on the spatial distribution data of the environmental obstacles includes the following steps: The spatial distribution data of environmental obstacle objects and the initial position data of the spatial obstacles in the unmanned driving environment are mapped and transformed in the same spatial coordinate system to obtain the actual spatial distribution points of the environmental obstacle objects and the initial points of the spatial obstacles in the unmanned driving environment in the same spatial coordinate system; Based on the actual distribution points of environmental obstacles in the same spatial coordinate system, the initial points of the corresponding unmanned driving environment space obstacles are detected for spatial deviation, and the initial position deviation points of the unmanned driving environment obstacles are obtained; Based on the actual spatial distribution points of environmental obstacles in the same spatial coordinate system, the initial position deviation points of the unmanned driving environment obstacles are matched with the position deviation compensation calculation to obtain the environmental obstacle position deviation compensation value of each initial position deviation point; According to the environmental obstacle position deviation compensation value of each initial position deviation point, the spatial position error of the corresponding unmanned driving environment obstacle initial position data in the unmanned driving environment spatial obstacle is corrected to obtain the unmanned driving environment obstacle position data.

4. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: Analyze the spatial distribution characteristics of the standard data of the environmental obstacle positions to obtain spatial distribution characteristic data of the environmental obstacle positions; Step S22: performing distance pattern feature analysis on the environmental object distance standard data to obtain environmental object distance pattern feature data; Step S23: analyzing the dynamic behavior characteristics of the environmental object motion state standard data to obtain the dynamic behavior characteristic data of the environmental object motion; Step S24: performing feature space vector conversion on the environmental obstacle position spatial distribution feature data, the environmental object distance pattern feature data, and the environmental object motion dynamic behavior feature data to obtain the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector, and the environmental object motion state spatial feature vector; Step S25: Perform multi-sensor particle filter fusion analysis on the environmental obstacle position spatial feature vector, the environmental object distance spatial feature vector and the environmental object motion state spatial feature vector to obtain environmental multi-sensor high-dimensional fusion feature representation data.

5. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 4, characterized in that: Step S25 includes the following steps: Step S251: performing time series state division processing on the environment obstacle position spatial feature vector, the environment object distance spatial feature vector and the environment object motion state spatial feature vector to obtain the obstacle position, object distance and object motion state spatial feature vector in the initial state and the obstacle position, object distance and object motion state spatial feature vector in the dynamically changing state; Step S252: performing state representation particle conversion on the obstacle position, object distance and object motion state space feature vector in the initial state to obtain an environment space feature vector state representation particle group; Step S253: performing future state change prediction analysis on the obstacle position, object distance and object motion state spatial feature vector in a dynamically changing state to obtain future change prediction data of the obstacle position, object distance and object motion state; Step S254: performing state update weighted calculation on the corresponding environment state representation particles in the environment space feature vector state representation particle group according to the obstacle position, object distance and object motion state future change prediction data, and obtaining the state change update weight of each environment state representation particle in each state update period; performing state update resampling on the corresponding obstacle position, object distance and object motion state space feature vector in a dynamically changing state based on the state change update weight of each environment state representation particle in each state update period, and obtaining the obstacle position state update vector, object distance state update vector and object motion state update vector of each environment state representation particle; Step S255: Perform high-dimensional feature fusion mapping on the obstacle position state update vector, object distance state update vector and object motion state update vector of each environment state representation particle to obtain environment multi-sensor high-dimensional fusion feature representation data.

6. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: Perform Bayesian prior distribution modeling on the spatial feature vector of the environmental obstacle position to generate a Bayesian prior distribution mathematical model of the obstacle position feature; Step S32: performing a posteriori uncertainty inference analysis on the environmental object distance space feature vector and the environmental object motion state space feature vector based on the obstacle position feature Bayesian prior distribution mathematical model to obtain the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data; Step S33: performing dynamic variation measurement analysis based on the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data to obtain the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value; Step S34: using a multi-sensor uncertainty adjustment calculation formula to perform uncertainty adjustment calculation on the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value to obtain an environmental multi-sensor uncertainty adjustment coefficient; Step S35: performing multi-sensor uncertainty fusion optimization on the environmental multi-sensor high-dimensional fusion feature representation data according to the environmental multi-sensor uncertainty adjustment coefficient to obtain environmental multi-sensor uncertainty optimized fusion feature data.

7. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 1, characterized in that: Step S33 includes the following steps: Perform uncertainty time series dynamic change analysis on the obstacle position-distance Bayesian posterior uncertainty inference data and the obstacle position-motion state Bayesian posterior uncertainty inference data to obtain obstacle position-distance uncertainty time series dynamic change data and obstacle position-motion state uncertainty time series dynamic change data; Draw a time series change curve graph for the obstacle position-distance uncertainty time series dynamic change data and the obstacle position-motion state uncertainty time series dynamic change data to generate an obstacle position-distance uncertainty time series change curve graph and an obstacle position-motion state uncertainty time series change curve graph; Performing statistical analysis on the curve amplitude of the obstacle position-distance uncertainty time series change curve and the obstacle position-motion state uncertainty time series change curve to obtain the obstacle position-distance uncertainty time series change fluctuation amplitude and the obstacle position-motion state uncertainty time series change fluctuation amplitude; Based on the fluctuation amplitude of the obstacle position-distance uncertainty time series change and the fluctuation amplitude of the obstacle position-motion state uncertainty time series change, a dynamic variation measurement analysis is performed on the obstacle position-distance uncertainty time series change curve and the obstacle position-motion state uncertainty time series change curve to obtain the obstacle position-distance dynamic uncertainty variation measurement value and the obstacle position-motion state dynamic uncertainty variation measurement value.

8. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 7, characterized in that: The method of performing dynamic variation measurement analysis on the obstacle position-distance uncertainty time series variation curve graph and the obstacle position-motion state uncertainty time series variation curve graph based on the obstacle position-distance uncertainty time series variation fluctuation amplitude and the obstacle position-motion state uncertainty time series variation fluctuation amplitude comprises the following steps: Based on the fluctuation amplitude of the obstacle position-distance uncertainty time series change, the obstacle position-distance uncertainty time series change curve is divided into distance time series change intervals to obtain the obstacle position-distance uncertainty change stable time series interval and the obstacle position-distance uncertainty change fluctuation time series interval; According to the obstacle position-distance uncertainty change stable time series interval and the obstacle position-distance uncertainty change fluctuating time series interval, the obstacle position-distance uncertainty time series change curve is analyzed for distance dynamic variation measurement to obtain the obstacle position-distance dynamic uncertainty variation measurement value; Based on the fluctuation amplitude of the obstacle position-motion state uncertainty time series change, the obstacle position-motion state uncertainty time series change curve is divided into the motion state time series change interval to obtain the obstacle position-motion state uncertainty change stable time series interval and the obstacle position-motion state uncertainty change fluctuation time series interval; According to the stable time series interval of obstacle position-motion state uncertainty change and the fluctuating time series interval of obstacle position-motion state uncertainty change, the obstacle position-motion state uncertainty time series change curve is analyzed for dynamic variation measurement of motion state, and the dynamic uncertainty variation measurement value of obstacle position-motion state is obtained.

9. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 6, characterized in that: The multi-sensor uncertainty adjustment calculation formula in step S34 is specifically: Where U t is the environmental multi-sensor uncertainty adjustment coefficient, n is the number of sensors in the unmanned driving environment, i is the sensor item measurement parameter, T is the total duration of the observation time window, t is the integral time variable parameter, and w i is the uncertainty weight coefficient of the i-th sensor, d i (t) is the obstacle distance measurement value of the i-th sensor at time t, α i is the obstacle position-distance dynamic uncertainty variation measure of the i-th sensor, m i (t) is the measured value of the motion state of the obstacle object by the i-th sensor at time t, β i is the obstacle position-motion state dynamic uncertainty variation measure of the i-th sensor, and η is the correction parameter of the environmental multi-sensor uncertainty adjustment coefficient.

10. The multi-sensor data fusion method for unmanned driving pre-controller according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing environment fusion feature point cloud sampling on the environment multi-sensor uncertain optimization fusion feature data to obtain an environment multi-sensor fusion feature point cloud set; Step S42: performing environmental space modeling processing according to the environmental multi-sensor fusion feature point cloud set to generate an unmanned driving multi-sensor environmental data space model; Step S43: using a model predictive control method based on the unmanned driving multi-sensor environment data space model to predict the environmental state change response control of the unmanned driving pre-controller, so as to generate an unmanned driving environmental state prediction change response control instruction; Step S44: Generate an unmanned driving environment state change driving control strategy by responding to the unmanned driving environment state prediction change response control instruction to the unmanned driving pre-controller and performing environment state change driving control strategy analysis to execute the corresponding unmanned driving environment change pre-control function.

Images