Obstacle detection system for unmanned driving

Abstract

The invention relates to the technical field of unmanned driving, and particularly discloses an obstacle detection system for unmanned driving, which is characterized in that a multi-modal acquisition module acquires vehicle surrounding environment data according to near, middle and far layers, a data preprocessing and synchronizing module performs standardization processing on the acquired data and then outputs a standardized heterogeneous data set with a detection interval identifier; two types of feature libraries are arranged in the dynamic weighted fusion judgment module, after obstacle features are extracted from a data set, an obstacle judgment result is output through hierarchical weight distribution, dynamic correction, feature fusion and two-dimensional judgment, and the master control terminal receives the judgment result and outputs normal driving, constant speed deceleration, steering avoidance or emergency braking instructions. The obstacle detection accuracy, the response timeliness and the environmental adaptability of the unmanned vehicle under the complex road condition are improved, and reliable guarantee is provided for safe driving of the unmanned vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an obstacle detection system for autonomous driving. Background Technology

[0002] Environmental perception and obstacle detection are core components of autonomous driving technology. Their detection accuracy, response speed, and environmental adaptability directly determine vehicle driving safety and are key bottlenecks restricting the mass production and deployment of autonomous driving technology from Level 2 assisted driving to higher levels. With the rapid development of the intelligent connected vehicle industry, the technical defects of existing obstacle detection systems in complex real-world road conditions have become increasingly prominent, and have become one of the main causes of safety accidents.

[0003] Existing obstacle detection methods for autonomous vehicles mostly employ fixed weighting ratios for data fusion, failing to consider the distance characteristics between obstacles and vehicles when dividing detection zones. For sudden obstacles requiring immediate response at close range, the weighting of key data is not increased, leading to response delays. For occluded obstacles requiring accurate identification at mid-range distance, the fusion weighting of effective features is not specifically strengthened, resulting in insufficient recognition accuracy. For static obstacles requiring early warning at far distance, excessive weighting leads to data redundancy and affects system computational efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide an obstacle detection system for unmanned vehicles, which aims to solve the technical problems in the prior art.

[0005] To achieve the above objectives, the present invention employs an obstacle detection system for unmanned driving, comprising a multimodal acquisition module: layered acquisition of vehicle surrounding environment data: near-end acquisition of 3D point cloud, road vibration and near-range ranging data, mid-end acquisition of multispectral visual images, millimeter-wave dynamic ranging and infrared thermal imaging data, and far-end acquisition of long-focus lidar point cloud and panoramic visual data;

[0006] Data preprocessing and synchronization module: Denoises, normalizes, timestamps, and registers spatial coordinates on the collected data, and outputs a standardized heterogeneous dataset with near, middle, and far detection interval labels;

[0007] Dynamic weighted fusion judgment module: Built-in flexible occlusion obstacle feature library, dynamic sudden obstacle feature library and hierarchical dynamic weighted fusion judgment module;

[0008] The hierarchical dynamic weighted fusion determination module is used to extract obstacle features from the multispectral visual images and panoramic visual data of the corresponding detection range in the standardized heterogeneous dataset, and to perform hierarchical weight allocation, dynamic weight correction, feature fusion and obstacle recognition determination through the extracted obstacle feature data;

[0009] Main control terminal: Used to connect to vehicle positioning and speed control modules, receive obstacle judgment results from dynamic weighted fusion judgment module, and output execution commands including normal driving, constant speed deceleration, steering avoidance, and emergency braking.

[0010] The flexible obstruction feature library stores the three-dimensional contour features, infrared spectral features, and local texture features of obstacles such as green plants obstructing roadblocks, fog obstructing pedestrians, and construction fences obstructing obstacles.

[0011] The dynamic sudden obstacle feature database stores the motion trajectory features, speed change features, and contour dynamic change features of pedestrians suddenly crossing, electric vehicles changing direction, and rolling debris.

[0012] The algorithm formula for the hierarchical weight allocation is as follows: ,in Weighting is applied to near-end data, with a weighting ratio of 60%-70%. For mid-range data, the weighting ratio is 20%-25%. The weighting of remote data is 5%-20%, and it meets the following requirements: ;

[0013] The hierarchical weight allocation outputs the initial weight vector. , which serves as the input data for dynamic weight correction.

[0014] The dynamic weight correction formula is as follows: ,in For environmental adaptability coefficient, Quantification of the degree of occlusion or dynamic suddenness. For the magnitude of the extracted feature vector, The magnitude of the corresponding standard feature vector in the feature library;

[0015] The dynamic weight correction uses the initial weight vector and feature extraction Input is the corrected weight vector. ,in Compared with the original weight Corresponding, and maintaining .

[0016] The feature fusion formula is as follows: ,in For the first The obstacle feature vector corresponding to the data collected by each sensor for L2 norm, For the revised first Individual sensor data weights;

[0017] The feature fusion uses the feature vectors output by each sensor. and the corrected weight vector Input is fused feature vector. Output is fused feature vector. This enables the synergistic integration of multi-source heterogeneous features.

[0018] The identification and determination formula is as follows: ,in For matching probability, Let k be the standard feature vector of the k-th type of obstacle in the feature library. The width of the Gaussian kernel. ,in It is the set of all obstacle categories in the feature library;

[0019] when At that time, it was determined to be related to the first If an obstacle is successfully matched, the obstacle category, distance range, and risk level are output; the identification judgment is based on the fused feature vector. The core input is used as the basis for generating instructions to be executed by the main control terminal.

[0020] The hierarchical dynamic weighted fusion determination module also includes a feature self-updating submodule: when When a new obstacle feature is identified or a variant feature of a known obstacle is identified, the formula is used to determine whether it is a new obstacle feature or a variant feature of a known obstacle. Perform a smooth update, in which To update the coefficients, For the feature vector of the newly identified obstacle, The standard feature vector in the feature library that has the highest similarity to the newly identified obstacle;

[0021] Updated feature vector Based on their physical and motion attributes, they are classified and added to the flexible occlusion obstacle feature library or the dynamic sudden obstacle feature library to achieve incremental iterative optimization of the feature library.

[0022] The hierarchical dynamic weighted fusion determination module employs a two-dimensional determination model in the obstacle recognition process: the fused feature vector Decomposed into physical attribute feature sub-vectors and motion attribute feature vector ;

[0023] The two-dimensional matching probability is calculated separately using the recognition and judgment formula:

[0024] and ,in For the first Standard physical property feature vectors of obstacle classes For the first Standard motion attribute feature vectors for obstacle classes;

[0025] The overall matching probability is when The match was ultimately determined to be successful.

[0026] The data preprocessing and the noise reduction of the synchronization module adopt the Kalman filter model. Data noise reduction is achieved through preset filtering equations and update equations. The filtering equations are used for state prediction, and the update equations are combined with the observations to correct the prediction results in order to improve data accuracy.

[0027] The process noise matrix and observation noise matrix of the Kalman filter model are preset according to the sensor type.

[0028] The complete execution flow of the hierarchical dynamic weighted fusion determination module is as follows:

[0029] S1: Receives the standardized heterogeneous dataset output by the data preprocessing and synchronization modules, and extracts obstacle feature vectors for each detection interval. and mold length ;

[0030] S2: Calculate the initial weight vector using the hierarchical weight allocation formula. ;

[0031] S3: Obtain by combining feature library matching results , , Substituting into the dynamic weight correction formula yields the corrected weight vector. ;

[0032] S4: Will , and Substitute into the feature fusion formula to generate the fused feature vector. ;

[0033] S5: Based on a two-dimensional judgment model, the comprehensive matching probability is calculated by identifying the judgment formula. Output the obstacle determination result;

[0034] S6: According to The numerical trigger feature self-update submodule completes feature library optimization.

[0035] The obstacle detection system for unmanned driving according to the present invention has the following beneficial effects:

[0036] 1. Near-end data has the highest weight, and combined with dynamic weight correction, it effectively improves the response speed to sudden obstacles; mid-end weight is adapted to the needs of accurate recognition, effectively improving the accuracy of occluded obstacle recognition; far-end weight is simplified, effectively reducing data redundancy and effectively improving system computing efficiency, balancing response speed, recognition accuracy and computing efficiency.

[0037] 2. By splitting physical attributes and motion attributes for dual matching and combining two special feature libraries for flexible occlusion and dynamic emergencies, the error rate of obstacle misjudgment in complex scenarios such as fog, green plant occlusion, and sudden crossing can be effectively reduced, and the recall rate of typical obstacle recognition can be effectively improved.

[0038] 3. By dynamically supplementing new or mutated obstacle features through smooth update formulas, the system can adapt to more unknown scenarios without manual intervention, effectively improving the system's adaptability to new obstacles and expanding the system's scope of application. Detailed Implementation

[0039] The embodiments of the present invention are described in detail below. These embodiments are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0040] This invention provides an obstacle detection system for unmanned driving.

[0041] In this embodiment, the multimodal acquisition module adopts a hierarchical acquisition strategy of near-end, mid-end, and far-end, adapting differentiated acquisition data to the detection needs of different distance ranges, ensuring the relevance and comprehensiveness of environmental data acquisition. The data preprocessing and synchronization module eliminates heterogeneous interference from multi-source sensor data through standardized processing such as denoising and normalization, while adding detection range identifiers. The hierarchical processing provides a unified format dataset. The dynamic weighted fusion judgment module has two types of specialized feature libraries and a hierarchical fusion judgment unit built in, responsible for feature extraction and multi-step fusion judgment, realizing accurate judgment of dynamic or static occlusions. The main control terminal realizes the connection between the system and the vehicle actuator, receives the judgment results and outputs corresponding driving commands, thereby ensuring the driving safety of the autonomous vehicle.

[0042] Furthermore, a flexible occlusion obstacle feature library is developed. Focusing on occlusion-related obstacle scenarios, it stores the 3D contours, infrared spectra, and local texture features of obstacles in typical occlusion scenarios such as vegetation occlusion and fog occlusion, along with related images. A dynamic sudden obstacle feature library focuses on sudden obstacle scenarios, storing the motion trajectory, speed changes, and dynamic contour variations of sudden obstacles such as pedestrians and swerving electric vehicles. This adapts to the rapid obstacle recognition needs in dynamic scenarios. Thus, these two feature libraries specifically cover common high-risk obstacle scenarios in autonomous driving, providing accurate standard feature references for feature matching and recognition.

[0043] Furthermore, using the detection interval identifiers of the standardized heterogeneous dataset as implicit input, the initial weight vector is output. The data is directly used as input data for the dynamic weight correction module. Near-end obstacles require an urgent response, so the highest weight is assigned to ensure response speed; mid-range obstacles require accurate identification, so a medium weight is assigned to balance accuracy and efficiency; and far-end obstacles only require early warning, so the lowest weight is assigned to reduce data redundancy.

[0044] Furthermore, through For initial weights Make corrections and introduce an environmental adaptability coefficient. , occlusion or dynamic burst quantization value and the ratio of the magnitude of the eigenvectors This causes the weights to change dynamically with the environment and obstacle states, where Adapt to changes in environment such as weather and lighting. Quantify the degree of obstruction or the severity of suddenness. To ensure the compatibility between extracted features and standard features, an initial weight vector is used. and feature extraction Input is the corrected weight vector. Furthermore, the weights are kept to a total of 1 to ensure the rationality and operability of the corrected weights. This addresses the problem that the initial weights cannot adapt to complex environments and dynamic changes in obstacles, making the data fusion weights more targeted.

[0045] Furthermore, through feature fusion formula Feature vectors of each sensor Weighted fusion is performed using the feature vectors of each sensor. and the corrected weight vector Input is fused feature vector. Output is fused feature vector. By weighting, high-value features, such as those of near-end high-weight data, are highlighted, while low-value interference features are suppressed. At the same time, normalization is used to eliminate scale differences in features from different sensors, thus achieving complementary advantages of multi-source data.

[0046] Furthermore, by fusing feature vectors The core input is the output, which serves as the basis for the main control terminal to generate execution instructions. This transforms the fused high-dimensional features into quantifiable matching probabilities, clarifies the threshold for obstacle recognition, and ensures the accuracy and consistency of the recognition results.

[0047] Furthermore, through the updated feature vector Based on their physical and motion attributes, they are classified and added to the flexible occlusion obstacle feature library or the dynamic sudden obstacle feature library, so as to realize the dynamic expansion of the feature library and allow the system to continuously optimize its recognition capabilities as the usage scenarios expand.

[0048] Furthermore, by combining physical and motion attributes for dual verification, the impact of environmental interference on single-dimensional judgment is reduced, and the recognition accuracy is improved in complex scenarios such as occlusion and sudden events.

[0049] Furthermore, the denoising scheme, the Kalman filter model, achieves data denoising through a two-step logic of state prediction and observation correction. The specific equations are directly applied to real-world scenarios, with state prediction being... The equation predicts the current data state based on historical data, where Here is the state transition matrix. To control the input matrix, This is the optimal estimate from the previous moment. This is the predicted value at the current moment;

[0050] Observations corrected to This equation incorporates actual observations from the current sensors.

[0051] The prediction results were revised, in which For the observation matrix, The Kalman gain is used to correct and eliminate prediction bias, outputting the optimal estimate for the current time step. The process noise matrix of the Kalman filter model and observation noise matrix Differentiated values ​​are preset according to the sensor type. For example, the noise characteristics of LiDAR are different from those of vision sensors, and corresponding values ​​are configured for each. and The matrix ensures that the filtering effect is adapted to the data characteristics of different sensors. This denoising scheme is directly applied to the raw data output by the multimodal acquisition module to eliminate random noise and systematic errors in 3D point clouds, visual images, and ranging data. The processed data is then normalized and timestamp aligned to form a standardized heterogeneous dataset, providing a data foundation for hierarchical feature extraction and dynamic weighted fusion, and avoiding feature extraction distortion and judgment errors caused by noise.

[0052] Example 1:

[0053] This embodiment is applied to an autonomous driving scenario on urban roads:

[0054] 1. The multimodal acquisition module collects data in three layers: near, middle and far. The near sensor detects sudden crossing of obstacles, the middle sensor captures the outline and motion state of the obstacles, and the far sensor does not detect any abnormalities.

[0055] 2. The data preprocessing and synchronization module uses a Kalman filter model for noise reduction, based on the state monitoring formula. Observation correction formula After normalization, timestamp alignment, and spatial coordinate registration, a standardized heterogeneous dataset with near-end labels is output.

[0056] 3. The dynamic weighted fusion judgment module extracts the obstacle feature vectors and magnitudes of each detection interval, and assigns them using a hierarchical weight allocation formula. and Calculate the initial weight vector;

[0057] Combining environmental adaptability coefficients and quantified values ​​of occlusion or dynamic burstiness, a dynamic weighting correction formula is used. The corrected weight vector is obtained.

[0058] Substitute into the feature fusion formula Generate a fused feature vector;

[0059] Based on a two-dimensional judgment model, the judgment formula is identified. and the two-dimensional comprehensive formula Calculate the overall matching probability and determine whether it is a sudden obstacle, near end, or high risk;

[0060] 4. The main control terminal receives the judgment result and outputs an emergency braking command to achieve obstacle avoidance.

[0061] The above-disclosed embodiments are merely preferred embodiments of an obstacle detection system for unmanned driving according to the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. An obstacle detection system for unmanned driving, characterized in that, Includes a multimodal acquisition module: layered acquisition of vehicle surrounding environment data: near-end acquisition of 3D point cloud, road vibration and short-range ranging data, mid-end acquisition of multispectral visual images, millimeter-wave dynamic ranging and infrared thermal imaging data, and far-end acquisition of long-range lidar point cloud and panoramic visual data; Data preprocessing and synchronization module: Denoises, normalizes, timestamps, and registers spatial coordinates on the collected data, and outputs a standardized heterogeneous dataset with near, middle, and far detection interval labels; Dynamic weighted fusion judgment module: Built-in flexible occlusion obstacle feature library, dynamic sudden obstacle feature library and hierarchical dynamic weighted fusion judgment module; The hierarchical dynamic weighted fusion determination module is used to extract obstacle features from the multispectral visual images and panoramic visual data of the corresponding detection interval of the standardized heterogeneous dataset, and to perform hierarchical weight allocation, dynamic weight correction, feature fusion and obstacle recognition determination through the extracted obstacle feature data; Main control terminal: Used to connect to vehicle positioning and speed control modules, receive obstacle judgment results from dynamic weighted fusion judgment module, and output execution commands including normal driving, constant speed deceleration, steering avoidance, and emergency braking.

2. The obstacle detection system for unmanned driving as described in claim 1, characterized in that, The flexible obstruction feature library stores the three-dimensional contour features, infrared spectral features, and local texture features of obstacles such as green plants obscuring roadblocks, fog obscuring pedestrians, and construction fences obscuring obstacles. The dynamic sudden obstacle feature database stores the motion trajectory features, speed change features, and contour dynamic change features of pedestrians suddenly crossing, electric vehicles changing direction, and rolling debris.

3. The obstacle detection system for unmanned driving as described in claim 2, characterized in that, The algorithm formula for the hierarchical weight allocation is as follows: ,in Weighting is applied to near-end data, with a weighting ratio of 60%-70%. For mid-range data, the weighting ratio is 20%-25%. The weighting of remote data is 5%-20%, and it meets the following requirements: ; The hierarchical weight allocation outputs the initial weight vector. , which serves as the input data for dynamic weight correction.

4. The obstacle detection system for unmanned driving as described in claim 3, characterized in that, The dynamic weight correction formula is as follows: ,in For environmental adaptability coefficient, Quantification of the degree of occlusion or dynamic suddenness. For the magnitude of the extracted feature vector, The magnitude of the corresponding standard feature vector in the feature library; The dynamic weight correction uses the initial weight vector and feature extraction Input is the corrected weight vector. ,in Compared with the original weight Corresponding, and maintaining .

5. The obstacle detection system for unmanned driving as described in claim 4, characterized in that, The feature fusion formula is as follows: ,in For the first The obstacle feature vector corresponding to the data collected by each sensor for L2 norm, For the revised first Individual sensor data weights; The feature fusion uses the feature vectors output by each sensor. and the corrected weight vector Input is fused feature vector. Output is fused feature vector. This enables the synergistic integration of multi-source heterogeneous features.

6. The obstacle detection system for unmanned driving as described in claim 5, characterized in that, The identification and determination formula is as follows: ,in For matching probability, Let k be the standard feature vector of the k-th type of obstacle in the feature library. The width of the Gaussian kernel. ,in It is the set of all obstacle categories in the feature library; when At that time, it was determined to be related to the first If an obstacle is successfully matched, the obstacle category, distance range, and risk level are output; the identification judgment is based on the fused feature vector. The core input is used as the basis for generating instructions to be executed by the main control terminal.

7. The obstacle detection system for unmanned driving as described in claim 6, characterized in that, The hierarchical dynamic weighted fusion determination module also includes a feature self-updating submodule: when When a new obstacle feature is identified or a variant feature of a known obstacle is identified, the formula is used to determine whether it is a new obstacle feature or a variant feature of a known obstacle. Perform a smooth update, in which To update the coefficients, For the feature vector of the newly identified obstacle, The standard feature vector in the feature library that has the highest similarity to the newly identified obstacle; Updated feature vector Based on their physical and motion attributes, they are classified and added to the flexible occlusion obstacle feature library or the dynamic sudden obstacle feature library to achieve incremental iterative optimization of the feature library.

8. The obstacle detection system for unmanned driving as described in claim 7, characterized in that, The hierarchical dynamic weighted fusion determination module employs a two-dimensional determination model in the obstacle recognition process: the fused feature vector Decomposed into physical attribute feature sub-vectors and motion attribute feature vector ; The two-dimensional matching probability is calculated separately using the recognition and judgment formula: and ,in For the first Standard physical property feature vectors of obstacle classes For the first Standard motion attribute feature vectors for obstacle classes; The overall matching probability is when The match was ultimately determined to be successful.

9. The obstacle detection system for unmanned driving as described in claim 8, characterized in that, The data preprocessing and the noise reduction of the synchronization module adopt the Kalman filter model. Data noise reduction is achieved through preset filtering equations and update equations. The filtering equations are used for state prediction, and the update equations are combined with the observations to correct the prediction results in order to improve data accuracy. The process noise matrix and observation noise matrix of the Kalman filter model are preset according to the sensor type.

10. The obstacle detection system for unmanned driving as described in claim 9, characterized in that, The complete execution flow of the hierarchical dynamic weighted fusion determination module is as follows: S1: Receives the standardized heterogeneous dataset output by the data preprocessing and synchronization modules, and extracts obstacle feature vectors for each detection interval. and mold length ; S2: Calculate the initial weight vector using the hierarchical weight allocation formula. ; S3: Obtain by combining feature library matching results , , Substituting into the dynamic weight correction formula yields the corrected weight vector. ; S4: Will , and Substitute into the feature fusion formula to generate the fused feature vector. ; S5: Based on a two-dimensional judgment model, the comprehensive matching probability is calculated by identifying the judgment formula. Output the obstacle determination result; S6: According to The numerical trigger feature self-update submodule completes feature library optimization.

Images