A road condition recognition method based on multi-source heterogeneous data fusion

Abstract

The application discloses a road condition recognition method based on multi-source heterogeneous data fusion. Firstly, the application adopts multiple sensors to collect road image data and vehicle speed, acceleration and wheel speed data, then pre-processes each part of data and extracts features by using different algorithms, then performs feature-level fusion and dimension reduction processing on the extracted feature vectors by using a multi-source heterogeneous data space-time fusion strategy, and finally trains and tests by using a least squares support vector machine based on particle swarm optimization to establish a road condition recognition model. Through the pre-processing, feature extraction and recognition model establishment of the data collected by multiple sensors, the application realizes accurate and efficient recognition of road conditions caused by different severe weather. The application can fully exploit multiple sensor resources, and then comprehensively analyze the collected data, so as to improve the road condition classification recognition accuracy and environmental adaptability.

Description

Technical Field

[0001] This invention relates to the field of road condition recognition technology, and more specifically to a road condition recognition method based on multi-source heterogeneous data fusion. Background Technology

[0002] With the development of autonomous driving technology, intelligent and unmanned vehicles have become a trend in modern automobiles, but safety during driving remains crucial. Studies have shown that severe weather and its resulting road conditions, such as water accumulation, snow accumulation, and ice, are major factors affecting vehicle safety. Traffic accident rates rise significantly, especially on icy and snowy roads.

[0003] Modern intelligent vehicles inherently incorporate numerous sensors, such as wheel speed sensors, onboard radar, and cameras. This abundance of sensors enables intelligent vehicles to automatically learn, recognize, and adapt to road conditions, much like humans do. In the field of road condition recognition, based on the underlying principles, methods can be broadly categorized into two types: indirect recognition methods based on vehicle dynamics and direct recognition methods based on sensors.

[0004] 1. Indirect identification methods based on vehicle dynamics are mainly used to identify different road surface parameters. They require the establishment of a model that reflects the relationship between various factors and road surface parameters. By measuring these related factors and using the model, the road surface parameters can be calculated. They are mostly used to identify the road surface adhesion coefficient and road surface roughness. However, they do not have a certain predictive ability and the identification accuracy depends entirely on the accuracy of the model.

[0005] 2. Direct identification methods based on vehicle-mounted sensors can identify different road conditions before the tires come into contact with the road surface. Although they have a certain predictive ability, if data from a single sensor is used, the road surface features will not be fully reflected, which can easily lead to low identification accuracy.

[0006] With the development of autonomous driving technology, various methods are emerging to improve vehicle driving safety by recognizing different road conditions.

[0007] Chinese patent CN111695418A discloses a method and system for safe driving based on road condition detection. This method first uses roadside equipment to identify and classify collected road images; second, it generates message information from the output road condition information; and finally, it broadcasts this information via vehicle broadcasting to enable the driver to make driving decisions. While it can improve vehicle driving safety to some extent, the roadside equipment is expensive to install and has a limited scope of application. Furthermore, the broadcast range of the message information is restricted, and vehicles cannot actively identify road condition information.

[0008] Chinese patent CN111860322A discloses a method for identifying unstructured road surface types based on multi-source sensor information fusion. This method collects road image information, vehicle status, and GPS information, and trains different classifiers to identify and classify unstructured road surfaces. While it employs multi-source information fusion to identify different road surface conditions, the training time for the different classifiers is long, and the initial decision-making operation is costly. Furthermore, the accuracy of the output depends on the decision-making settings between the two classifiers.

[0009] Existing fusion recognition methods either focus on identifying road surface unevenness or on research into decision-level fusion of data, without fully utilizing vehicle dynamics feature data and road image feature data under different conditions, and the recognition accuracy is generally low. Summary of the Invention

[0010] In view of this, the present invention provides a road condition recognition method based on multi-source heterogeneous data fusion, which integrates vehicle dynamics feature data that can directly reflect the road surface adhesion coefficient with image feature data to provide a more comprehensive description of road features caused by different weather conditions, thereby improving the accuracy and environmental adaptability of road recognition.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0012] A road condition identification method based on multi-source heterogeneous data fusion includes the following steps:

[0013] Step 1) Road surface perception information collection:

[0014] Unmanned intelligent vehicles are used to collect road surface image data and vehicle dynamics-related data under different road conditions.

[0015] Step 2), Data Preprocessing and Feature Extraction:

[0016] The road surface image data and vehicle dynamics related data collected in step 1) are preprocessed; the corresponding feature values ​​are extracted using a variety of feature extraction algorithms, and finally two feature matrices describing different road conditions are constructed.

[0017] Step 3) Multi-source heterogeneous data fusion and dimensionality reduction:

[0018] The two feature matrices describing different road conditions in step 2) are fused at the feature level using a multi-source heterogeneous data spatiotemporal fusion strategy, and then the dimensionality of the fused feature matrix is ​​reduced using principal component analysis.

[0019] Step 4) Establishing a road condition recognition model:

[0020] The feature matrix after dimensionality reduction in step 3) is combined with the corresponding road type label to form a dataset; then the dataset is randomly divided into a training set and a test set in a 4:1 ratio; finally, the training set and the test set are input into a least squares support vector machine model based on particle swarm optimization for training and testing to obtain a road condition recognition model.

[0021] Furthermore, the vehicle dynamics-related data in step 1) includes vehicle longitudinal velocity, acceleration, and wheel speed information.

[0022] Furthermore, the method for road surface image data preprocessing and feature extraction in step 2) is as follows:

[0023] The road image data was preprocessed using USM and histogram equalization methods; color and texture features of the road images were extracted using color histogram, ULBP and GLCM methods respectively, and finally the image feature matrix was formed.

[0024] The method for vehicle dynamics preprocessing and feature extraction is as follows: the vehicle dynamics data is preprocessed using wavelet threshold denoising; the frequency domain features of the vehicle dynamics signal are extracted using power spectral density, and finally, the vehicle dynamics feature matrix is ​​formed.

[0025] Furthermore, the specific method for step 3) is as follows:

[0026] Step 3.1) The spatiotemporal fusion strategy for multi-source heterogeneous data includes temporal data fusion and spatial data fusion:

[0027] Time data fusion: The vehicle's starting point is used as the coordinate origin, and the output time of the integrated navigation system is used as the standard. Different sensors maintain the same sampling frequency and are required to start sampling at the same time. Data is recorded offline to ensure that the collected data has a consistent system timestamp.

[0028] Spatial data fusion: First, acquire the vehicle's mileage information and establish the vehicle's position axis x, with the center of the vehicle's front wheels as the origin. Second, divide the vehicle's dynamics feature vector into segments of equal length to the image data. Then, fuse the feature vector of each image with the segmented vehicle dynamics feature vector. The feature vector formed by each segment can be represented as...

[0029]

[0030] in f represents the first type of road surface. LA f WS and f LS This represents the longitudinal acceleration, wheel speed, and longitudinal velocity characteristic values. Finally, after the vehicle acquires a road image, it will be fused with the vehicle dynamics data within the next 0.25 seconds.

[0031] Step 3.2) Feature matrix dimensionality reduction: The fused feature matrix is ​​standardized and normalized, and then the PCA algorithm is used to reduce the dimensionality of the fused feature matrix to 88 dimensions, retaining 75% of the effective information, which is then provided to the least squares support vector machine based on particle swarm optimization for classification and recognition training.

[0032] Furthermore, the specific method for step 4 is as follows:

[0033] Step 4.1: Combine the road image features, vehicle dynamics features, and the fusion feature matrix after dimensionality reduction with the corresponding road type labels to form a dataset. Then, randomly divide the dataset into a training set and a test set in a 4:1 ratio.

[0034] Step 4.2: Improve the original support vector machine by introducing a particle swarm optimization algorithm to optimize the parameter set of the least squares support vector machine and establish a road condition recognition model.

[0035] Furthermore, the specific method for step 4.2 is as follows:

[0036] Step 4.2.1: When the data scale reaches several thousand or even tens of thousands, the least squares support vector machine model is used, the squared error term is selected as the optimization objective, the equality constraint is used as the constraint condition, and the final decision function is achieved by solving the linear equation system.

[0037] Introducing the Lagrange function, and simplifying, we obtain the optimal classification function for least squares support vector machines as follows:

[0038]

[0039] In the formula: k(x) k Let x be the kernel function, using the Gaussian radial basis function, whose mathematical expression is:

[0040] k(x,x k )=exp{-|xx k | 2 / σ 2}

[0041] In the formula, σ is the width of the radial basis kernel function;

[0042] Step 4.2.2: Select a one-to-one coding classification method, construct multiple binary LSSVMs, and combine them with a combined coding method to achieve multi-class classification;

[0043] Step 4.2.3: Select the particle swarm optimization algorithm to optimize the parameter set (γ, σ) in the least squares support vector machine. 2 Optimization is performed; the road condition recognition model modeling process is as follows:

[0044] (1) Read in the road condition data sample set that has been fused and dimensionality reduced;

[0045] (2) Initialize the PSO population, setting the particle swarm size to 50, the number of iterations to 200, the maximum value of the inertia weight for velocity updates to 0.95, and the minimum value to 0.6, and randomly generate a set of (γ,σ) 2 () as the initial position of the particle;

[0046] (3) Based on the current (γ,σ) 2 Perform LSSVM training, calculate the model fitting error under given parameters, and use it as the fitness function, where the fitness function is defined as:

[0047] E = 1 - P i

[0048] In the formula, E is the error, and P is the error. i It is the training accuracy given the parameters;

[0049] (4) Using the model fitting error as the fitness value, update the optimal fitness values ​​of the particle itself and the population based on the particle's fitness value, and update the velocity and position of each particle to obtain new LSSVM parameter values ​​(γ, σ). 2 );

[0050] (5) If the maximum number of iterations is not met or the termination condition is met, return to step (3); otherwise, output the optimal parameter value.

[0051] (6) Retrain LSSVM with the final optimal parameters to establish a road condition recognition model.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] 1. This invention fuses vehicle dynamics feature data, which directly reflects the road surface adhesion coefficient, with image feature data to provide a more comprehensive description of road characteristics caused by different weather conditions. Specifically, for image features, vehicle dynamics features can provide characteristics that directly reflect different road surface friction coefficients, to some extent compensating for the inherent susceptibility of image features to environmental influences. Conversely, for vehicle dynamics features, image features can provide visually distinguishable features for road surfaces with similar vehicle dynamics characteristics. Therefore, using multi-source heterogeneous fusion features can improve the accuracy of road condition classification and recognition, as well as environmental adaptability.

[0054] 2. This invention proposes using a least squares support vector machine based on particle swarm optimization as a road condition recognition model. The parameter set is optimized using the least squares (LS) method combined with particle swarm optimization (PSO). This invention employs a Gaussian radial basis function kernel; therefore, the parameter set to be optimized is (γ, σ). 2 The model fitting error under given parameters was set as the fitness function, and appropriate parameter sets were iteratively selected to improve the model's classification accuracy. This resulted in a maximum recognition rate of 89.06%, which is 3.38% higher than the traditional support vector machine and 0.78% higher than the least squares support vector machine, enhancing the applicability, accuracy, and robustness of support vector machines in different road condition classifications.

[0055] 3. Through training and testing, this invention fully verifies that the model can be applied to actual road condition recognition.

[0056] 4. This invention uses the least squares support vector machine model to select the squared error term as the optimization objective and uses equality constraints as the constraint conditions. The final decision function is achieved by solving a system of linear equations, which reduces the difficulty of solving the problem to a certain extent and improves the solution speed, making it more suitable for road condition recognition applications. Attached Figure Description

[0057] Figure 1 This is a technical roadmap for a road condition recognition method based on multi-source heterogeneous data fusion according to the present invention;

[0058] Figure 2 This is a data map of the eight road conditions to be identified in this invention;

[0059] Figure 3 This is a schematic diagram illustrating the accuracy of classification and recognition based on the fusion feature matrix of the present invention.

[0060] Figure 4 This is a comparison chart of the average recognition accuracy of different optimized algorithm road condition recognition models of the present invention.

[0061] Figure 5 The diagram shows the actual classification and predicted classification of the road condition recognition model of this invention, as well as the confusion matrix of recognition accuracy. Here, a represents the actual classification and predicted classification of the test set, and b represents the confusion matrix of recognition accuracy. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0063] like Figure 1 As shown, this invention provides a road surface type recognition method based on multi-texture feature fusion of road surface images, specifically including the following steps:

[0064] Step 1: Road surface perception information collection.

[0065] Road surface image data and vehicle dynamics data under different road conditions were collected through field experiments. An unmanned intelligent vehicle was used to collect image data on roads in different seasons and conditions (dry road surface, wet road surface, waterlogged road surface, rain and snow road surface, muddy snow road surface, compacted snow road surface, icy snow road surface, and ice road surface), with asphalt road surfaces as the baseline. During data collection, the unmanned intelligent vehicle was set to travel in a straight line and perform acceleration, deceleration, and constant speed reciprocating motion at a speed of 0 to 20 km / h. Road image data was collected using an onboard camera with a frame rate of 100 fps.

[0066] Vehicle dynamics data includes vehicle speed, acceleration, and wheel speed (based on the left front wheel). Vehicle speed and acceleration are collected using a combined navigation system and attitude angle sensors, while wheel speed is collected using Hall effect sensors. All sensors operate at a frequency of 100Hz. All data were collected using the ROS system and annotated with the same timestamp. The actual collected data sets are as follows: Figure 2 As shown.

[0067] Step 2: Data Preprocessing and Feature Extraction: The road image data and vehicle dynamics-related data collected in Step 1 are preprocessed. Various feature extraction algorithms are used to extract corresponding feature values, forming two feature matrices describing different road conditions. The specific steps are as follows:

[0068] Step 2.1: Road image data preprocessing and feature extraction. This mainly includes the following steps:

[0069] Step 2.1.1: Standardize the size of all road image data to 256×256, and preprocess the road image data using USM and histogram equalization to improve image contrast, brightness, and texture clarity.

[0070] Step 2.1.2: Extract color and texture features from the road image using color histogram, ULBP, and GLCM methods respectively. For color feature extraction, the image is converted to the HSV color space, and hue (H), saturation (S), and brightness (V) are used to represent color, resulting in a 256-dimensional color feature vector. For texture features, the window size for each image is set to 32×32, resulting in a 3776-dimensional ULBP texture feature vector. The angular second moment of the gray-level co-occurrence matrix in the four directions of deflection (0°, 45°, 90°, and 135°), as well as five statistical measures (contrast, correlation, entropy, and inverse difference), are calculated, and their corresponding mean, variance, and standard deviation are determined. Finally, a 15-dimensional GLCM texture feature vector is extracted.

[0071] Step 2.2, Vehicle Dynamics Data Preprocessing and Feature Extraction. This mainly includes the following steps:

[0072] Step 2.2.1: Preprocess the acquired raw signals of vehicle speed, acceleration, and wheel speed using wavelet thresholding. The db6 wavelet is selected as the base wavelet, and a 4-level wavelet decomposition is performed. Heuristic thresholding and hard thresholding functions are then used for denoising.

[0073] Step 2.2.2: The signal is transformed into the frequency domain, and the power spectral density is used to extract features from the vehicle dynamics signal. The power spectral density unit is logarithmically converted to dB. This amplifies the components with lower amplitudes, making it easier to observe periodic signals hidden in low-amplitude noise. A total of 75-dimensional vehicle dynamics feature vectors are ultimately extracted.

[0074] Step 3, Multi-source heterogeneous data fusion and dimensionality reduction: A spatiotemporal fusion strategy is used to fuse the two feature matrices describing different road conditions from Step 2 at the feature level, and the fused matrix is ​​then subjected to dimensionality reduction. The specific steps are as follows:

[0075] Step 3.1: Feature fusion of multi-source heterogeneous data. This mainly includes the following steps:

[0076] Step 3.1.1, Time Data Fusion. During the information acquisition process of the vehicle-mounted sensors, the camera captures road image information within a certain range in front of the vehicle, while the vehicle dynamics data acquired by the vehicle-mounted sensors is the response information of the vehicle to the contact marks on the road surface during operation. Because the different sensors used in this invention have the same operating frequency, achieving time data fusion only requires ensuring that the different sensors start working at the same time and have the same system timestamp. Taking the vehicle's starting point as the coordinate origin and the output time of the integrated navigation system as the standard, the sampling frequency of different sensors remains the same and they are required to start sampling at the same time. Data is recorded offline to ensure that the collected data has a consistent system timestamp.

[0077] Step 3.1.2, Spatial Data Fusion. First, acquire the vehicle's mileage information and establish the vehicle's position axis x, with the center of the front wheels as the origin. Second, after extracting vehicle dynamics signal features, a total of 6000-dimensional feature vectors are generated and divided into 240 equal segments. Each image can be combined with a 25-dimensional vehicle dynamics feature vector. The feature vector formed by each signal segment is...

[0078]

[0079] in f represents the first type of road surface. LA f WS and f LS Represents longitudinal acceleration, wheel speed, and longitudinal velocity.

[0080] The velocity feature value is then used. Finally, after each road image is captured, it is fused with the vehicle dynamics data from the next 0.25 seconds.

[0081] Step 3.2, Feature Matrix Dimensionality Reduction. This mainly includes the following steps:

[0082] Step 3.2.1: Feature Matrix Standardization and Normalization. The fused feature matrix is ​​standardized and normalized using the z-score method and the extreme value normalization method to eliminate differences in dimensions and orders of magnitude between different eigenvalues.

[0083] Step 3.2.2: Feature Matrix Dimensionality Reduction. The fused feature matrix is ​​standardized and normalized, and then the PCA algorithm is used to reduce the dimensionality of the fused feature matrix to 88 dimensions, retaining 75% of the effective information, which is then provided to the least squares support vector machine based on particle swarm optimization for classification and recognition training.

[0084] Step 4: Establishing a road condition recognition model: Input the dimensionality-reduced feature matrix from Step 3 into a least squares support vector machine model based on particle swarm optimization for training and testing to obtain recognition results for different road conditions.

[0085] The specific steps are as follows:

[0086] Step 4.1: Combine the road image features, vehicle dynamics features, and the fusion feature matrix after dimensionality reduction with the corresponding road type labels to form a dataset. Then, randomly divide the dataset into a training set and a test set in a 4:1 ratio.

[0087] Step 4.2: Improve the original Support Vector Machine by introducing a Particle Swarm Optimization (PSO) algorithm to optimize the parameter set of the least squares support vector machine and establish a road condition recognition model. This mainly includes the following steps:

[0088] Step 4.2.1: In the road condition recognition model, due to the support vector machine problem, the constraints are inequality constraints:

[0089]

[0090] sty k [w T Φ(x k +b)]≥1-e k

[0091] In the formula: w is the weight vector; C is the penalty coefficient, representing the degree of influence of training error on the objective function; e is a slack variable, which is used to introduce outliers into the support vectors; e k ≥0 and k=1,…,N, Φ(x) is a nonlinear mapping function; x is the input vector; b is the bias.

[0092] When the data scale reaches a certain level, the solution scale of the SVM algorithm becomes unsuitable for some traditional methods. The least squares support vector machine model, however, chooses the squared error term as the optimization objective and uses equality constraints as conditions, achieving the final decision function by solving a system of linear equations. This reduces the difficulty of the solution to some extent, improves the solution speed, and makes it more suitable for road condition recognition applications.

[0093] The original problem of least squares support vector machines becomes an equality constraint:

[0094]

[0095] sty k [w T Φ(x k +b)]=1-ξ k

[0096] In the formula, γ is the penalty factor, which has the same meaning as C in the previous formula; ξ k Let w be the error vector. Introduce the Lagrange function to solve the above equation, and substitute the result into f(x) = w T In Φ(x)+b, the optimal classification function can be obtained by rearranging:

[0097]

[0098] In the formula: k(x) k Let x be the kernel function. This invention uses the Gaussian radial basis kernel function, whose mathematical expression is:

[0099] k(x,x k )=exp{-|xx k | 2 / σ 2}

[0100] In the formula, σ is the width of the radial basis kernel function, which can affect the distribution of the feature vector in the new feature space after the feature vector is mapped by the function.

[0101] Step 4.2.2: Standard LSSVMs solve binary classification problems. For multi-class classification, multi-objective optimization and combined encoding methods are mainly adopted. This invention selects a one-to-one encoding classification method, constructs multiple binary LSSVMs, and combines them with the combined encoding method to achieve multi-class classification.

[0102] Step 4.2.3, Parameter set (γ, σ) 2 The value of γ affects the accuracy of model classification. This invention selects the particle swarm optimization algorithm to optimize the parameter set (γ, σ) in the least squares support vector machine. 2 Optimization is performed. The modeling process for the road condition recognition model is as follows:

[0103] (1) Read in the road condition data sample set that has been fused and dimensionality reduced;

[0104] (2) The PSO population is initialized. This invention sets the particle swarm size to 50, the number of iterations to 200, and the maximum value of the inertia weight for velocity updates to 0.95 and the minimum value to 0.6. A set of (γ, σ) values ​​is then randomly generated. 2 () as the initial position of the particle;

[0105] (3) Based on the current (γ,σ) 2 LSSVM training is performed, the model fitting error under given parameters is calculated, and this error is used as the fitness function. The fitness function is defined as follows:

[0106] E = 1 - P i

[0107] In the formula, E is the error, and P is the error. i It is the training accuracy given the parameters.

[0108] (4) Using the model fitting error as the fitness value, update the optimal fitness value P corresponding to the particle itself and the population based on the particle's fitness value. i k and And according to the formula

[0109]

[0110] Update the velocity V of each particle i k and location Thus, new LSSVM parameter values ​​(γ, σ) are obtained. 2 );

[0111] (5) If the maximum number of iterations is not met or the termination condition is met, return to step (3); otherwise, output the optimal parameter value.

[0112] (6) Retrain LSSVM with the final optimal parameters to establish a road condition recognition model.

[0113] This invention utilizes a test set to assess accuracy, tests road condition recognition models, and analyzes experimental results. The main steps include:

[0114] Step 1: Compare the classification accuracy of road type recognition methods using single features and fused features. Each feature uses an improved support vector machine as the machine learning classification model. The accuracy of road type recognition models using different features on the test set is shown below. Figure 3 As shown, the vehicle dynamics feature matrix and image feature matrix exhibit some distinguishability for classifying and recognizing different road conditions. However, the recognition accuracy of individual features is generally lower than that of the fused feature matrix. Using the fused features improves the recognition accuracy for different road conditions, especially for waterlogged roads, dry asphalt roads, muddy snow roads, compacted snow roads, and rainy / snowy roads, where the recognition accuracy exceeds 90%. This verifies the effectiveness of the multi-source heterogeneous data spatiotemporal fusion strategy proposed in this invention for road condition classification.

[0115] Step 2: To select the most suitable machine learning classification model for road condition recognition, the accuracy performance of five machine learning classification algorithms—Support Vector Machine (SVM), Least Squares Support Vector Machine (LS-SVM), Particle Swarm Optimization Support Vector Machine (PSO-SVM), Grid Search Optimization Support Vector Machine (GS-SVM), and Particle Swarm Optimization Least Squares Support Vector Machine (PSO-LSSVM)—was compared. Due to the uncertainty of the optimization results of the algorithms involved, each model underwent at least five training runs. The average recognition accuracy of different optimization algorithms for road condition recognition is compared in the following figure. Figure 4 As shown, the least squares support vector machine based on particle swarm optimization achieved an optimal recognition rate of 89.06%, which is 3.38% higher than the recognition accuracy of the traditional support vector machine and 0.78% higher than the recognition accuracy of the least squares support vector machine. This indicates that the model proposed in this invention has excellent performance in road classification under different conditions.

[0116] Step 3: Actual classification and predicted classification diagrams of the road condition recognition model, and confusion matrix diagram of recognition accuracy, as shown below. Figure 5As shown in the figure, the least squares support vector machine based on particle swarm optimization of this invention has the fewest false detections, and the recognition accuracy for waterlogged roads, dry asphalt roads, muddy snow roads, compacted snow roads, and rainy and snowy roads is higher than 90%, indicating that this model is very suitable for road condition recognition.

[0117] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. All components not explicitly stated in this embodiment can be implemented using existing technology.

Claims

1.A road condition recognition method based on multi-source heterogeneous data fusion, characterized in that: Includes the following steps: Step 1) Road surface perception information collection: Unmanned intelligent vehicles are used to collect road surface image data and vehicle dynamics-related data under different road conditions. Step 2), Data preprocessing and feature extraction: The road surface image data and vehicle dynamics related data collected in step 1) are preprocessed; the corresponding feature values ​​are extracted using various feature extraction algorithms, and finally two feature matrices describing different road conditions are constructed. The method for road surface image data preprocessing and feature extraction in step 2) is as follows: The road image data was preprocessed using USM and histogram equalization methods; color and texture features of the road images were extracted using color histogram, ULBP and GLCM methods respectively, and finally the image feature matrix was formed. The method for vehicle dynamics preprocessing and feature extraction is as follows: wavelet threshold denoising is used to preprocess the vehicle dynamics data; The power spectral density is used to extract the frequency domain features of vehicle dynamics signals, and finally a vehicle dynamics feature matrix is ​​formed. Step 3) Multi-source heterogeneous data fusion and dimensionality reduction: The two feature matrices describing different road conditions in step 2) are fused at the feature level using a multi-source heterogeneous data spatiotemporal fusion strategy, and then the dimensionality of the fused feature matrix is ​​reduced using principal component analysis. Feature matrix dimensionality reduction: The fused feature matrix is ​​standardized and normalized, and then the PCA algorithm is used to reduce the dimensionality of the fused feature matrix to 88 dimensions, retaining 75% of the effective information, which is then provided to the least squares support vector machine based on particle swarm optimization for classification and recognition training. The specific method for step 3) is as follows: Step 3.1) The spatiotemporal fusion strategy for multi-source heterogeneous data includes temporal data fusion and spatial data fusion: Time data fusion: The vehicle's starting point is used as the coordinate origin, and the output time of the integrated navigation system is used as the standard. Different sensors maintain the same sampling frequency and are required to start sampling at the same time. Data is recorded offline to ensure that the collected data has a consistent system timestamp. Spatial data fusion: First, the driving mileage information of the vehicle is acquired and the vehicle position axis is established with the center of the front wheel of the vehicle as the coordinate origin, second, the vehicle dynamics feature vector is divided into small segments with the same length as the image data, and then each image feature vector is fused with the segmented vehicle dynamics feature vector. The feature vector formed by each segment of signal can be expressed as wherein represents a first type of road surface, , and represent longitudinal acceleration, wheel speed and longitudinal speed characteristic values, which are fused with the vehicle dynamics data for the next 0.25 seconds after each picture of the road is captured by the vehicle. Step 4) Establishing a road condition recognition model: The feature matrix after dimensionality reduction in step 3) is combined with the corresponding road type label to form a dataset; then the dataset is randomly divided into a training set and a test set in a 4:1 ratio; finally, the training set and the test set are input into a least squares support vector machine model based on particle swarm optimization for training and testing to obtain a road condition recognition model; the road condition recognition model selects a one-to-one encoding classification method, and achieves multi-classification by constructing multiple binary LSSVMs and combining them with a combined encoding method. 2.The road condition recognition method based on multi-source heterogeneous data fusion according to claim 1, characterized in that: The vehicle dynamics-related data in step 1) include vehicle longitudinal velocity, acceleration, and wheel speed information. 3.The road condition recognition method based on multi-source heterogeneous data fusion according to claim 2, characterized in that, The specific method for step 4 is as follows: Step 4.1: Combine the road image features, vehicle dynamics features, and the fusion feature matrix after dimensionality reduction with the corresponding road type labels to form a dataset. Then, randomly divide the dataset into a training set and a test set in a 4:1 ratio. Step 4.2: Improve the original support vector machine by introducing a particle swarm optimization algorithm to optimize the parameter set of the least squares support vector machine and establish a road condition recognition model. 4.The road condition recognition method based on multi-source heterogeneous data fusion according to claim 3, characterized in that, The specific method of step 4.2 is: Step 4.2.1, when the data size reaches several thousand or even tens of thousands, the least square support vector machine model is used to select the error square term as the optimization target, the equation constraint is taken as the constraint condition, and the final decision function is realized by solving the linear equation set; The Lagrange function is introduced, and through arrangement, the optimal classification function of the least square support vector machine is: In the formula: is a kernel function, and a Gaussian radial basis kernel function is adopted, and the mathematical expression thereof is as follows: wherein is a radial basis function width; Step 4.2.2, select one-to-one encoding classification method, realize multi-classification by constructing multiple binary classification LSSVM and combining with the combination encoding method; Step 4.2.3, Selecting particle swarm optimization algorithm to optimize parameter set in least squares support vector machine Optimization is carried out; the road condition recognition model modeling process is as follows: (1) read in the road condition data sample set after fusion and dimension reduction; (2) The PSO population is initialized, the number of particle swarm is set to 50, the iteration number is set to 200, the maximum value of the inertia weight of velocity update is set to 0.95, the minimum value is set to 0.6, and a set of as the initial position of the particle; (3) According to the current LSSVM training is performed, the model fitting error under given parameters is calculated, and it is taken as the fitness function, wherein the fitness function is defined as: wherein is the error, is the training accuracy under the given parameters; (4) Take the model fitting error as the fitness value, update the best fitness value and of the particle itself and the group according to the fitness value of the particle, and update the speed and position of each particle to obtain the new LSSVM parameter value ; (5) if the maximum iteration number is not met or the end condition is met, return to step (3), otherwise output the optimal parameter value; (6) retrain the LSSVM with the last obtained optimal parameter to establish a road condition recognition model.

Images