Comprehensive estimation method of road adhesion coefficient by combining visual prediction and tactile verification
By combining visual prediction with tactile verification, and utilizing an adaptive square root cubature Kalman filter and the YOLOv5 deep learning algorithm, the problems of slow response, poor stability, and low accuracy in road adhesion coefficient estimation are resolved, enabling real-time and accurate estimation of the road adhesion coefficient and improving vehicle safety control.
Patent Information
- Application Number
- CN202411086044.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2044-08-08
AI Technical Summary
Existing road adhesion coefficient estimation methods have slow response speed, poor stability and low accuracy. In particular, the filtering process involves complex calculations such as matrix square root, inversion and transposition, which affect the estimation accuracy and fail to achieve real-time estimation.
A method combining visual prediction and tactile verification is adopted to obtain road information in real time through the on-board camera. A nonlinear four-wheel vehicle dynamics model and tire model are established. An adaptive square root cubature Kalman filter with orthogonal triangular decomposition is introduced. The time-varying process noise and visual prediction results are combined to perform real-time estimation of the road adhesion coefficient. Online training and dataset expansion are performed using the YOLOv5 deep learning algorithm.
The response speed, stability and accuracy of road adhesion coefficient estimation are improved, real-time and accurate estimation of road adhesion coefficient is achieved, and the effect of vehicle active safety control is enhanced.
Smart Images
Figure CN119190036B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of automatic driving control, and specifically relates to a comprehensive estimation method of road adhesion coefficient that combines visual prediction with tactile verification. Background Art
[0002] Vehicle-road interaction is a crucial factor affecting traffic safety. Traditionally, vehicle safety relies on the interaction between driver perception and anticipation. However, with the advancement of automotive electronics and intelligent technologies, intelligent vehicle motion control systems have assumed an even greater role in vehicle active safety control. The road adhesion coefficient, a key parameter influencing the effectiveness of active safety control, cannot be directly measured by on-board sensors. Current research generally employs indirect measurement methods, either cause-based or effect-based.
[0003] Cause-based methods estimate the road adhesion coefficient by installing specialized sensors on the vehicle or tires to measure various parameters related to the adhesion coefficient. This method requires specialized equipment, is costly, and is susceptible to external factors, resulting in poor robustness. Effect-based methods use common on-board sensors to measure the vehicle's dynamic response signals caused by the adhesion coefficient, establish a vehicle dynamics model, and combine filtering algorithms to estimate the road adhesion coefficient. Currently used filtering algorithms for the road adhesion coefficient include the extended Kalman filter, the unscented Kalman filter, and the cubic Kalman filter. The cubic Kalman filter uses the spherical-radial rule for approximate estimation and offers improved numerical stability. However, the filtering process involves complex computations such as matrix square root extraction, inversion, and transposition, which can easily violate the symmetry and positive definiteness of the covariance matrix, affecting estimation accuracy. Furthermore, vehicles experience dynamic changes during actual driving. Conventional filtering methods set process noise as a fixed constant, resulting in reduced estimation accuracy and poor convergence. Current vision-dynamics fusion estimation methods typically employ offline data fusion, failing to leverage the combined strengths of both approaches and failing to achieve real-time estimation of the road adhesion coefficient. Summary of the Invention
[0004] The purpose of the present invention is to provide a comprehensive estimation method for road adhesion coefficient that combines visual prediction with tactile verification. Its technical purpose is to solve the problems of slow response speed, poor stability and low accuracy of existing road adhesion coefficient estimation methods.
[0005] The technical solution to achieve the purpose of the present invention is: a comprehensive estimation method of road adhesion coefficient combining visual prediction and tactile verification, comprising the following steps:
[0006] S1: The vehicle-mounted camera obtains the road ahead information in real time, and then uses the visual method to obtain the road adhesion coefficient prediction range;
[0007] S2: Establish nonlinear four-wheel vehicle dynamics model and tire model;
[0008] S3: Based on the cubature Kalman filter, orthogonal triangular decomposition is introduced, the covariance matrix is processed in square root form, and the square root cubature Kalman filter (SRCKF) is constructed. Considering the time-varying process noise, the residual is generated based on the actual measurement value and the predicted measurement value, and the residual is compared with the preset threshold. The process noise covariance update rule is designed to construct the adaptive square root cubature Kalman filter (ASRCKF).
[0009] S4: The road adhesion coefficient prediction range obtained in S1 is introduced into the adaptive square root cubature Kalman filter ASRCKF as the initial value of the target state vector to estimate the road adhesion coefficient;
[0010] S5: Feedback the road image corresponding to the estimation result of S4 to S1. The visual method evolves automatically, realizing the online combination of visual prediction and tactile verification methods, and accurately estimating the road adhesion coefficient in real time.
[0011] Furthermore, step S1 is specifically as follows:
[0012] The on-board camera acquires images including road ahead information in real time, labels the images using Labelimg to form a data set, and uses the data set to train the YOLOv5 deep learning algorithm. The trained YOLOv5 deep learning algorithm then identifies and classifies the road ahead information acquired by the camera, and estimates the predicted range of the road adhesion coefficient.
[0013] Furthermore, the nonlinear four-wheel vehicle dynamics model established in step S2 is expressed as:
[0014]
[0015] Among them, a x , a y are the lateral acceleration and longitudinal acceleration of the vehicle, v m is the vehicle speed, m is the vehicle mass, δ is the front wheel turning angle, ω is the yaw angle, β is the sideslip angle, a and b are the front and rear wheelbases respectively, I z is the yaw moment of inertia, T f is the front axle width, T r is the rear axle width, They are the lateral force and longitudinal force of the left front wheel, right front wheel, left rear wheel and right rear wheel of the vehicle respectively;
[0016] The tire model established in step S2 is expressed as:
[0017] Y=Dsin[Carctan{Bx-E(Bx-arctan(Bx)}]#(5)
[0018]
[0019] Where Y is the tire longitudinal force or lateral force, B is the stiffness factor, C is the shape factor, D is the peak factor, E is the curvature factor, and x is the sideslip angle α or longitudinal slip λ of each wheel.
[0020] Furthermore, step S3 specifically includes:
[0021] S31: Discretize the vehicle dynamics model and tire model and express them as:
[0022]
[0023] Where x is the state vector, z is the measurement vector, u is the input vector, f(*) is the state transfer function, h(*) is the output function, w is the process noise, and v is the measurement noise;
[0024] S32: Time update, expressed as:
[0025] Build volume points:
[0026]
[0027] Where i = 1, 2, ... m, m = 2n, m is the number of volume points, and n is the dimension of the state vector x;
[0028] Weight:
[0029]
[0030] Construct a volume point set:
[0031]
[0032] Compute the propagated volume points:
[0033]
[0034] Orthogonal triangular decomposition is introduced to calculate the square root factor of the state prediction and prediction error covariance matrix, which is expressed as:
[0035]
[0036]
[0037] Among them, Tria(*) represents the orthogonal triangular decomposition of the matrix, and obtains an upper triangular matrix. represents a weight matrix;
[0038] S33: Measurement update, expressed as:
[0039] Update volume point set:
[0040]
[0041] Compute the propagated volume points:
[0042]
[0043] Compute the square root factor of the measurement forecast and innovation covariance matrix:
[0044]
[0045] Compute the innovation covariance matrix and the measurement autocovariance matrix:
[0046]
[0047] P z,t+1|t =P zz,t+1|t -R t+1 #(twenty two)
[0048] Compute the cross-correlation covariance matrix:
[0049]
[0050] Calculate the Kalman gain:
[0051]
[0052] State Estimation:
[0053]
[0054] Compute the square root factor of the estimated error covariance matrix:
[0055]
[0056] S34: Generate residuals based on the difference between the actual measured value and its estimated value, and design the process noise covariance adaptive rule, which is expressed as:
[0057]
[0058] Among them, the parameters ρ>0, δ>0, when When it is larger or smaller, the corresponding increase is achieved by adjusting ρ and δ. or reduce To reduce the estimation error, improve the estimation accuracy and convergence speed of the filtering algorithm; for the selection of ρ, according to the residual Will Compare with the preset threshold θ and then adjust ρ; the specific rules are as follows:
[0059]
[0060] Furthermore, the preset threshold θ is set to 0.075.
[0061] Furthermore, in step S5, the road image corresponding to the estimation result annotation is added to the YOLOv5 training set, and the data set is expanded online and the deep learning method is trained online to achieve algorithm self-evolution, narrow the prediction range of the visual prediction method, provide more accurate prediction initial values for the subsequent filtering algorithm, and realize the comprehensive estimation of the road adhesion coefficient by the visual method and the tactile verification method.
[0062] Compared with the prior art, the present invention has the following significant advantages:
[0063] In order to improve the convergence speed of the filtering algorithm, this application introduces the visual prediction results into the filter as the initial value of the state vector; in the filtering process, orthogonal triangular decomposition is introduced to replace the complex calculations of the ordinary cubature Kalman filter on the matrix square root, inversion and transposition, thereby avoiding the positive definiteness and symmetry destruction of the covariance matrix and improving the stability of the algorithm. In order to further improve the estimation accuracy of the algorithm, a process noise covariance matrix adaptive change rule is designed to adapt to the time-varying external environment. Through the comprehensive estimation method of the road adhesion coefficient combining visual prediction with tactile verification, the convergence speed, stability and estimation accuracy of the algorithm are improved; at the same time, the corresponding road image is annotated with the estimation result and added to the YOLOv5 training set, and the data set is expanded online and the YOLOv5 deep learning method is trained online to realize the self-evolution of the algorithm and improve the prediction accuracy of the visual method, thereby realizing the comprehensive estimation of the road adhesion coefficient combining visual prediction with tactile verification. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] Figure 1 A flow chart of the comprehensive estimation method of road adhesion coefficient for this application;
[0065] Figure 2 Schematic diagram of the adaptive square root cubature Kalman filter of this application;
[0066] Figure 3(a) is a comparison diagram of the estimation results obtained by the method of the present application under the condition of fixed road adhesion coefficient and the results of other estimation methods, and Figure 3(b) is a comparison diagram of the estimation results obtained by the method of the present application under the condition of sudden change of road adhesion coefficient and the results of other estimation methods. DETAILED DESCRIPTION
[0067] The present invention is further described in detail below with reference to the accompanying drawings.
[0068] Figure 1This is a flow chart of the comprehensive estimation method of the road adhesion coefficient described in this application, as shown in Figure 1 As shown in FIG, the comprehensive estimation method of the road adhesion coefficient based on the combination of visual prediction and tactile verification includes:
[0069] S1: The vehicle-mounted camera obtains the road ahead information in real time, and then uses the visual method to obtain the road adhesion coefficient prediction range;
[0070] S2: Establish nonlinear four-wheel vehicle dynamics model and tire model;
[0071] S3: Based on the cubature Kalman filter, we introduce orthogonal triangular decomposition and use the square root form to process the covariance matrix, thus constructing the square root cubature Kalman filter (SRCKF). Taking into account the time-varying process noise, we generate residuals based on the actual and predicted measurements, compare the residuals with a preset threshold, and design a process noise covariance update rule to construct the adaptive square root cubature Kalman filter (ASRCKF).
[0072] S4: The predicted range of the front road adhesion coefficient obtained in S1 is introduced into the filter as the initial value of the target state vector to improve the convergence speed of the filtering method and estimate the road adhesion coefficient;
[0073] S5: Add the road images corresponding to the estimation results to the YOLOv5 training set, perform online expansion of the data set and online training of the YOLOv5 deep learning method, realize algorithm self-evolution, improve the accuracy of the visual prediction method, realize the online combination of visual prediction and tactile verification methods, and accurately estimate the road adhesion coefficient in real time.
[0074] In step S1, the range of the road adhesion coefficient ahead is predicted using a visual method:
[0075] The road surface is classified into dry cement road surface, wet cement road surface, dry asphalt road surface, wet asphalt road surface, dry gravel road surface, wet gravel road surface, and icy and snowy road surface. The images are annotated using Labelimg, and the annotated dataset is used to train the YOLOv5 deep learning algorithm. The trained YOLOv5 deep learning algorithm then identifies and classifies the road surface information obtained by the camera and estimates the predicted range of the road adhesion coefficient. The deep learning is based on the Pytorch framework.
[0076] The YOLOv5 target detection algorithm model structure includes five components: feature extraction network (Backbone), feature fusion part (Neck), detection head (Head), loss function (Loss Function) and anchor box (Anchor Boxes). The Backbone layer consists of Focus structure and CSP structure. The Focus structure is used to compress and combine the feature information in the input image to extract higher-level feature representations. The CSP structure is used to improve feature extraction efficiency. The Neck network refers to the intermediate feature extraction network added on the basis of the backbone network. It is mainly used to enhance the feature expression ability of the model and further improve the detection performance of the model. The loss function adopts the Intersection over Union loss function, which is mainly used to measure the difference between the predicted bounding box and the real bounding box. It is expressed as:
[0077]
[0078] where b, b gt Represents the center point of the predicted box and the real box, p 2 (b,b gt ) represents the Euclidean distance between the two center points, c represents the diagonal distance between the minimum enclosed area of the predicted box and the true box. a is a weight parameter, and v is used to measure the consistency of the aspect ratio.
[0079] In step S2, the nonlinear four-wheel vehicle dynamics model and tire model are established as follows:
[0080]
[0081] Among them, a x , a y are the lateral acceleration and longitudinal acceleration of the vehicle, v m is the vehicle speed, m is the vehicle mass, δ is the front wheel turning angle, ω is the yaw angle, β is the sideslip angle, a and b are the front and rear wheelbases respectively, I z is the yaw moment of inertia, T f is the front axle width, T r is the rear axle width, They are the lateral and longitudinal forces on the left front wheel, right front wheel, left rear wheel and right rear wheel of the vehicle respectively.
[0082] Y=D sin[Carctan{Bx-E(Bx-arctan(Bx)}]#(9)
[0083] Where Y is the tire longitudinal force or lateral force, B is the stiffness factor, C is the shape factor, D is the peak factor, E is the curvature factor, and x is the sideslip angle α or longitudinal slip λ of each wheel, expressed as follows:
[0084]
[0085] Where v xij is the wheel speed, v m is the vehicle speed, T f is the front axle width, Tr 为 The width of the rear axle, a and b are the front and rear wheelbases respectively, ω is the yaw angle, β is the sideslip angle, is the wheel radius, and w is the wheel angular velocity.
[0086] In step S3, the dynamic model and tire model established in S2 are discretized, and the cubature Kalman filter is improved by introducing orthogonal triangular decomposition. At the same time, a process noise covariance update rule is designed:
[0087] S31 model discretization and filter initialization are expressed as:
[0088]
[0089] Where x is the state vector, z is the measurement vector, u is the input vector, f(*) is the state transfer function, h(*) is the output function, w is the process noise, and v is the measurement noise.
[0090]
[0091] S32 time update, expressed as:
[0092] Build volume points:
[0093]
[0094] Where i = 1, 2, ... m, m = 2n, m is the number of volume points, and n is the dimension of the state vector x.
[0095] Weight:
[0096]
[0097] Construct a volume point set:
[0098]
[0099] Compute the propagated volume points:
[0100]
[0101] Compute the square root factor of the state prediction and prediction error covariance matrix:
[0102]
[0103] Tria(*) represents the orthogonal triangular decomposition of the matrix, and obtains an upper triangular matrix. This replaces the complex calculations such as matrix square root, inversion, and transposition in traditional cubature Kalman filtering, avoiding the problem of matrix positive definiteness and symmetry being destroyed due to complex calculations, thereby improving the stability of the algorithm. Represents a weighting matrix.
[0104] S33 measurement update, expressed as:
[0105] Update volume point set:
[0106]
[0107] Compute the propagated volume points:
[0108]
[0109] Compute the square root factor of the measurement forecast and innovation covariance matrix:
[0110]
[0111] Compute the innovation covariance matrix and the measurement autocovariance matrix:
[0112]
[0113] P z,t+1|t =P zz,t+1|t -R t+1 #(37)
[0114] Compute the cross-correlation covariance matrix:
[0115]
[0116] Calculate the Kalman gain:
[0117]
[0118] State Estimation:
[0119]
[0120] Compute the square root factor of the estimated error covariance matrix:
[0121]
[0122] S34 generates a residual based on the difference between the actual measured value and its estimated value, and realizes the adaptive update of the process noise covariance, which is expressed as:
[0123]
[0124] Among the parameters ρ>0,δ>0, when When it is larger or smaller, the corresponding increase is achieved by adjusting ρ and δ. or reduce To reduce the estimation error and improve the estimation accuracy and convergence speed of the filtering algorithm. For δ, select the smallest value possible, and for ρ, select the value according to the residual Will Compare with the preset threshold θ and then adjust ρ: the specific rules are as follows:
[0125]
[0126] In step S4, the predicted range of the front road adhesion coefficient obtained by the visual method is introduced into the filter as the initial value of the target state quantity. The road adhesion coefficient is estimated by the established ASRCKF filter. Compared with the traditional filtering method, the convergence speed of the algorithm is improved in the case of sudden changes in the road adhesion coefficient.
[0127] In step S5, the road image corresponding to the estimation result annotation is added to the YOLOv5 training set, and the data set is expanded online and the YOLOv5 deep learning method is trained online to achieve algorithm self-evolution, narrow the prediction range of the visual method, provide more accurate prediction initial values for the subsequent filtering algorithm, and realize the online combination of visual prediction and tactile verification methods to accurately estimate the road adhesion coefficient in real time.
[0128] The following simulation test verifies the comprehensive estimation method for the road adhesion coefficient of the above embodiment. To verify the superiority of this method, two sets of different road conditions were set for simulation. Under the double lane change condition with a fixed vehicle speed of 72 km / h, the driving time was first set to 8 seconds, and the road was fixed to an asphalt road with a road adhesion coefficient of 0.8. The estimation results were compared with other methods, as shown in Figure 3(a). Then, the driving time was set to 10 seconds, and the asphalt road with a road adhesion coefficient of 0.8 suddenly changed to a slippery road with a road adhesion coefficient of 0.3 at 50 meters, and then suddenly changed to 0.4 at 160 meters. The estimation results were compared with other methods, as shown in Figure 3(b). The simulation result curves show that the comprehensive estimation method for the road adhesion coefficient based on the combination of visual prediction and tactile verification proposed in this paper has faster response speed, higher estimation accuracy, and better stability and convergence than other methods.
[0129] This application uses a combination of visual prediction and tactile verification to comprehensively estimate the road adhesion coefficient. Compared to other methods, this improved vision and tactile fusion strategy improves the algorithm's response speed, estimation accuracy, and stability. This plays a significant role in achieving active vehicle safety control and improving driving safety and comfort.
[0130] The above are exemplary embodiments of the present application, and the scope of protection of the present application is defined by the claims and their equivalents. Those skilled in the art should understand that the present application is not limited by the above examples. The above examples and descriptions are merely illustrative of the principles of the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention, and such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.
Claims
1. A comprehensive estimation method for road adhesion coefficient combining visual prediction and tactile verification, characterized in that: The steps include: S1: The vehicle-mounted camera obtains the road ahead information in real time, and then uses the visual method to obtain the road adhesion coefficient prediction range; S2: Establish nonlinear four-wheel vehicle dynamics model and tire model; S3: Based on the cubature Kalman filter, orthogonal triangular decomposition is introduced, the covariance matrix is processed in square root form, and the square root cubature Kalman filter (SRCKF) is constructed. Considering the time-varying process noise, the residual is generated based on the actual measurement value and the predicted measurement value, and the residual is compared with the preset threshold. The process noise covariance update rule is designed to construct the adaptive square root cubature Kalman filter (ASRCKF). S4: The road adhesion coefficient prediction range obtained in S1 is introduced into the adaptive square root cubature Kalman filter ASRCKF as the initial value of the target state vector to estimate the road adhesion coefficient; S5: Feeds the road image corresponding to the estimation result of S4 back to S1. The visual method evolves automatically, realizing the online combination of visual prediction and tactile verification methods, and accurately estimating the road adhesion coefficient in real time. Step S3 specifically includes: S31: Discretize the vehicle dynamics model and tire model and express them as: Where x is the state vector, z is the measurement vector, u is the input vector, f(*) is the state transfer function, h(*) is the output function, w is the process noise, and v is the measurement noise; S32: Time update, expressed as: Build volume points: Where i = 1, 2, ... m, m = 2n, m is the number of volume points, and n is the dimension of the state vector x; Weight: Construct a volume point set: Compute the propagated volume points: Orthogonal triangular decomposition is introduced to calculate the square root factor of the state prediction and prediction error covariance matrix, which is expressed as: Among them, Tria(*) represents the orthogonal triangular decomposition of the matrix, and obtains an upper triangular matrix. represents a weight matrix; S33: Measurement update, expressed as: Update volume point set: Compute the propagated volume points: Compute the square root factor of the measurement forecast and innovation covariance matrix: Compute the innovation covariance matrix and the measurement autocovariance matrix: P z,t+1|t =P zz,t+1|t -R t+1 #(22) Compute the cross-correlation covariance matrix: Calculate the Kalman gain: State Estimation: Compute the square root factor of the estimated error covariance matrix: S34: Generate residuals based on the difference between the actual measured value and its estimated value, and design the process noise covariance adaptive rule, which is expressed as: Among them, the parameters ρ>0, δ>0, when When it is larger or smaller, the corresponding increase is achieved by adjusting ρ and δ. or reduce To reduce the estimation error, improve the estimation accuracy and convergence speed of the filtering algorithm; for the selection of ρ, according to the residual Will Compare with the preset threshold θ and then adjust ρ; the specific rules are as follows:
2. The method according to claim 1, characterized in that Step S1 is specifically as follows: The on-board camera acquires images including road ahead information in real time, labels the images using Labelimg to form a data set, and uses the data set to train the YOLOv5 deep learning algorithm. The trained YOLOv5 deep learning algorithm then identifies and classifies the road ahead information acquired by the camera, and estimates the predicted range of the road adhesion coefficient.
3. The method according to claim 2, characterized in that The nonlinear four-wheel vehicle dynamics model established in step S2 is expressed as: Among them, a x , a y are the lateral acceleration and longitudinal acceleration of the vehicle, v m is the vehicle speed, m is the vehicle mass, δ is the front wheel turning angle, ω is the yaw angle, β is the sideslip angle, a and b are the front and rear wheelbases respectively, I z is the yaw moment of inertia, T f is the front axle width, T r is the rear axle width, They are the lateral force and longitudinal force of the left front wheel, right front wheel, left rear wheel and right rear wheel of the vehicle respectively; The tire model established in step S2 is expressed as: Y=D sin[C arctan{Bx-E(Bx-arctan(Bx)}]#(5) Where Y is the tire longitudinal force or lateral force, B is the stiffness factor, C is the shape factor, D is the peak factor, E is the curvature factor, and x is the sideslip angle α or longitudinal slip λ of each wheel.
4. The method according to claim 3, characterized in that The preset threshold θ is set to 0.
075.
5. The method according to claim 3, characterized in that In step S5, the road image corresponding to the estimation result annotation is added to the YOLOv5 training set, and the data set is expanded online and the deep learning method is trained online to achieve algorithm self-evolution, narrow the prediction range of the visual prediction method, provide more accurate prediction initial values for the subsequent filtering algorithm, and realize the comprehensive estimation of the road adhesion coefficient by the visual method and the tactile verification method.