Visual prediction and tactile verification combined road adhesion coefficient comprehensive estimation method

Abstract

The invention relates to a road adhesion coefficient comprehensive estimation method combining visual prediction and tactile verification. Comprising the following steps: S1, acquiring front road information in real time, and obtaining a road adhesion coefficient prediction range based on a visual method; s2, establishing a nonlinear four-wheel vehicle dynamic model and a tire model; s3, introducing orthogonal triangular decomposition, and constructing SRCKF (square root cubature Kalman filter); time-varying process noise is considered, and an adaptive square root cubature Kalman filter ASRCKF is constructed; s4, using the road adhesion coefficient prediction range as an initial value of a filter target state vector, and estimating a road adhesion coefficient; and S5, feeding back the road image corresponding to the estimation result label in the step S4 to the step S1. According to the invention, a visual prediction result is used as an initial value of a filtering algorithm, a tactile verification result is fed back to a visual method training process online, and the convergence speed, stability and prediction precision of the algorithm are improved; the designed adaptive updating rule adjusts the process noise covariance matrix in real time, and the stability and precision of the filtering algorithm are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic driving control, and specifically relates to a comprehensive estimation method of road adhesion coefficient combining visual prediction with tactile verification. Background Art

[0002] The interaction between vehicles and roads is an important factor affecting traffic safety. The safety of traditional vehicles depends on the interaction between the driver's perception and prediction. With the development of automotive electronic technology and intelligent technology, intelligent vehicle motion control systems have assumed a more important responsibility in vehicle active safety control. As a key parameter affecting the effect of active safety control, the road adhesion coefficient cannot be directly measured by on-board sensors. In current research, the cause-based method or the effect-based method is generally used for indirect measurement.

[0003] The cause-based method estimates the road adhesion coefficient by installing special types of sensors on the vehicle or tire to measure various parameters related to the adhesion coefficient. This method requires the installation of professional equipment, is costly, and is easily affected by external factors and has poor robustness. The effect-based method measures the vehicle dynamic response signal caused by the adhesion coefficient through common on-board sensors, establishes a vehicle dynamics model, and estimates the road adhesion coefficient by combining a filtering algorithm. The filtering algorithms currently used for the road adhesion coefficient include extended Kalman filtering, unscented Kalman filtering, and cubature Kalman filtering. Among them, cubature Kalman filtering uses the ball-radial rule for approximate estimation and has better numerical stability. However, in the filtering process, there are complex calculations such as square rooting, inversion, and transposition of the matrix, which easily lead to the destruction of the symmetry and positive definiteness of the covariance matrix, affecting the estimation accuracy. At the same time, the car changes dynamically during actual driving. The ordinary filtering method sets the process noise as a fixed constant, which will lead to reduced estimation accuracy and poor convergence. The current estimation method of vision and dynamics fusion is usually offline data fusion, which fails to give full play to the combined advantages of the two methods and cannot achieve real-time estimation of the road adhesion coefficient. Summary of the invention

[0004] The purpose of the present invention is to provide a comprehensive estimation method for road adhesion coefficient combining visual prediction with tactile verification, and its technical purpose is to solve the problems of slow response speed, poor stability and low accuracy of existing road adhesion coefficient estimation methods.

[0005] The technical solution to achieve the purpose of the present invention is: a comprehensive estimation method of road adhesion coefficient combining visual prediction and tactile verification, comprising the following steps:

[0006] S1: The vehicle-mounted camera obtains the road information in real time, and then obtains the road adhesion coefficient prediction range based on the visual method;

[0007] S2: Establish nonlinear four-wheel vehicle dynamics model and tire model;

[0008] S3: Based on the cubature Kalman filter, orthogonal triangular decomposition is introduced, the covariance matrix is ​​processed in square root form, and the square root cubature Kalman filter SRCKF is constructed; considering the time-varying process noise, the residual is generated according to the actual measurement value and the predicted measurement value, the residual is compared with the preset threshold, the process noise covariance update rule is designed, and the adaptive square root cubature Kalman filter ASRCKF is constructed;

[0009] S4: The predicted range of the front road adhesion coefficient obtained in S1 is introduced into the adaptive square root volumetric Kalman filter ASRCKF as the initial value of the target state vector to estimate the road adhesion coefficient;

[0010] S5: Feedback the road image corresponding to the estimation result of S4 to S1, and the visual method will evolve automatically to realize the online combination of visual prediction and tactile verification methods to accurately estimate the road adhesion coefficient in real time.

[0011] Furthermore, step S1 is specifically as follows:

[0012] The on-board camera acquires images including road information ahead in real time, uses Labelimg to annotate the images and form a data set, and uses the data set to train the YOLOv5 deep learning algorithm. The trained YOLOv5 deep learning algorithm then identifies and classifies the road surface information ahead acquired by the camera, and estimates the predicted range of the road adhesion coefficient.

[0013] Furthermore, the nonlinear four-wheel vehicle dynamics model established in step S2 is expressed as:

[0014]

[0015] Among them, a x , a y are the lateral acceleration and longitudinal acceleration of the vehicle, v m is the vehicle speed, m is the vehicle mass, δ is the front wheel turning angle, ω is the yaw angle, β is the sideslip angle, a and b are the front and rear wheelbases respectively, I z is the yaw moment of inertia, T f is the front axle width, T r is the rear axle width, They are the lateral force and longitudinal force of the left front wheel, right front wheel, left rear wheel and right rear wheel of the vehicle respectively;

[0016] The tire model established in step S2 is expressed as:

[0017] Y=Dsin[Carctan{Bx-E(Bx-arctan(Bx)}]#(5)

[0018]

[0019] Among them, Y is the tire longitudinal force or lateral force, B is the stiffness factor, C is the shape factor, D is the peak factor, E is the curvature factor, and x is the sideslip angle α or longitudinal slip λ of each wheel.

[0020] Furthermore, step S3 specifically includes:

[0021] S31: Discretize the vehicle dynamics model and tire model and express them as:

[0022]

[0023] Where x is the state vector, z is the measurement vector, u is the input vector, f(*) is the state transfer function, h(*) is the output function, w is the process noise, and v is the measurement noise;

[0024] S32: Time update, expressed as:

[0025] Build volume points:

[0026]

[0027] Where i = 1, 2, ... m, m = 2n, m is the number of volume points, and n is the dimension of the state vector x;

[0028] Weight:

[0029]

[0030] Construct a volume point set:

[0031]

[0032] Compute the propagated volume points:

[0033]

[0034] Orthogonal triangular decomposition is introduced to calculate the square root factor of the state prediction and prediction error covariance matrix, which is expressed as:

[0035]

[0036]

[0037] Among them, Tria(*) represents the orthogonal triangular decomposition of the matrix, and obtains an upper triangular matrix, represents a weighting matrix;

[0038] S33: Measurement update, expressed as:

[0039] Update volume point set:

[0040]

[0041] Compute the propagated volume points:

[0042]

[0043] Compute the square root factor of the measurement prediction and innovation covariance matrix:

[0044]

[0045] Compute the innovation covariance matrix and the measurement autocovariance matrix:

[0046]

[0047] P z,t+1|t =P zz,t+1|t -R t+1 #(twenty two)

[0048] Compute the cross-correlation covariance matrix:

[0049]

[0050] Calculate the Kalman gain:

[0051]

[0052] State Estimation:

[0053]

[0054] Compute the square root factor of the estimated error covariance matrix:

[0055]

[0056] S34: Generate residuals based on the difference between the actual measured value and its estimated value, and design the process noise covariance adaptive rule, which is expressed as:

[0057]

[0058] Among them, the parameters ρ>0,δ>0, when When it is larger or smaller, it can be increased by adjusting ρ and δ or reduce To reduce the estimation error, improve the estimation accuracy and convergence speed of the filtering algorithm; for the selection of ρ, according to the residual Will Compare with the preset threshold θ, and then adjust ρ; the specific rules are as follows:

[0059]

[0060] Furthermore, the preset threshold θ is set to 0.075.

[0061] Furthermore, in step S5, the road image corresponding to the estimation result annotation is added to the YOLOv5 training set, and the data set is expanded online and the deep learning method is trained online to realize algorithm self-evolution, narrow the prediction range of the visual prediction method, provide a more accurate prediction initial value for the subsequent filtering algorithm, and realize the comprehensive estimation of the road adhesion coefficient by the visual method and the tactile verification method.

[0062] Compared with the prior art, the present invention has the following significant advantages:

[0063] In order to improve the convergence speed of the filtering algorithm, the present application introduces the visual prediction result into the filter as the initial value of the state vector; in the filtering process, orthogonal triangular decomposition is introduced to replace the complex calculations of the ordinary volumetric Kalman filter such as matrix square root, inversion and transposition, thereby avoiding the positive definiteness and symmetry destruction of the covariance matrix and improving the stability of the algorithm. In order to further improve the estimation accuracy of the algorithm, a process noise covariance matrix adaptive change rule is designed to adapt to the time-varying external environment. The road adhesion coefficient comprehensive estimation method combining visual prediction with tactile verification is used to improve the convergence speed, stability and estimation accuracy of the algorithm; at the same time, the corresponding road image is annotated with the estimation result and added to the YOLOv5 training set, and the data set is expanded online and the YOLOv5 deep learning method is trained online to realize the self-evolution of the algorithm and improve the prediction accuracy of the visual method, thereby realizing the comprehensive estimation of the road adhesion coefficient combining visual prediction with tactile verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A flow chart of the comprehensive estimation method of the road adhesion coefficient for this application;

[0065] Figure 2 A schematic diagram of the adaptive square root cubature Kalman filter of this application;

[0066] FIG3(a) is a comparison diagram of the estimation results obtained by the method of the present application under the condition of fixed road adhesion coefficient and the results of other estimation methods, and FIG3(b) is a comparison diagram of the estimation results obtained by the method of the present application under the condition of sudden change of road adhesion coefficient and the results of other estimation methods. DETAILED DESCRIPTION

[0067] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0068] Figure 1This is a flow chart of the comprehensive estimation method of the road adhesion coefficient described in this application, such as Figure 1 As shown in FIG. 1 , the comprehensive estimation method of the road adhesion coefficient based on the combination of visual prediction and tactile verification includes:

[0069] S1: The vehicle-mounted camera obtains the road information in real time, and then obtains the road adhesion coefficient prediction range based on the visual method;

[0070] S2: Establish nonlinear four-wheel vehicle dynamics model and tire model;

[0071] S3: Based on the cubature Kalman filter, orthogonal triangular decomposition is introduced, and the covariance matrix is ​​processed in square root form to construct the square root cubature Kalman filter (SRCKF); considering the time-varying process noise, the residual is generated according to the actual measurement value and the predicted measurement value, and the residual is compared with the preset threshold. A process noise covariance update rule is designed to construct an adaptive square root cubature Kalman filter (ASRCKF);

[0072] S4: introducing the predicted range of the front road adhesion coefficient obtained in S1 into the filter as the initial value of the target state vector, improving the convergence speed of the filtering method, and estimating the road adhesion coefficient;

[0073] S5: Add the road images corresponding to the estimation results to the YOLOv5 training set, perform online expansion of the data set and online training of the YOLOv5 deep learning method, realize algorithm self-evolution, improve the accuracy of the visual prediction method, realize the online combination of visual prediction and tactile verification methods, and accurately estimate the road adhesion coefficient in real time.

[0074] In step S1, the range of the adhesion coefficient of the road ahead is predicted using a visual method:

[0075] The road surface is classified into dry cement road surface, wet cement road surface, dry asphalt road surface, wet asphalt road surface, dry gravel road surface, wet gravel road surface and ice and snow road surface, etc. The images are annotated using Labelimg, and the annotated data set is used to train the YOLOv5 deep learning algorithm. The trained YOLOv5 deep learning algorithm then recognizes and classifies the road surface information ahead obtained by the camera, and estimates the prediction range of the road adhesion coefficient. The deep learning is based on the Pytorch framework.

[0076] The YOLOv5 target detection algorithm model structure includes five components: feature extraction network (Backbone), feature fusion part (Neck), detection head (Head), loss function (Loss Function) and anchor box (Anchor Boxes). The Backbone layer consists of Focus structure and CSP structure. The Focus structure is used to compress and combine the feature information in the input image to extract a higher level of feature representation. The CSP structure is used to improve the efficiency of feature extraction. The Neck network refers to the intermediate feature extraction network added on the basis of the backbone network. It is mainly used to enhance the feature expression ability of the model and further improve the detection performance of the model. The loss function uses the Intersection over Union loss function, which is mainly used to measure the difference between the predicted bounding box and the real bounding box. It is expressed as:

[0077]

[0078] where b, b gt Represents the center point of the predicted box and the true box, p 2 (b,b gt ) represents the Euclidean distance between two center points, c represents the diagonal distance of the minimum closure area between the predicted box and the real box. a is the weight parameter, and v is used to measure the consistency of the aspect ratio.

[0079] In step S2, the nonlinear four-wheel vehicle dynamics model and tire model are established as follows:

[0080]

[0081] Among them, a x , a y are the lateral acceleration and longitudinal acceleration of the vehicle, v m is the vehicle speed, m is the vehicle mass, δ is the front wheel turning angle, ω is the yaw angle, β is the sideslip angle, a and b are the front and rear wheelbases respectively, I z is the yaw moment of inertia, T f is the front axle width, T r is the rear axle width, They are the lateral and longitudinal forces of the vehicle's left front wheel, right front wheel, left rear wheel, and right rear wheel.

[0082] Y=D sin[Carctan{Bx-E(Bx-arctan(Bx)}]#(9)

[0083] Among them, Y is the tire longitudinal force or lateral force, B is the stiffness factor, C is the shape factor, D is the peak factor, E is the curvature factor, and x is the sideslip angle α or longitudinal slip λ of each wheel, which is expressed as follows:

[0084]

[0085] Where v xij is the wheel speed, v m is the vehicle speed, T f is the front axle width, Tr 为 The width of the rear axle, a and b are the front and rear wheelbases, ω is the yaw angle, β is the sideslip angle, is the wheel radius, and w is the wheel angular velocity.

[0086] In step S3, the dynamic model and tire model established in S2 are discretized, and the volumetric Kalman filter is improved by introducing orthogonal triangular decomposition. At the same time, a process noise covariance update rule is designed:

[0087] S31 model discretization and filter initialization are expressed as:

[0088]

[0089] Where x is the state vector, z is the measurement vector, u is the input vector, f(*) is the state transfer function, h(*) is the output function, w is the process noise, and v is the measurement noise.

[0090]

[0091] S32 time update, expressed as:

[0092] Build volume points:

[0093]

[0094] Among them, i=1,2,…m,m=2n, m is the number of volume points, and n is the dimension of the state vector x.

[0095] Weight:

[0096]

[0097] Construct a volume point set:

[0098]

[0099] Compute the propagated volume points:

[0100]

[0101] Compute the square root factor of the state prediction and prediction error covariance matrix:

[0102]

[0103] Tria(*) represents the orthogonal triangular decomposition of the matrix, and an upper triangular matrix is ​​obtained, which replaces the complex calculations of matrix square root, inversion and transposition in the traditional cubature Kalman filter, avoids the problem of matrix positive definiteness and symmetry being destroyed due to complex calculations, and improves the stability of the algorithm. Represents a weighting matrix.

[0104] S33 measurement update, expressed as:

[0105] Update volume point set:

[0106]

[0107] Compute the propagated volume points:

[0108]

[0109] Compute the square root factor of the measurement prediction and innovation covariance matrix:

[0110]

[0111] Compute the innovation covariance matrix and the measurement autocovariance matrix:

[0112]

[0113] P z,t+1|t =P zz,t+1|t -R t+1 #(37)

[0114] Compute the cross-correlation covariance matrix:

[0115]

[0116] Calculate the Kalman gain:

[0117]

[0118] State Estimation:

[0119]

[0120] Compute the square root factor of the estimated error covariance matrix:

[0121]

[0122] S34 generates a residual based on the difference between the actual measured value and its estimated value, and realizes the adaptive update of the process noise covariance, which is expressed as:

[0123]

[0124] Among the parameters ρ>0,δ>0, when When it is larger or smaller, it can be increased by adjusting ρ and δ or reduce To reduce the estimation error and improve the estimation accuracy and convergence speed of the filtering algorithm. For δ, select the smallest value possible, and for ρ, select the value according to the residual Will Compare with the preset threshold θ, and then adjust ρ: The specific rules are as follows:

[0125]

[0126] In step S4, the predicted range of the front road adhesion coefficient obtained by the visual method is introduced into the filter as the initial value of the target state quantity, and the road adhesion coefficient is estimated by the established ASRCKF filter. Compared with the traditional filtering method, the convergence speed of the algorithm is improved in the case of a sudden change in the road adhesion coefficient.

[0127] In step S5, the road image corresponding to the estimation result annotation is added to the YOLOv5 training set, and the data set is expanded online and the YOLOv5 deep learning method is trained online to realize algorithm self-evolution, narrow the prediction range of the visual method, provide more accurate prediction initial values ​​for the subsequent filtering algorithm, realize the online combination of visual prediction and tactile verification method, and estimate the road adhesion coefficient accurately in real time.

[0128] The following simulation test verifies the comprehensive estimation method of the road adhesion coefficient of the above embodiment. In order to verify the superiority of this method, two groups of different road conditions are set for simulation. Under the double lane change condition with a fixed speed of 72km / h, the driving time is first set to 8 seconds, and the road is fixed to an asphalt road with a road adhesion coefficient of 0.8. The estimation results are compared with other methods, as shown in Figure 3(a); then the driving time is set to 10 seconds, and the asphalt road with a road adhesion coefficient of 0.8 suddenly changes to a slippery road with a road adhesion coefficient of 0.3 at 50m, and suddenly changes to 0.4 at 160m. The estimation results are compared with other methods, as shown in Figure 3(b). It can be seen from the simulation result curve that the comprehensive estimation method of road adhesion coefficient based on the combination of visual prediction and tactile verification proposed in this paper has faster response speed, higher estimation accuracy, better stability and convergence than other methods.

[0129] This application uses a combination of visual prediction and tactile verification to comprehensively estimate the road adhesion coefficient. Compared with other methods, it improves the fusion strategy of visual and tactile methods, improves the algorithm response speed, estimation accuracy and stability. It plays an important role in realizing active vehicle safety control and improving vehicle driving safety and driving comfort.

[0130] The above are exemplary embodiments of the present application, and the protection scope of the present application is defined by the claims and their equivalents. Those skilled in the art should understand that the present application is not limited by the above examples, and the above examples and descriptions are only for illustrating the principles of the present invention. The present invention may have various changes and improvements without departing from the spirit and scope of the present invention, and these changes and improvements fall within the scope of the present invention to be protected. The protection scope of the present invention is defined by the attached claims and their equivalents.

Claims

1. A comprehensive estimation method of road adhesion coefficient combining visual prediction and tactile verification, characterized in that: The steps include: S1: The vehicle-mounted camera obtains the road information in real time, and then obtains the road adhesion coefficient prediction range based on the visual method; S2: Establish nonlinear four-wheel vehicle dynamics model and tire model; S3: Based on the cubature Kalman filter, orthogonal triangular decomposition is introduced, the covariance matrix is ​​processed in square root form, and the square root cubature Kalman filter SRCKF is constructed; considering the time-varying process noise, the residual is generated according to the actual measurement value and the predicted measurement value, the residual is compared with the preset threshold, the process noise covariance update rule is designed, and the adaptive square root cubature Kalman filter ASRCKF is constructed; S4: The predicted range of the front road adhesion coefficient obtained in S1 is introduced into the adaptive square root volumetric Kalman filter ASRCKF as the initial value of the target state vector to estimate the road adhesion coefficient; S5: Feedback the road image corresponding to the estimation result of S4 to S1, and the visual method will evolve automatically to realize the online combination of visual prediction and tactile verification methods to accurately estimate the road adhesion coefficient in real time.

2. The method according to claim 1, characterized in that Step S1 is specifically as follows: The on-board camera acquires images including road information ahead in real time, uses Labelimg to annotate the images and form a data set, and uses the data set to train the YOLOv5 deep learning algorithm. The trained YOLOv5 deep learning algorithm then identifies and classifies the road surface information ahead acquired by the camera, and estimates the predicted range of the road adhesion coefficient.

3. The method according to claim 2, characterized in that The nonlinear four-wheel vehicle dynamics model established in step S2 is expressed as: Among them, a x , a y are the lateral acceleration and longitudinal acceleration of the vehicle, v m is the vehicle speed, m is the vehicle mass, δ is the front wheel turning angle, ω is the yaw angle, β is the sideslip angle, a and b are the front and rear wheelbases respectively, I z is the yaw moment of inertia, T f is the front axle width, T r is the rear axle width, They are the lateral force and longitudinal force of the left front wheel, right front wheel, left rear wheel and right rear wheel of the vehicle respectively; The tire model established in step S2 is expressed as: Y=Dsin[Carctan{Bx-E(Bx-arctan(Bx)}]#(5) Among them, Y is the tire longitudinal force or lateral force, B is the stiffness factor, C is the shape factor, D is the peak factor, E is the curvature factor, and x is the sideslip angle α or longitudinal slip λ of each wheel.

4. The method according to claim 3, characterized in that Step S3 specifically includes: S31: Discretize the vehicle dynamics model and tire model and express them as: Where x is the state vector, z is the measurement vector, u is the input vector, f(*) is the state transfer function, h(*) is the output function, w is the process noise, and v is the measurement noise; S32: Time update, expressed as: Build volume points: Where i = 1, 2, ... m, m = 2n, m is the number of volume points, and n is the dimension of the state vector x; Weight: Construct a volume point set: Compute the propagated volume points: Orthogonal triangular decomposition is introduced to calculate the square root factor of the state prediction and prediction error covariance matrix, which is expressed as: Among them, Tria(*) represents the orthogonal triangular decomposition of the matrix, and obtains an upper triangular matrix, represents a weighting matrix; S33: Measurement update, expressed as: Update volume point set: Compute the propagated volume points: Compute the square root factor of the measurement prediction and innovation covariance matrix: Compute the innovation covariance matrix and the measurement autocovariance matrix: P z,t+1|t =P zz,t+1|t -R t+1 #(22) Compute the cross-correlation covariance matrix: Calculate the Kalman gain: State Estimation: Compute the square root factor of the estimated error covariance matrix: S34: Generate residuals based on the difference between the actual measured value and its estimated value, and design the process noise covariance adaptive rule, which is expressed as: Among them, the parameters ρ>0,δ>0, when When it is larger or smaller, it can be increased by adjusting ρ and δ or reduce To reduce the estimation error, improve the estimation accuracy and convergence speed of the filtering algorithm; for the selection of ρ, according to the residual Will Compare with the preset threshold θ, and then adjust ρ; the specific rules are as follows:

5. The method according to claim 4, characterized in that The preset threshold θ is set to 0.

075.

6. The method according to claim 4, characterized in that In step S5, the road image corresponding to the estimation result annotation is added to the YOLOv5 training set, and the data set is expanded online and the deep learning method is trained online to realize the self-evolution of the algorithm, narrow the prediction range of the visual prediction method, provide a more accurate prediction initial value for the subsequent filtering algorithm, and realize the comprehensive estimation of the road adhesion coefficient by the visual method and the tactile verification method.

Images