A method for estimating road adhesion coefficient by fusing image and vehicle dynamics

Abstract

The application discloses a road surface adhesion coefficient estimation method based on image and vehicle dynamics fusion, and belongs to the field of automatic driving vehicle control. The method is as follows: upper and lower bounds of the road surface adhesion coefficient are obtained by using road image information; a three-degree-of-freedom vehicle dynamics model and a Dugoff tire model are established; and the road surface adhesion coefficient is estimated by using a rolling time domain method. In order to reduce the influence of the deviation of the model, especially the side stiffness, on the estimation of the road surface adhesion coefficient, correction factors of front and rear tire side stiffness are introduced into the model, and the correction factors are taken as a parameter to be estimated together with the road surface adhesion coefficient, so that the estimation accuracy of the road surface adhesion coefficient is further improved. In order to reduce the estimation range of the road surface adhesion coefficient, the rolling time domain estimation method is adopted, and the result of image recognition is taken as the upper and lower bounds of the constraint of the rolling time domain estimation method, so that online fusion is carried out, and the real-time performance of the fusion estimation is improved.

Description

Technical Field

[0001] This invention belongs to the field of autonomous vehicle control, and in particular to a method for estimating road adhesion coefficient by fusing image and vehicle dynamics. Background Technology

[0002] Real-time estimation and monitoring of the road surface adhesion coefficient (BPCC) is crucial for vehicle emergency collision avoidance control, allowing for corresponding adjustments to braking and steering control to improve vehicle safety. Existing estimation methods mainly fall into two categories: one is based on dynamic estimators, which offers high accuracy and low cost. However, its estimation stability is significantly affected under driving conditions such as high speeds, continuous cornering, and road surface changes, resulting in estimation lag. Furthermore, it requires complex algorithm design and data processing, and real-time performance is challenging. The other category uses advanced sensors to directly measure the BPCC. This method can predict road conditions ahead. However, it can only determine the road type, not the specific BPCC value, and is easily affected by external environmental factors, resulting in longer prediction times and poorer real-time performance.

[0003] Currently, estimation methods for dynamics and image fusion mainly involve data-level fusion. For example, CN110765909A uses a support vector machine to identify the road surface type, obtains an empirical value for the adhesion coefficient, and constructs a fusion estimator using the difference between this value and the road surface adhesion coefficient estimated by a disturbance estimator based on tire longitudinal force, thus obtaining the fusion estimation result. CN111688707A identifies the road surface type through a deep neural network, obtains a visual estimate of the road surface adhesion coefficient, constructs a road surface adhesion coefficient-tire longitudinal force estimator, obtains the current road surface adhesion coefficient estimate, and finally uses a fuzzy inference method to obtain the fusion estimate. CN111845709A corrects the empirical values ​​of the road surface adhesion coefficient corresponding to the road surface type obtained from the image and LiDAR by using the road surface adhesion coefficient estimated from the tire longitudinal dynamic response, thus obtaining the final fusion value. However, the fusion methods used in the above patents all use the road surface adhesion coefficient determined by the road image as the parameter of the fusion estimator, and the estimation is still mainly based on dynamics, failing to fully utilize the advantages of both image information and dynamics-based methods. Furthermore, current algorithms all fuse image and dynamic information offline, resulting in poor real-time estimation performance. Summary of the Invention

[0004] The purpose of this invention is to solve the problems of poor real-time performance in existing fusion methods, and to provide a road surface adhesion coefficient estimation method that fuses images and vehicle dynamics. This method makes full use of visual and sensor information and can achieve online fusion, thereby improving estimation accuracy and real-time performance.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for estimating road surface adhesion coefficient by fusing image and vehicle dynamics, the method being:

[0007] Step 1: Obtain the upper and lower bounds of the road surface adhesion coefficient using road image information;

[0008] Step 2: Establish a three-degree-of-freedom vehicle dynamics model and a Dugoff tire model;

[0009] Step 3: Estimate the road surface adhesion coefficient using the rolling time-domain method.

[0010] Furthermore, step one specifically includes:

[0011] Step 11: Select different types of road images from the RCSD dataset to form training and validation sets;

[0012] Steps 1 and 2: Use ResNet residual neural network to simultaneously train and validate road surface types offline on road surface images. In the process of image classification, the role of the validation set is to adjust the parameters of the model during training and to conduct a preliminary evaluation of the model's capabilities, thereby obtaining the road surface classification model.

[0013] Step 13: Based on the road surface classification model described above, identify road images online;

[0014] Step 14: Based on the mapping relationship between different road surface types and road surface adhesion coefficient at high and low speeds, obtain the upper and lower bounds of the road surface adhesion coefficient at the current moment.

[0015] Furthermore, step two specifically involves:

[0016] Step 21: Three-DOF vehicle dynamics model

[0017] The expression for a vehicle dynamics model with nonlinear three degrees of freedom (longitudinal, lateral, and yaw) is as follows:

[0018]

[0019]

[0020]

[0021] In the formula, a x F is the longitudinal acceleration, m is the total mass of the vehicle, and F is the longitudinal acceleration. xij (i = f, r represents front and rear; j = l, r represents left and right) represents the longitudinal force of the four wheels, δ f For the front wheel steering angle, F yij(i = f, r represents front and rear; j = l, r represents left and right) represents the lateral force of the four wheels, a y It is lateral acceleration. It is the yaw acceleration, and it is the derivative of γ; I z Let l be the moment of inertia about the z-axis. f l is the distance from the center of mass to the front axle. r B is the distance from the center of mass to the rear axle. f B is the front wheel track. r This refers to the rear wheel track.

[0022] Step 22: Dugoff Tire Model

[0023] This invention only estimates the road surface adhesion coefficient. As can be seen from the following formula, other input parameters in the Dugoff tire model, including the vertical force F acting on the tire, are... z Tire stiffness C x and C y The slip ratio *s* and sideslip angle *α* are both independent of the road adhesion coefficient *μ*. Therefore, the tire forces in the Dugoff model can be normalized, as expressed by:

[0024]

[0025]

[0026]

[0027]

[0028] Where μ is the road surface adhesion coefficient, F zij (i = f, r represents front and rear; j = l, r represents left and right) represents the vertical force of the four wheels, C x C represents the longitudinal stiffness of the tire. y S represents the lateral stiffness of the tire. ij Let L be the slip ratio of the four wheels, L be a parameter of the Dugoff tire model, and f(L) be a function of L. (i = f, r represents front and rear; j = l, r represents left and right) represents the normalized longitudinal force of the four wheels. (i = f, r represents front and rear; j = l, r represents left and right) represents the normalized lateral force of the four wheels, α ij The slip angle is the side slip angle of the four wheels.

[0029] Furthermore, step three specifically includes:

[0030] Step 31: Road Adhesion Coefficient Estimation Model

[0031] Based on the two expressions from steps 21 and 22, the discretized nonlinear difference equations are rearranged to obtain the state equation and observation equation of the estimator:

[0032]

[0033]

[0034]

[0035] In the formula, x k+1 =[μ k ,v x,k ,v y,k ,γ k ] represents the state variable of the estimator at time k+1, μ k Let v be the road adhesion coefficient at time k. x,k Let v be the longitudinal velocity at time k. y,k Let γ be the lateral velocity at time k. k Let f(x) be the yaw rate at time k. k ,u k Let x be the state equation at time k. k =[μ k ,v x,k ,v y,k ,γ k ] represents the state variable of the estimator at time k, u k =[δ f,k ] represents the input to the estimator at time k, w k For process noise, y k =[a x,k ,a y,k ,γ k ] represents the output of the estimator at time k, a x,k It is the longitudinal acceleration at time k, a y,k It is the lateral acceleration at time k, h(x) k ,u k Let ) be the output equation at time k, and e k To measure noise, Δt is the sampling interval, and the subscript k in the above formula indicates the physical quantity at time k; w k and e k The variances are Q k and R k That is, E(w) k w k T )=Q k and E(e k e k T ) = R k .

[0036] Step 32: Road Adhesion Coefficient Estimation Model with Correction Factor

[0037] To reduce the impact of lateral stiffness on the road adhesion coefficient estimation model, the tire lateral stiffness is adaptively adjusted using the difference between the actual tire force and the calculated result. The correction factor expressions for the front and rear tire lateral stiffness are as follows:

[0038]

[0039] By treating the correction factor as a state variable, the above estimation model is reconstructed: the state equation and observation equation of the estimator are modified as follows:

[0040]

[0041] In the formula, λ f λ is the correction factor for the front wheel lateral stiffness. r This is a correction factor for the rear wheel lateral stiffness. The front wheel lateral force F calculated for the Dugoff tire model. yf To obtain the actual front wheel lateral force in Carsim software, F is the rear wheel lateral force calculated for the Dugoff tire model. yr To obtain the actual rear wheel lateral force in Carsim software, x λ,k+1 =[x k+1 ,λ fk+1 ,λ rk+1 ] T Let y be the augmented state vector after introducing a correction factor at time k+1. λ,k =[y k ,λ fk ,λ rk ] T f is the augmented output after introducing a correction factor at time k. λ (·) denotes the augmented state equation, h λ (·) represents the augmented output equation.

[0042] Step 33: Rolling Time Domain Estimation Method

[0043] The rolling time-domain estimation method can estimate the road surface adhesion coefficient by solving a numerical optimization problem in the finite time domain. The specific description of the numerical optimization problem in this invention is as follows:

[0044]

[0045]

[0046] In the formula, The whole is a cost function. Let be the independent variable of the function, representing the estimated value of the augmented state variable at time k. N represents the predicted value of the augmented state variable at time k. e For the rolling prediction time domain, the symmetric positive definite matrices P, Q, and R are the covariance matrices of the state variables, model errors, and output variables, respectively. j Let j be the output at time j. Let j be the output function at time j. Let w be the independent variable of the function, representing the estimated value of the augmented state variable at time k. j Let x be the process noise at time j. min and x max λ represents the upper and lower bounds of the state variables. f,min , λ r,min and λ f,max , λ r,max These are the upper and lower bounds of the front and rear lateral stiffness correction factors, respectively.

[0047] The upper and lower bounds of the road surface adhesion coefficient at the current moment corresponding to the online image recognition in step one are determined by the time synchronization algorithm, and then substituted into the constraint conditions of the above formula. At the same time, the correction factor and the road surface adhesion coefficient are estimated, and the correction factor at this moment is input into the estimation model. The rolling time domain algorithm is used for online real-time estimation, thereby obtaining an accurate estimate of the road surface adhesion coefficient.

[0048] Compared with existing technologies, the beneficial effects of this invention are as follows: To reduce the impact of model deviations, especially lateral stiffness, on the estimation of the road adhesion coefficient, this invention introduces a correction factor for the lateral stiffness of the front and rear tires into the model, estimating it as a parameter simultaneously with the road adhesion coefficient, thereby further improving the estimation accuracy of the road adhesion coefficient. To reduce the estimation range of the road adhesion coefficient, this invention employs a rolling time-domain estimation method and uses the image recognition results from a camera as the upper and lower bounds of its constraints, thereby performing online fusion and improving the real-time performance of the fusion estimation. Attached Figure Description

[0049] Figure 1 This is a flowchart of the present invention;

[0050] Figure 2 This is a schematic diagram of the three-degree-of-freedom dynamic model of the present invention;

[0051] Figure 3 This is a schematic diagram of the spatiotemporal synchronization of the present invention. Detailed Implementation

[0052] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0053] Example 1:

[0054] like Figures 1-3 As shown, a method for estimating road surface adhesion coefficient by fusing image and vehicle dynamics is presented, with a specific process combined with... Figure 1 As shown, it includes the following steps:

[0055] Step 1: Obtain the upper and lower bounds of the road surface adhesion coefficient using road image information.

[0056] The road image information used was from the Autonomous Driving Large-Scale Road Image Classification Dataset (RCSD) released by the research team of the School of Vehicle and Transportation at Tsinghua University. The original road images were acquired by a USB camera and collected and stored via IPC. Specifically, 2600 images of each of the four types of road surfaces—gravel, snow, rain, and asphalt—were selected as the training set, and 1000 images as the test set.

[0057] Residual networks (ResNet) aim to design ultra-deep networks that, compared to traditional convolutional neural networks, avoid the vanishing and exploding gradient problems. As the formula shows, even when the gradients of newly added layers in the neural network are 0, the gradient update in the residual network has an extra "1". Gradients from deeper layers can directly reach the previous layer, allowing for effective training of the parameters of shallower layers and avoiding the vanishing gradient problem. Secondly, this network incorporates skip connections, allowing direct access to the optimal solution. This ensures that the weight parameters are at least as good as the previous training results and accelerates network convergence. The most common example is ResNet50, which contains 49 convolutional layers and one fully connected layer.

[0058]

[0059] Where ε is the output of the residual network, x i x is the input to the i-th layer residual network. L The network input is the Lth layer of the direct part, where i, L, and l are the layer numbers of the residual network, F(·) is the residual function, and w i Let w be the weight of the i-th layer residual network, w be the weight of the residual network, and x be the input of the residual network.

[0060] The above network was trained offline using the road surface classification training dataset to obtain a road surface classification model. The deep learning framework on which this network is based is the Deep Learning Toolbox in MATLAB.

[0061] Based on the road surface classification model described above, the road surface is identified online. According to the vehicle speed, the upper and lower bounds of the road surface adhesion coefficient at the current moment are determined by the mapping relationship between the road surface type and the road surface adhesion coefficient, as shown in Table 1.

[0062] Table 1 Comparison of Road Surface Type and Adhesion Coefficient Values

[0063] Road surface type Coefficient of adhesion (below 50km / h) Coefficient of adhesion (above 50km / h) Sand and gravel 0.4-0.6 0.45-0.65 Snowy day 0.2-0.45 0.2-0.35 rain 0.35-0.5 0.3-0.4 asphalt 0.7-0.9 0.7-0.9

[0064] Step 2: Establish a three-degree-of-freedom vehicle dynamics model and a Dugoff tire model.

[0065] Step 2.1 Three-degree-of-freedom vehicle dynamics model

[0066] The expression for a vehicle dynamics model with nonlinear three degrees of freedom (longitudinal, lateral, and yaw) is as follows, assuming that the left and right steering wheels have the same steering angle:

[0067]

[0068]

[0069]

[0070] In the formula, F xij (i = f, r represents front and rear; j = l, r represents left and right) represents the longitudinal force of the four wheels, F. yij (i = f, r represents front and rear; j = l, r represents left and right) represents the lateral forces of the four wheels, B f For wheel track, l f and l r δ represents the distance from the center of mass to the front and rear axes, respectively. f Let m be the front wheel steering angle, m be the total vehicle mass, γ be the vehicle yaw rate, and I be the front wheel steering angle. z Let a be the moment of inertia about the z-axis. x and a y These are longitudinal and lateral accelerations.

[0071] Step 2.2 Dugoff tire model

[0072] The Dugoff tire model only has longitudinal lateral stiffness C. x And lateral deflection stiffness C y It has two basic parameters, a simple model, and can explicitly describe the relationship between tire force and road adhesion coefficient.

[0073] Since this invention only estimates the road adhesion coefficient, while other input parameters in the Dugoff tire model, such as the vertical force F acting on the tire, are not considered... z Both the slip ratio *s* and the sideslip angle *α* are independent of the road adhesion coefficient *μ*. Therefore, the tire forces in the Dugoff model can be normalized, as shown in the following expression:

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] Among them, C x and C y For the longitudinal and lateral stiffness of the tire, F zij (i = f, r represents front and rear; j = l, r represents left and right) represents the vertical force of the four wheels, v x and v y For longitudinal and lateral vehicle speeds, v xij (i = f, r represents forward and backward; j = l, r represents left and right) represents the wheel speeds of the four wheels, ω ij (i = f, r represents forward and backward; j = l, r represents left and right) represents the angular velocity of the four wheels, S ij Let α be the slip ratio of the four wheels. ij The slip angle is the side slip angle of the four wheels.

[0083] Step 3: Estimate the road adhesion coefficient using the rolling time-domain method

[0084] Step 3.1 Road Adhesion Coefficient Estimation Model

[0085] Based on the nonlinear three-degree-of-freedom vehicle dynamics model constructed in step 2.1, a road adhesion coefficient estimator is built. The normalized longitudinal and lateral tire forces from step 2 are input into the road adhesion coefficient estimator to obtain the state equation and measurement equation of the discretized nonlinear system:

[0086]

[0087]

[0088]

[0089] In the formula, w k and e k The process noise and measurement noise are respectively, with variances Q and Q', respectively. k and R k That is, E(w) k w k T )=Q k and E(e k e k T ) = R k Δt is the sampling interval, x k =[μ k ,v x,k ,v y,k ,γ k ] represents the state variable of the estimator at time k, u k =[δ f,k ] represents the input to the estimator at time k, y k =[a x,k ,a y,k ,γ k ] represents the output of the estimator at time k.

[0090] Step 3.2 Road Adhesion Coefficient Estimation Model with Correction Factor

[0091] In establishing the Dugoff tire model, the lateral stiffness of the tire is assumed to be constant. However, in reality, the lateral stiffness varies with vehicle speed and road surface conditions, introducing bias into the estimation model. Therefore, a method can be used to adaptively adjust the tire lateral stiffness by using the difference between the linear tire force obtained from the linear tire lateral stiffness and the tire force estimated by the Dugoff tire model. This involves introducing a correction factor defined as...

[0092]

[0093] Therefore, the lateral stiffness of the front and rear axle tires can be expressed using the correction coefficients obtained from the above formula, and the correction values ​​are:

[0094]

[0095] Substituting the corrected lateral stiffness into the estimation model, and assuming that the correction factor remains constant in adjacent time intervals, we obtain the road adhesion coefficient estimation model with the correction factor:

[0096]

[0097] In the formula, xλ,k =[x k ,λ fk ,λ rk ] T and y λ,k =[y k ,λ fk ,λ rk ] T Let f be the augmented state vector and augmented output after introducing a correction factor at time k. λ (·) and h λ (·) represent the augmented state equation and the output equation, respectively.

[0098] However, when using the above model, it is necessary to determine the augmented state vector x. λ,k Whether it is observable. Therefore, the above nonlinear model is linearized, and the observation matrix O is determined. k Is it full rank? The observation matrix of the model is calculated to be full rank, allowing for the aforementioned reconstruction process.

[0099]

[0100]

[0101] In the formula, A λ,k and C λ,k Let be the augmented state matrix and the augmented output matrix after linearization at time k, respectively.

[0102] Step 3.3 Rolling Time Domain Estimation Method

[0103] The Rolling Time-Domain Estimation (MHE) method describes the estimation problem as an optimization problem, directly incorporating the system's constraints into the optimization problem. This algorithm is robust and an effective method for solving state estimation problems of constrained systems. To improve the estimation accuracy of the pavement adhesion coefficient, linearization approximation of the nonlinear estimation model is unnecessary.

[0104] Using the estimation model from step 3.2, the road surface adhesion coefficient can be estimated online by solving a finite-time domain optimization problem. The specific description of this optimization problem is as follows:

[0105]

[0106]

[0107] In the formula, N e For the rolling prediction time domain, x min and x max λ represents the upper and lower bounds of the state variables. f,min , λ r,min and λ f,max , λ r,max, respectively, are the upper and lower bounds of the front and rear lateral stiffness correction factors, the symmetric positive definite matrices P and Q are the covariance matrices of the state variables and output variables, and R is the covariance matrix of the model error.

[0108] Based on the principle of rolling optimization, the implementation steps of online estimation of road surface adhesion coefficient using the rolling time-domain estimation algorithm are as follows:

[0109] (1) Set the weight parameters P, Q, R and the estimated time domain length N. e And the output data y0 at the current moment;

[0110] (2) Obtain the new output data y1, and let y k-Ne =y0,k=1,...,N e -1, constituting kN e Measurement sequence up to time k Similarly, we obtain kN e State noise sequence up to time k-1

[0111] (3) Based on the augmented state quantity estimated in the previous time step The predicted value of the augmented state quantity is obtained by the following formula.

[0112]

[0113] (4) Solve the aforementioned nonlinear programming problem to obtain the estimated value of the augmented state variables at the current time step.

[0114]

[0115] (5) Let k+1 replace time k, and change the correction factor λ at time k. fk ,λ rk Input the data into the estimation model and repeat steps (2) to (4) to iterate, thereby obtaining the road surface adhesion coefficient and correction factor at each time step.

[0116] During vehicle operation, cameras capture images of the road ahead, while onboard sensors collect vehicle dynamics information—the response information at the tire-road contact patch. There is a time lag between acquiring these two types of information. Therefore, before incorporating the upper and lower bounds of the road adhesion coefficient obtained from image recognition into the constraints of the rolling time-domain algorithm, it is necessary to synchronize the road image information and the vehicle dynamics response information in time.

[0117] Set the camera's preview position to the center of the road area in the image, and measure the distance P1 between the preview position and the vehicle's front axle. Add a time delay t to the information from the camera image. delay The following can be calculated:

[0118]

[0119] Where Δt is the image recognition time, v x This represents the longitudinal speed of the vehicle.

[0120] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0121] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style of the specification is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other implementations that can be understood by those skilled in the art.

Claims

1. A method for estimating road surface adhesion coefficient by fusing image and vehicle dynamics, characterized in that: The method is as follows: Step S1: Obtain the upper and lower bounds of the road surface adhesion coefficient using road image information; Step S2: Establish a three-degree-of-freedom vehicle dynamics model and a Dugoff tire model; specifically: Step S21: Three-degree-of-freedom vehicle dynamics model: It contains longitudinal, lateral, and yaw nonlinear three-degree-of-freedom vehicle dynamics model expressions; Step S22: Dugoff tire model: It contains the expression obtained by normalizing the tire forces of the Dugoff model; Step S3: Estimate the road surface adhesion coefficient using the rolling time-domain method, specifically: Step S31: Road surface adhesion coefficient estimation model Based on the two expressions from steps S21 and S22, the equations are rearranged into nonlinear difference equations, yielding the state equation and observation equation of the estimator: In the formula, = [ , , , ]for The state quantity of the estimator at time +1 for The road surface adhesion coefficient at time t, for longitudinal velocity at time t, for Lateral velocity at time, for yaw rate at time t, for The state equation at time t, = [ , , , ]for The state variables of the time estimator = [ ]for The input to the time estimator For process noise, = [ , , ]for The output of the time estimator yes longitudinal acceleration at time t, yes Lateral acceleration at time t, for The output equation at time t, To measure noise, For the sampling interval, the subscript in the above formula is... express The physical quantity at a given moment; and The variances are respectively and ,Right now and ; Step S32: Road Adhesion Coefficient Estimation Model with Correction Factor The tire lateral stiffness is adaptively adjusted using the difference between the actual tire force and the calculated result. The correction factor expressions for the front and rear tire lateral stiffness are as follows: By treating the correction factor as a state variable, the above estimation model is reconstructed: the state equation and observation equation of the estimator are modified as follows: In the formula, This is a correction factor for the front wheel lateral stiffness. This is a correction factor for the rear wheel lateral stiffness. The front wheel lateral force calculated for the Dugoff tire model. To obtain the actual front wheel lateral force in Carsim software, The rear wheel lateral force calculated for the Dugoff tire model. To obtain the actual rear wheel lateral force in Carsim software, for The augmented state vector after introducing the correction factor at time +1 for The augmented output after introducing the correction factor at each step. Representing the augmented state equation, Indicate the output equation; Step S33: Rolling Temporal Estimation Method The rolling time-domain estimation method estimates the road surface adhesion coefficient by solving a numerical optimization problem in the finite time domain. The specific description of the numerical optimization problem is as follows: In the formula, The whole is a cost function. Let be the independent variable of the function, representing the th . Estimates of augmented state variables at time intervals. For the first Predicted values ​​of augmented state variables at any given time. For the rolling prediction time domain, a symmetric positive definite matrix , , These are the covariance matrices of the state variables, model errors, and output variables, respectively. For the first Output at any given time For the first The output function at time t, Let be the independent variable of the function, representing the th . Estimates of augmented state variables at time intervals. For the first Time-based process noise, and These are the upper and lower bounds of the state variables. , and , These are the upper and lower bounds of the front and rear lateral stiffness correction factors, respectively.

2. The method for estimating road surface adhesion coefficient by fusing image and vehicle dynamics according to claim 1, characterized in that: Step S1 specifically involves: Step S11: Select different types of road images using the RCSD dataset to form training and validation sets; Step S12: Use ResNet residual neural network to simultaneously train and validate road surface types offline on road surface images. In the process of image classification, the role of the validation set is to adjust the parameters of the model during training and to conduct a preliminary evaluation of the model's capabilities, thereby obtaining the road surface classification model. Step S13: Based on the road surface classification model described above, identify road images online; Step S14: Based on the mapping relationship between different road surface types and road surface adhesion coefficient at high speed and low speed, obtain the upper and lower bounds of the road surface adhesion coefficient at the current moment.

3. The method for estimating road surface adhesion coefficient by fusing image and vehicle dynamics according to claim 1, characterized in that: Step S2 specifically involves: Step S21: Three-degree-of-freedom vehicle dynamics model The expression for a vehicle dynamics model with nonlinear three degrees of freedom (longitudinal, lateral, and yaw) is as follows: In the formula, For longitudinal acceleration, For the overall vehicle quality, ( = Representing before and after; = (Representing left and right) represents the longitudinal force on the four wheels. For the front wheel steering angle, ( = Representing before and after; = (Representing left and right) represents the lateral force of the four wheels. It is lateral acceleration. It is the yaw acceleration. The derivative; To bypass Moment of inertia of the shaft, This is the distance from the center of gravity to the front axle. This is the distance from the center of mass to the rear axle. The front wheel track. This refers to the rear wheel track. Step S22: Dugoff tire model The tire forces in the Dugoff model are normalized, and the expression is: in, The road surface adhesion coefficient, ( = Representing before and after; = (Representing left and right) represents the vertical force on the four wheels. This refers to the longitudinal stiffness of the tire. The slip ratio of the four wheels. For about The function, ( = Representing before and after; = (Representing left and right) represents the normalized longitudinal force of the four wheels. This refers to the lateral stiffness of the tire. The sideslip angles of the four wheels. ( = Representing before and after; = (Representing left and right) represents the normalized lateral force of the four wheels. This is a parameter of the Dugoff tire model.

Images