Road adhesion coefficient and vehicle state estimation method, terminal and medium

Abstract

The invention provides a road adhesion coefficient estimation method, a vehicle state estimation method, a terminal and a medium. The road adhesion coefficient estimation method comprises the following steps: acquiring vehicle sensor information and an image of a road in front of a vehicle; inputting vehicle sensor information into the BiLSTM model to obtain vehicle estimated mass; determining the road category at the current moment by processing the image of the road in front of the vehicle, and obtaining a first estimated road adhesion coefficient according to the mapping relation between the road category and the road adhesion coefficient; and processing the vehicle estimated mass, the first estimated road adhesion coefficient and the vehicle sensor information through an MMCC + SCKF algorithm to obtain an estimated road adhesion coefficient. A tire model and a seven-degree-of-freedom vehicle dynamics model are introduced into an adhesion coefficient determination module, and a square root volume Kalman filtering algorithm of a maximum mixed correlation entropy criterion is combined, so that interference of non-Gaussian noise on road adhesion coefficient estimation is effectively inhibited, and the precision and robustness of road adhesion coefficient estimation are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control of vehicles, and in particular to a road surface adhesion coefficient estimation method, a terminal and a medium. BACKGROUND

[0002] With the rapid development of intelligent driving technology, as a key execution layer of intelligent driving, the road surface adhesion coefficient has become a core technology module to ensure driving safety, comfort and maneuverability. Real-time high-precision estimation of the road surface adhesion coefficient is not only directly related to the effective implementation of the intelligent chassis control strategy, but also plays a crucial supporting role in the path planning and control decision of the upper layer automatic driving system. Traditional road surface adhesion coefficient estimation methods mainly rely on dynamic or kinematic models based on physical modeling, combined with Kalman filtering and other estimation algorithms to solve the state quantity. Although such methods have good physical interpretability and engineering deployability, in the face of complex road conditions, non-stationary dynamic conditions or missing sensor information, they often face problems such as decreased estimation accuracy, strong dependence on model parameters, and insufficient real-time performance and robustness. In recent years, with the wide application of deep learning technology, especially convolutional neural networks (CNN) and long short-term memory networks (LSTM) in time series modeling and image recognition, the estimation of road surface adhesion coefficient has gradually shifted from "model-driven" to "data-driven". Deep neural networks can automatically extract complex nonlinear features from a large amount of historical sensor data, and mine the implicit relationship between vehicle state and sensor observation, significantly improving the adaptability and accuracy of state estimation in complex conditions. However, relying solely on data-driven methods also has problems such as insufficient generalization ability and lack of protection for boundary conditions.

[0003] Model-driven methods have the problems of modeling uncertainty and poor adaptability to non-Gaussian noise. In actual operation, key parameters such as vehicle mass have significant fluctuations, and traditional model methods are highly sensitive to these changes, resulting in significant accumulation of errors in the estimation of other parameters. At the same time, most existing methods assume that the sensor noise satisfies the Gaussian distribution, however, the vehicle is often in a non-Gaussian noise environment in actual operation, such as sudden acceleration, crosswind interference, etc., at which time the estimation accuracy is significantly reduced, affecting the system robustness. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application proposes a road surface adhesion coefficient estimation method, aiming to improve the accuracy and robustness of road surface adhesion coefficient estimation.

[0005] In a first aspect, the embodiments of the present application provide a road surface adhesion coefficient estimation method, comprising: obtaining vehicle sensor information and a road image in front of the vehicle; inputting the vehicle sensor information into a BiLSTM model to obtain an estimated vehicle mass; determining a road category at a current time by processing the image of the road ahead of the vehicle, and obtaining a first estimated road adhesion coefficient according to a mapping relationship between the road category and the road adhesion coefficient; processing the vehicle estimated mass, the first estimated road adhesion coefficient and the vehicle sensor information by the MMCC+SCKF algorithm to obtain an estimated road adhesion coefficient.

[0006] In a second aspect, an embodiment of the present application provides a vehicle state estimation method, including a road adhesion coefficient estimation method, and the vehicle state estimation method further includes: processing the vehicle estimated mass, the vehicle sensor information and the road adhesion estimation coefficient by the MMCC+SCKF algorithm to obtain vehicle state estimation information.

[0007] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the road adhesion coefficient estimation method according to any one of the first aspect or the vehicle state estimation method according to any one of the first aspect when executing the computer program.

[0008] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the road adhesion coefficient estimation method according to any one of the first aspect or the vehicle state estimation method according to any one of the first aspect.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, and when the computer program product is executed on a terminal device, the terminal device executes the road adhesion coefficient estimation method according to any one of the first aspect or the vehicle state estimation method according to any one of the first aspect.

[0010] In the embodiment of the present application, the vehicle estimated mass, the first estimated road adhesion coefficient and the vehicle sensor information are processed by the MMCC+SCKF algorithm to obtain an estimated road adhesion coefficient. The tire model and the seven-degree-of-freedom vehicle dynamics model are introduced into the adhesion coefficient determination module, and the maximum mixed correlation entropy criterion square root cubage Kalman filter algorithm (MMCC+SCKF) is combined to effectively suppress the interference of non-Gaussian noise on the road adhesion coefficient estimation, and the precision and robustness of the road adhesion coefficient estimation are improved. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a flowchart of a first embodiment of a road adhesion coefficient estimation method provided by an embodiment of the present application; Figure 2 is a flowchart of a first embodiment of a vehicle state estimation method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0012] Figure 1 A flowchart of a first embodiment of a road adhesion coefficient estimation method provided by an embodiment of the present application is shown, which can be applied in the road adhesion coefficient estimation device described above as an example but not limitation. As shown in the flowchart, the method can include: Figure 1 101, obtaining vehicle sensor information and a road image in front of the vehicle; In order to improve the accuracy and robustness of road adhesion coefficient estimation, the road adhesion coefficient estimation device obtains vehicle sensor information through a vehicle IMU sensor or a vehicle wheel speed sensor. That is, the road adhesion coefficient estimation device obtains the left front wheel speed , right front wheel speed , left rear wheel speed , right rear wheel speed , longitudinal acceleration , lateral acceleration , vertical acceleration , yaw angular velocity , pitch angular velocity, front wheel steering angle , longitudinal vehicle speed , lateral vehicle speed , roll angular velocity, longitudinal acceleration at the center of mass , lateral acceleration at the center of mass , yaw angular velocity , yaw moment , engine torque , engine speed n, left front wheel tire longitudinal force , right front wheel tire longitudinal force , left front wheel tire lateral force , right front wheel tire lateral force , left rear wheel tire longitudinal force , right rear wheel tire longitudinal force , left rear wheel tire lateral force , and right rear wheel tire lateral force ; the road adhesion coefficient estimation device obtains the jerk of the longitudinal acceleration , the jerk of the vertical acceleration ​The road surface adhesion coefficient estimation device also obtains a road image in front of the vehicle through a camera arranged at a driving position of the vehicle; that is, the road surface adhesion coefficient estimation device mainly processes picture information from the camera sensor through the information layer in the automatic driving domain, and the picture is cropped along the future wheel track. The vehicle sensor information can be low-pass filtered, denoised and time-synchronized to ensure that the vehicle state parameters output from the information layer to the state estimation layer have high precision, low time delay and consistency, and to provide data for the subsequent state estimation module.

[0013] As an implementation manner, the obtaining of the vehicle sensor information and the vehicle dynamic information can include: obtaining initial vehicle sensor information; performing preprocessing such as low-pass filtering, denoising and time synchronization on the initial vehicle sensor information; and performing standardized filtering processing on the preprocessed initial vehicle sensor information by using a moving average filter to obtain the vehicle sensor information. The initial vehicle sensor information can be initial left front wheel speed, initial right front wheel speed, initial left rear wheel speed, initial right rear wheel speed, initial longitudinal acceleration, initial lateral acceleration, initial vertical acceleration, initial jerk of the initial longitudinal acceleration, initial jerk of the initial vertical acceleration, initial yaw rate, initial pitch rate, initial front wheel steering angle, initial longitudinal vehicle speed, initial lateral vehicle speed, initial roll rate, initial longitudinal acceleration at the center of mass, initial lateral acceleration at the center of mass, initial yaw rate, initial yaw moment, initial engine torque, initial engine speed, initial left front wheel tire longitudinal force, initial right front wheel tire longitudinal force, initial left front wheel tire lateral force, initial right front wheel tire lateral force, initial left rear wheel tire longitudinal force, initial right rear wheel tire longitudinal force, initial left rear wheel tire lateral force and initial right rear wheel tire lateral force of the vehicle obtained by the vehicle IMU sensor or the vehicle wheel speed sensor. After the initial vehicle sensor information is obtained, the road surface adhesion coefficient estimation device performs preprocessing such as low-pass filtering, denoising and time synchronization on the initial vehicle sensor information; that is, the initial vehicle sensor information is low-pass filtered, denoised and time synchronized to ensure that the vehicle state parameters output from the information layer to the state estimation layer have high precision, low time delay and consistency, and to provide data for the subsequent state estimation module. Then, the preprocessed initial vehicle sensor information is resampled and standardized filtered by using a moving average filter to obtain the vehicle sensor information.

[0014] In the BiLSTM model, the input data of each dimension and the maximum value of the entire training set need to be standardized to ensure that all inputs are scaled in the range of -1 to 1. The information layer input data (preprocessed initial vehicle sensor information) needs to be resampled and filtered; when the preprocessed initial vehicle sensor information is processed, a moving average filter is used, and the filtering principle is to smooth the input data in a specified window. wherein, is a filter window length.

[0015] 102, inputting the vehicle sensor information into the BiLSTM model to obtain a vehicle estimated quality; The road adhesion coefficient estimation device inputs the vehicle sensor information into the BiLSTM model to obtain a vehicle estimated quality after obtaining the vehicle sensor information.

[0016] As an implementation form, the vehicle sensor information is inputted into the BiLSTM model to obtain a vehicle estimated quality, specifically comprising: A1, obtaining a vehicle speed according to the vehicle sensor information; The road adhesion coefficient estimation device obtains a vehicle speed according to the left front wheel speed and the right front wheel speed in the vehicle sensor information after obtaining the vehicle sensor information.

[0017] The vehicle speed can be calculated by the wheel speed obtained by the wheel speed sensor, and the calculation formula is: wherein, is a vehicle speed, is a left front wheel speed, is a left front wheel speed, is a wheel diameter.

[0018] A2, performing correlation coefficient analysis on the vehicle speed, the vehicle speed information and the vehicle power information to determine a feature input vector; The road adhesion coefficient estimation device performs Pearson correlation coefficient analysis on the vehicle speed, the vehicle speed information and the vehicle power information to determine a feature input vector after obtaining the vehicle speed. The feature input vector includes longitudinal acceleration , vertical acceleration , vehicle speed , engine torque , longitudinal acceleration jerk , and vertical acceleration jerk .

[0019] As an implementation form, the correlation coefficient analysis on the vehicle speed, the vehicle speed information and the vehicle power information to determine a feature input vector can include: performing Pearson correlation coefficient analysis on the vehicle speed information and the vehicle speed information to determine a preliminary feature input vector; incorporating the longitudinal acceleration jerk and the vertical acceleration jerk in the vehicle sensor information into the preliminary feature input vector to obtain the feature input vector.

[0020] Since road bump mainly affects the vertical acceleration of the vehicle, the vertical acceleration of the vehicle is introduced as an input, which can effectively reduce the influence of road bump on the estimation accuracy. At the same time, in order to avoid model overfitting caused by high correlation dimension information and simplify the model, Pearson correlation coefficient analysis is performed on the time sequence signals from the IMU and the CAN bus longitudinal acceleration , lateral acceleration , vertical acceleration , vehicle speed , engine torque and engine speed n. For the highly correlated data pairs, the lateral acceleration and engine speed n are filtered. Therefore, the feature vector of the input signal can be represented as: Since the longitudinal acceleration signal has a significant impact on the estimation of the vehicle mass. At the same time, in order to enhance the importance of the acceleration signal and emphasize its impact on the estimation of the vehicle mass, the jerk of the longitudinal acceleration and the vertical acceleration is calculated. Finally, 6-dimensional features are obtained as inputs. The final feature input vector of the neural network model is: wherein, is the longitudinal acceleration; is the vertical acceleration; is the longitudinal jerk; is the vertical jerk; is the vehicle speed; is the engine torque.

[0021] A3, inputting the feature input vector into the full connection layer of the BiLSTM model to obtain the estimated mass of the vehicle.

[0022] After determining the feature input vector, the road adhesion coefficient estimation device inputs the feature input vector into the full connection layer of the BiLSTM model to obtain the estimated mass of the vehicle.

[0023] A bidirectional long short-term memory network architecture is constructed, which consists of two LSTM nodes: one is responsible for processing the forward sequence of time series data, and the other is processing the reverse sequence. The output is the output of the forward sequence and the output of the reverse sequence. The hidden state of the LSTM node at the last time step is used as the extracted time series feature. In addition, by connecting the hidden state of each node in the LSTM hidden layer, high-dimensional features representing the pattern of the time series can be generated. These features are then used as the input of the full connection layer for estimating the mass of the heavy commercial vehicle. This architecture effectively solves the problems of long-term memory decay and gradient disappearance in the backpropagation process, and improves the robustness and accuracy of the mass estimation.

[0024] In a single LSTM structure, the input gate ft, the forget gate it, and the output gate ot can be represented as ; ; ; ; ; wherein, is the information of the previous time sequence, is the input at the current moment, is the output information of the current time sequence, is the cell state at the current moment, is a linear combination weight, is a bias, denotes a Sigmoid activation function.

[0025] 103, determining the road category at the current moment by processing the road image in front of the vehicle, and obtaining the first estimated road adhesion coefficient according to the mapping relationship between the road category and the road adhesion coefficient; After obtaining the road image in front of the vehicle, the road adhesion coefficient estimation device determines the road category at the current moment by processing the road image in front of the vehicle, and obtains the first estimated road adhesion coefficient according to the mapping relationship between the road category and the road adhesion coefficient. After obtaining the road image in front of the vehicle, the road adhesion coefficient estimation device estimates the road adhesion coefficient range based on EffcientNetV2 and calculates the image classification confidence.

[0026] As an implementation manner, the determination of the road category at the current moment by processing the road image in front of the vehicle, and the obtaining of the first estimated road adhesion coefficient according to the mapping relationship between the road category and the road adhesion coefficient, comprises: B1, obtaining the adhesion mapping relationship between the road type and the road adhesion coefficient range; Collect and construct: comprehensively consider the road friction state (dry, wet, water accumulation, icing, snow accumulation and snow melting), road material (asphalt, concrete, gravel, soil) and flatness grade (flat, light uneven, serious uneven), three types of attribute combinations, a total of 27 typical road types. The data set contains more than one million high-quality labeled images, covering diversified road conditions, shooting angles, brightness and weather conditions, and can fully reflect the complexity and variability in real driving environment.

[0027] The mapping relationship between the road surface type and the road surface adhesion coefficient range is constructed. The upper and lower limit ranges of the road surface adhesion coefficient of each type of road are determined by referring to the road design specification and the experimental report.

[0028] B2, by utilizing the time sequence correlation in the road image in front of the vehicle, a Gaussian weighted sliding window mechanism is introduced on the Softmax output probability vector of the last layer of the EfficientNetV2 network to determine the road surface type probability vector and the image confidence; EffcientNetV2 network construction. EfficientNetV2 is selected as the CNN backbone network, and progressive training strategy, Fused-MBConv module and compound scaling mechanism are introduced to significantly improve the model precision and training / inference efficiency while keeping the network lightweight. At the same time, with the help of large-scale pre-training of ImageNet-21k, it performs better in the small sample transfer task of road type and has good hardware adaptability.

[0029] Road surface type classification confidence calculation. By utilizing the time sequence correlation in the video sequence, a Gaussian weighted sliding window mechanism is introduced on the Softmax output probability vector of the last layer of the EfficientNetV2 network to perform time domain smoothing processing on the classification probability of each frame image. The prediction probability vector of the network at the kth moment is represented as: ; In the formula, is the prediction probability vector at the kth moment, is the prediction probability of road surface type 1 at the kth moment, is the prediction probability of road surface type 2 at the kth moment, is the prediction probability of road surface type C at the kth moment, C is the total number of categories, corresponding to 15 road categories in this paper.

[0030] In order to strengthen the dominant role of the reliable frame in the Gaussian weighting and provide quality evaluation basis for the weighted smoothing, the maximum component of Pk is taken as the confidence, that is, For the frame with high classification confidence, its weight will be amplified accordingly, so that when the real road surface type switches, the network's own more confident judgment can quickly affect the smoothing result, speed up the real RAC switching response and avoid the problem of delayed response caused by excessive smoothing.

[0031] B3, based on the confidence adaptive Gaussian weighted sliding window, the road surface type probability vector is processed to obtain the final output category at the current moment; The probability vector after the smoothing processing based on the confidence adaptive Gaussian weighted sliding window can be represented as ; wherein, ; ; ; wherein, Zk is a normalization coefficient, N is a window radius, e represents a time offset relative to a current frame, g(e) is a Gaussian position weight, sigma is a standard deviation of a Gaussian distribution, and G is a sum of all Gaussian weights in the window.

[0032] After the confidence-based adaptive Gaussian weighted sliding window processes the probability vector output by the EfficientNetV2, the final output class at the current time can be taken as ; B4, according to the final output class at the current time in the adhesion mapping relationship, determines the first estimated road adhesion coefficient corresponding to the final output class at the current time.

[0033] wherein, after the road adhesion coefficient estimation device determines the final output class at the current time, according to the final output class at the current time in the adhesion mapping relationship, the first estimated road adhesion coefficient corresponding to the final output class at the current time is determined. Wherein, the first estimated road adhesion coefficient includes the front frame adhesion coefficient identification result , the image class adhesion coefficient lower limit , the image class adhesion coefficient upper limit and the image confidence (that is, the road type classification confidence).

[0034] When determining the first estimated road adhesion coefficient through the EfficientNetV2 network model, the accuracy of identification is improved through the judgment of the image confidence, avoiding the identification of incorrect images, ensuring the stability and accuracy of the results; improving the accuracy of image identification.

[0035] 104, the vehicle estimated mass, the first estimated road adhesion coefficient and the vehicle sensor information are processed through the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient.

[0036] After the road adhesion coefficient estimation device obtains the vehicle estimated mass, obtains the first estimated road adhesion coefficient, and obtains the vehicle sensor information, the vehicle estimated mass, the first estimated road adhesion coefficient and the vehicle sensor information are processed through the MMCC+SCKF (Maximum Mixed Covariance-Square Root Cubic Kalman Filter) algorithm to obtain the estimated road adhesion coefficient.

[0037] that is, the vehicle estimated mass obtained by the mass estimation module , the first estimated road adhesion coefficient obtained by the image adhesion coefficient estimation module, and the vehicle sensor information obtained by the information layer as inputs, a road adhesion coefficient estimation module based on maximum mixed covariance-square root cubature Kalman filtering is constructed to estimate the road adhesion coefficient and obtain an estimated road adhesion coefficient. The first estimated road adhesion coefficient includes a front frame adhesion coefficient identification result , an image category adhesion coefficient lower limit , an image category adhesion coefficient upper limit , and an image confidence .

[0038] As an implementation manner, the vehicle estimated mass, the first estimated road adhesion coefficient, and the vehicle sensor information are processed by the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient, specifically including: 204. The vehicle estimated mass and the vehicle sensor information are processed by the MMCC+SCKF algorithm to obtain a second estimated road adhesion coefficient. After the vehicle estimated mass is determined and the vehicle sensor information is obtained, the vehicle estimated mass and the vehicle sensor information are processed by the MMCC+SCKF algorithm to obtain a second estimated road adhesion coefficient.

[0039] As an implementation manner, the vehicle estimated mass and the vehicle sensor information are processed by the MMCC+SCKF algorithm to obtain a second estimated road adhesion coefficient, specifically including: C1. A first seven-degree-of-freedom dynamic model is established according to the vehicle estimated mass and the vehicle sensor information. After the vehicle estimated mass is determined and the vehicle sensor information is obtained, a first seven-degree-of-freedom vehicle double-track dynamic model is established according to the vehicle estimated mass and the vehicle sensor information. The estimated mass obtained by the mass estimation module is taken as a known parameter to establish an accurate first seven-degree-of-freedom vehicle double-track dynamic model, including seven degrees of freedom of longitudinal, lateral, yaw, and four wheel rolling of the vehicle. The dynamic equations of longitudinal motion, lateral motion, and yaw motion are as follows: ; ; ; , wherein is the moment of inertia of the whole vehicle around the Z axis; is the longitudinal acceleration at the center of mass; is the lateral acceleration at the center of mass; ​is the longitudinal velocity of the vehicle; is the lateral velocity of the vehicle; is the longitudinal acceleration of the vehicle; is the lateral acceleration of the vehicle; is the yaw rate of the vehicle, is the yaw moment of the vehicle.

[0040] wherein the relationship between the vehicle longitudinal acceleration , the vehicle lateral acceleration , the vehicle yaw moment , the tire longitudinal force and the tire lateral force is given by: ; ; ; wherein, is the distance from the vehicle center of mass to the front axle; is the distance from the vehicle center of mass to the rear axle, is the track of the front axle; is the track of the rear axle; is the front wheel steering angle input, is the left front tire longitudinal force; is the right front tire longitudinal force; is the left front tire lateral force; is the right front tire lateral force; is the left rear tire longitudinal force; is the right rear tire longitudinal force; is the left rear tire lateral force; is the right rear tire lateral force.

[0041] C2, determining, according to the vehicle estimated mass and the vehicle sensor information, a first system state equation, a first system measurement equation of a road adhesion coefficient estimation module based on the MMCC+SCKF algorithm and first variable information of the first system measurement equation; The road adhesion coefficient estimation device determines, according to the vehicle estimated mass and the vehicle sensor information, a first system state equation, a first system measurement equation of a road adhesion coefficient estimation module based on the MMCC+SCKF algorithm and first variable information of the first system measurement equation. The variable information includes a first state variable, a first observation variable and a first input variable; wherein the first state variable includes a road adhesion coefficient of the left wheel , a road adhesion coefficient of the right wheel , a longitudinal acceleration , a lateral acceleration and yaw angular velocity ; the first observation variables include longitudinal acceleration , lateral acceleration , and yaw angular velocity ; the first input variables include front wheel steering angle, longitudinal vehicle speed, lateral vehicle speed, and wheel speed.

[0042] Based on the first seven-degree-of-freedom vehicle double-track dynamics model, the SCKF is used to estimate the road slope, wherein the first system state equation of the nonlinear system is: ; wherein, is the state variable at time k; is the state variable at time k-1; is the input variable at time k-1; is the process noise at time k-1.

[0043] wherein the first system measurement equation is: ; wherein the first state variable is: ; wherein the first observation variable is: ; wherein the first input variable is: ; wherein, is the road adhesion coefficient of the left wheel; is the road adhesion coefficient of the right wheel; is the longitudinal acceleration; is the lateral acceleration; is the yaw angular velocity.

[0044] wherein, is the front wheel steering angle, is the longitudinal vehicle speed, is the lateral vehicle speed, is the wheel speed.

[0045] C3, the first seven-degree-of-freedom dynamics model, the first system state equation, the first system measurement equation, and the first variable information are iterated according to the MMCC+SCKF algorithm to obtain a second estimated road adhesion coefficient.

[0046] The road surface adhesion coefficient estimation device, after determining the first system state equation, the first system measurement equation and the first variable information, iterates the first seven-degree-of-freedom dynamic model, the first system state equation, the first system measurement equation and the first variable information according to the MMCC+SCKF algorithm to obtain the estimated road slope. That is, a maximum mixed covariance-square root cubature Kalman filter algorithm is constructed to be iterated to obtain the second estimated road surface adhesion coefficient.

[0047] In the MMCC+SCKF algorithm, the maximum mixed correlation entropy criterion is .

[0048] As an implementation, iterating the first seven-degree-of-freedom dynamic model, the first system state equation, the first system measurement equation and the first variable information according to the MMCC+SCKF algorithm to obtain the second estimated road surface adhesion coefficient can include: D1, initializing the first state variable, setting the initial value of the first process noise and the initial value of the first measurement noise covariance matrix, and calculating the square root factor of the first initial error covariance matrix; After the road surface adhesion coefficient estimation device determines the first system state equation, the first system measurement equation and the first variable information, the road surface adhesion coefficient estimation device initializes the first state variable , sets the initial value of the first process noise and the first measurement noise covariance matrix Q, R, and calculates the square root factor of the first state error covariance matrix . ; In the formula, Chol(·) is a Cholesky decomposition function.

[0049] D2, updating the time, and according to the updated first seven-degree-of-freedom dynamic model, the updated first system state equation, the updated first system measurement equation and the updated first variable information, obtaining the first state prediction value and the square root factor of the first error covariance matrix; After the road surface adhesion coefficient estimation device initializes the first state variable, the road surface adhesion coefficient estimation device updates the time. When the road surface adhesion coefficient estimation device updates the time, the road surface adhesion coefficient estimation device first calculates the cubature point and inputs it to the state transfer equation, such as: ; ; In the formula, n is the dimension of the state variable, is the i-th column of the cubature point weight matrix , and In is an n-order unit matrix.

[0050] Compute the first state prediction ; ; where the weighted central moment is defined as ; Further compute the square root factor of the first error covariance matrix ; where Tria(·) is the trace of a matrix.

[0051] D3, update the measurement, according to the updated first seven-degree-of-freedom dynamic model, the updated first system state equation, the updated first system measurement equation, the updated first variable information, and the first state prediction value and the square root factor of the first error covariance matrix, obtain the first measurement prediction value and the first Kalman gain; The road adhesion coefficient estimation device updates the measurement after the first state variable is initialized, according to the updated first seven-degree-of-freedom dynamic model, the updated first system state equation, the updated first system measurement equation, the updated first variable information, and the first state prediction value and the square root factor of the first error covariance matrix, obtains the first measurement prediction value and the first Kalman gain.

[0052] When updating the measurement, first update the volume points and input to the measurement equation: ; ; Secondly, compute the first measurement prediction value ; ; where the weighted central moment is defined as ; Thirdly, compute the square root factor of the first innovation covariance matrix: ; Fourthly, compute the first measurement covariance matrix and the first cross-covariance matrix: The first measurement covariance matrix ; The first cross-covariance matrix ; where the weighted central moment is defined as ; ​​​Finally, the first Kalman gain is calculated from the first measurement covariance matrix and the first cross-covariance matrix : .

[0053] D4, obtaining the second estimated road adhesion coefficient according to the first state prediction value, the first measurement prediction value, the first measurement value and the first Kalman gain.

[0054] The road adhesion coefficient estimation device obtains the second estimated road adhesion coefficient according to the first state prediction value, the first measurement prediction value, the first measurement value and the first Kalman gain after obtaining the first state prediction value, the first measurement prediction value, the first measurement value and the first Kalman gain.

[0055] The road adhesion coefficient estimation device estimates the road adhesion coefficient to obtain the second estimated road adhesion coefficient after updating the time to obtain the first state prediction value and updating the measurement to obtain the first measurement prediction value and the first Kalman gain and obtaining the first measurement value at the k time .

[0056] That is, the first state variable of the posteriori estimation can be obtained according to the first state prediction value , the first measurement prediction value , the first measurement value and the first Kalman gain , and the specific expression is: ; The updated posteriori first error covariance square root factor is: .

[0057] Wherein, is the first prediction value of the k time state at the k-1 time, is the Kalman gain matrix at the k time, is the true value of the observation at the k time, is the first prediction value of the k time observation variable at the k-1 time.

[0058] Therefore, the first state variable can be obtained through the adaptive covariance matrix of the last time as the second estimated road adhesion coefficient and the first process noise covariance matrix .

[0059] As an implementation form, the updating of the measurement, according to the updated first seven-degree-of-freedom dynamic model, the updated first system state equation, the updated first system measurement equation, the updated first variable information, and the first state prediction value and the first error covariance matrix square root factor, to obtain the first measurement prediction value and the first Kalman gain, can include: the updating of the measurement, according to the updated first seven-degree-of-freedom dynamic model, the updated first system state equation, the updated first system measurement equation, the updated first variable information, and the first state prediction value and the first error covariance matrix square root factor, to obtain the first measurement prediction value and the first innovation covariance matrix square root factor; according to the first innovation covariance matrix square root factor, obtaining the first measurement covariance matrix and the first cross-covariance matrix; according to the first measurement covariance matrix and the first cross-covariance matrix, obtaining the first Kalman gain.

[0060] 205, fusing the first estimated road adhesion coefficient and the second estimated road adhesion coefficient to obtain a road adhesion estimation coefficient.

[0061] The road adhesion coefficient estimation device fuses the first estimated road adhesion coefficient and the second estimated road adhesion coefficient to obtain a road adhesion estimation coefficient after obtaining the first estimated road adhesion coefficient and the second estimated road adhesion coefficient.

[0062] As an implementation form, the first estimated road adhesion coefficient includes a current frame adhesion coefficient recognition result, an image category adhesion coefficient upper limit, an image category adhesion coefficient lower limit, and an image confidence, and the second estimated road adhesion coefficient includes a left wheel road adhesion coefficient, a right wheel road adhesion coefficient, a longitudinal acceleration, and a lateral acceleration; and the fusing of the first estimated road adhesion coefficient and the second estimated road adhesion coefficient to obtain a road adhesion estimation coefficient includes: F1, obtaining an excitation level according to the longitudinal acceleration and the lateral acceleration; The road adhesion coefficient estimation device obtains an excitation level according to the longitudinal acceleration and the lateral acceleration after obtaining the longitudinal acceleration and the lateral acceleration.

[0063] F2, obtaining a road adhesion estimation coefficient according to the front frame adhesion coefficient recognition result, the image category adhesion coefficient upper limit, the image category adhesion coefficient lower limit, the image confidence, the left wheel road adhesion coefficient, the right wheel road adhesion coefficient, and the excitation level.

[0064] The road surface adhesion coefficient estimation device obtains the road surface adhesion estimation coefficient according to the previous frame adhesion coefficient recognition result, the image category adhesion coefficient upper limit, the image category adhesion coefficient lower limit, the image confidence, the left wheel road surface adhesion coefficient, the right wheel road surface adhesion coefficient and the excitation level after obtaining the previous frame adhesion coefficient recognition result, the image category adhesion coefficient upper limit, the image category adhesion coefficient lower limit, the image confidence, the left wheel road surface adhesion coefficient, the right wheel road surface adhesion coefficient and the excitation level.

[0065] As an implementation form, when the road surface adhesion coefficient of the left wheel is fused, the road surface adhesion estimation coefficient is obtained according to the previous frame adhesion coefficient recognition result, the image category adhesion coefficient upper limit, the image category adhesion coefficient lower limit, the image confidence, the left wheel road surface adhesion coefficient, the right wheel road surface adhesion coefficient and the excitation level, including: If , then ; wherein, is the left wheel road surface adhesion estimation coefficient, is the output road surface adhesion coefficient, is the image confidence, is the confidence threshold, is the image category adhesion coefficient lower limit, is the image category adhesion coefficient upper limit; is the excitation level; is the excitation threshold; is the left wheel road surface adhesion coefficient; If , then ; is the previous frame adhesion coefficient recognition result; If , then ; If , then ; is the road surface adhesion estimation coefficient of the previous moment; If , then ; is the fourth road surface adhesion coefficient obtained by updating the upper and lower bounds of the image recognition result as the constraint condition of MMCC+SCKF through the projection constraint method and the pseudo-measurement update.

[0066] As an implementation form, when the road surface adhesion coefficient of the right wheel is fused, the road surface adhesion estimation coefficient is obtained according to the previous frame adhesion coefficient recognition result, the image category adhesion coefficient upper limit, the image category adhesion coefficient lower limit, the image confidence, the left wheel road surface adhesion coefficient, the right wheel road surface adhesion coefficient and the excitation level, including: If , then ; wherein, is a right wheel road adhesion estimation coefficient, is an output road adhesion coefficient, is an image confidence, is a confidence threshold, is a lower bound of image category adhesion coefficient, is an upper bound of image category adhesion coefficient; is an excitation level; is an excitation threshold; is a right wheel road adhesion coefficient; if , then ; is a previous frame adhesion coefficient recognition result; if , then ; if , then ; is a previous time road adhesion estimation coefficient; if , then ; is a fourth road adhesion coefficient obtained by treating the upper and lower bounds based on the image recognition result as a constraint condition of MMCC+SCKF through projection constraint and pseudo-measurement update.

[0067] wherein the third road adhesion coefficient obtained by treating the upper and lower bounds based on the image recognition result as a constraint condition of MMCC+SCKF through projection constraint and pseudo-measurement update is a third road adhesion coefficient . The specific calculation and update process of the third road adhesion coefficient is as follows: First, the boundary of RAC is dynamically adjusted according to the confidence based on the image recognition result, so as to solve the uncertainty fluctuation of the visual-based method when the boundary is uniformly set.

[0068] The adaptive boundary can be expressed as: ; In the above formula, controls the coverage probability, such as when is 2, corresponding to a confidence interval covering 95%, is a Top-1 probability, which can be expressed as .

[0069] The constraint provided based on image recognition is an inequality constraint, which can be expressed as: ; The result estimated based on the MMCC+SCKF method is not constrained, and the unconstrained estimation result is projected ; After projection, the covariance is adjusted by the pseudo-measurement method to maintain the consistency of the covariance while reflecting the certainty enhancement after the introduction of the constraint. When exceeding the upper and lower boundaries, a virtual measurement is introduced ; ; Construct the Jacobian matrix ; Calculate the Kalman gain with state constraints ; Where the error covariance matrix .

[0070] Update the state vector and covariance with state constraints ; ; Update the posterior error covariance square root factor with state constraints ; Thus, through the adaptive covariance matrix of the last time, the state variable as the third road adhesion coefficient and process noise covariance matrix.

[0071] As an embodiment, the processing of the vehicle mass estimation, the first estimated road adhesion coefficient and vehicle sensor information by the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient can include: G1, according to the vehicle mass estimation and the vehicle sensor information, a second seven-degree-of-freedom vehicle double-track dynamics model is established; The road adhesion coefficient estimation device, after determining the vehicle mass estimation and obtaining the vehicle sensor information, establishes a two-seven-degree-of-freedom vehicle double-track dynamics model according to the vehicle mass estimation and the vehicle sensor information; The estimated mass obtained by the mass estimation module As a known parameter, an accurate seven-degree-of-freedom vehicle double-track dynamics model is established, including seven degrees of freedom of longitudinal, lateral, yaw and four wheel rolling of the vehicle. The dynamic equations of longitudinal motion, lateral motion and yaw motion are: ; ; ; wherein, is the moment of inertia of the whole vehicle around the Z axis; is the longitudinal acceleration at the center of mass; is the lateral acceleration at the center of mass; is the longitudinal velocity; is the lateral velocity; is the longitudinal acceleration; is the lateral acceleration; is the yaw rate, is the yaw moment.

[0072] wherein the vehicle longitudinal acceleration , the vehicle lateral acceleration , the vehicle yaw moment , the tire longitudinal force and the tire lateral force are related by: ; ; ; wherein, is the distance from the center of mass of the vehicle to the front axle; is the distance from the center of mass of the vehicle to the rear axle, is the track of the front axle; is the track of the rear axle; is the front wheel steering angle input, is the left front tire longitudinal force; is the right front tire longitudinal force; is the left front tire lateral force; is the right front tire lateral force; is the left rear tire longitudinal force; is the right rear tire longitudinal force; is the left rear tire lateral force; is the right rear tire lateral force.

[0073] G2, determines, based on the vehicle estimated mass and the vehicle sensor information, a second system state equation of a road adhesion coefficient estimation module based on an MMCC+SCKF algorithm, a second system measurement equation, and second variable information of the second system measurement equation; The road adhesion coefficient estimation device determines the vehicle estimated mass and after obtaining the vehicle sensor information, a second system state equation, a second system measurement equation and second variable information of the second system measurement equation of a road adhesion coefficient estimation module based on an MMCC+SCKF algorithm are determined according to the vehicle estimated mass and the vehicle sensor information. The variable information includes a second state variable, a second observation variable and a second input variable; wherein the second state variable includes a road adhesion coefficient of a left wheel , a road adhesion coefficient of a right wheel , a longitudinal acceleration , a lateral acceleration and a yaw rate ; the second observation variable includes a longitudinal acceleration , a lateral acceleration and a yaw rate ; and the second input variable includes a front wheel steering angle, a longitudinal vehicle speed, a lateral vehicle speed and a wheel speed.

[0074] Based on the second seven-degree-of-freedom vehicle double-track dynamics model, the SCKF is used to estimate the road slope, wherein the second system state equation of the nonlinear system is: ; wherein is a state variable at time k; is a state variable at time k-1; is an input variable at time k-1; is process noise at time k-1.

[0075] wherein the second system measurement equation is: ; wherein the second state variable is: ; wherein the second observation variable is: ; wherein the second input variable is: ; wherein is a road adhesion coefficient of a left wheel; is a road adhesion coefficient of a right wheel; is a longitudinal acceleration; is a lateral acceleration; is a yaw rate.

[0076] wherein is a front wheel steering angle, is a longitudinal vehicle speed, is a lateral vehicle speed, is a wheel speed. The wheel speed includes a left front wheel speed , a right front wheel speed left rear wheel speed and right rear wheel speed .

[0077] G3, taking the first estimated road adhesion coefficient as the initial state estimate, iterates the second seven-degree-of-freedom dynamic model, the second system state equation, the second system measurement equation, and the second variable information according to the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient.

[0078] After the first estimated road adhesion coefficient, the second system state equation, the second system measurement equation, and the second variable information are determined, the road adhesion coefficient estimation device takes the second estimated road adhesion coefficient as the initial state estimate, iterates the second seven-degree-of-freedom dynamic model, the second system state equation, the second system measurement equation, and the second variable information according to the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient.

[0079] In the MMCC+SCKF algorithm, the maximum mixed correlation entropy criterion is .

[0080] As an implementation mode, taking the first estimated road adhesion coefficient as the initial state estimate, iterates the second seven-degree-of-freedom dynamic model, the second system state equation, the second system measurement equation, and the second variable information according to the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient, which can include: H1, taking the first estimated road adhesion coefficient as the second state variable initialization value, setting the initial value of the second process noise and the initial value of the second measurement noise covariance matrix, calculating the square root factor of the second initial error covariance matrix; After the first estimated road adhesion coefficient, the second system state equation, the second system measurement equation, and the second variable information are determined, the road adhesion coefficient estimation device takes the first estimated road adhesion coefficient as the second state variable initialization value (that is, the initial state estimate), sets the initial value of the second process noise and the second measurement noise covariance matrix Q, R, and calculates the square root factor of the second state error covariance matrix ; In the formula, Chol(·) is the Cholesky decomposition function.

[0081] H2, updates the time, and according to the updated second seven-degree-of-freedom dynamic model, the updated second system state equation, the updated second system measurement equation, and the updated second variable information, obtains the second state prediction value and the square root factor of the second error covariance matrix; The road adhesion coefficient estimation device is time-updated after the second state variable is initialized. When the road adhesion coefficient estimation device is time-updated, the volume point is first calculated and input to the state transition equation, as follows: ; ; In the formula, n is the dimension of the state variable, is the i-th column of the volume point weight matrix , and In is an n-order unit matrix.

[0082] The second state prediction value is calculated ; ; wherein the weighted central moment is defined as ; The square root factor of the second error covariance matrix is further calculated ; wherein Tria(·) is the trace of a matrix.

[0083] H3, the measurement is updated, and the second measurement prediction value and the second Kalman gain are obtained according to the updated second seven-degree-of-freedom dynamic model, the updated second system state equation, the updated second system measurement equation, the updated second variable information, and the second state prediction value and the square root factor of the second error covariance matrix. The road adhesion coefficient estimation device is time-updated after the second state variable is initialized. When the road adhesion coefficient estimation device is time-updated, the volume point is first calculated and input to the state transition equation, as follows:

[0084] When the measurement is updated, the volume point is first updated and input to the measurement equation: ; ; Secondly, the second measurement prediction value ; ; wherein the weighted central moment is defined as ; Thirdly, the square root factor of the second innovation covariance matrix is calculated: ; Again, the second measurement covariance matrix and the second cross covariance matrix are calculated: The second measurement covariance matrix ; The second cross covariance matrix ; where the weighted center moment is: ; Finally, the second Kalman gain is calculated from the second measurement covariance matrix and the second cross covariance matrix: .

[0085] H4, obtains the estimated road adhesion coefficient according to the second state prediction value, the second measurement prediction value, the second measurement value and the second Kalman gain.

[0086] The road adhesion coefficient estimation device obtains the estimated road adhesion coefficient according to the second state prediction value, the second measurement prediction value, the second measurement value and the second Kalman gain after obtaining the second state prediction value, the second measurement prediction value, the second measurement value and the second Kalman gain.

[0087] The road adhesion coefficient estimation device estimates the road adhesion coefficient after updating the time to obtain the second state prediction value , updating the measurement to obtain the second measurement prediction value and the second Kalman gain , and obtaining the second measurement value at the k time .

[0088] That is, the second state variable of the posteriori estimation can be obtained according to the second state prediction value , the second measurement prediction value , the second measurement value and the second Kalman gain , and the specific expression is: ; The posteriori second error covariance square root factor is finally updated as: .

[0089] wherein is the second prediction value of the k time state at the k-1 time, is the Kalman gain matrix at the k time, is the true value of the observation at the k time, is a second prediction value of the observation variable at time k for time k+1.

[0090] is a second state variable at time k+1 obtained by the adaptive covariance matrix at time k. is an estimated road adhesion coefficient and a second process noise covariance matrix .

[0091] As an embodiment, the updating of the measurement, according to the updated second seven-degree-of-freedom dynamic model, the updated second system state equation, the updated second system measurement equation, the updated second variable information, and the second state prediction value and the square root factor of the second error covariance matrix, obtains the second measurement prediction value and the second Kalman gain, can include: updating the measurement, according to the updated second seven-degree-of-freedom dynamic model, the updated second system state equation, the updated second system measurement equation, the updated second variable information, and the second state prediction value and the square root factor of the second error covariance matrix, obtaining the second measurement prediction value and the square root factor of the second innovation covariance matrix; according to the square root factor of the second innovation covariance matrix, obtaining the second measurement covariance matrix and the second cross-covariance matrix; according to the second measurement covariance matrix and the second cross-covariance matrix, obtaining the second Kalman gain.

[0092] Based on the multi-source sensor data obtained by the information layer, the BiLSTM network is used to realize the accurate estimation of the vehicle mass, and the EfficientNetV2 network is used to process the front road surface image, so as to obtain the prior estimation value of the road adhesion coefficient, and provide reliable initial information for subsequent state decoupling. The tire model and the seven-degree-of-freedom vehicle dynamics model are introduced at the model end, and the maximum mixed correlation entropy criterion square root cubature Kalman filtering algorithm (MMCC+SCKF) is combined, which effectively suppresses the interference of non-Gaussian noise on the estimation of the road adhesion coefficient, and significantly improves the robustness and convergence of the algorithm under complex working conditions. Through the design of the space-time synchronization and fusion strategy, the estimation range of the prospective road adhesion coefficient obtained in the data-driven module is dynamically fused with the adhesion coefficient estimation value based on the dynamics model in the model-driven module, to realize the stable and reliable estimation of the road state. The hybrid-driven software architecture effectively improves the precision, stability and adaptability of the vehicle-road state parameter joint estimation, and provides solid data support and safety guarantee for intelligent chassis control system and advanced automatic driving decision.

[0093] In summary, the application estimates the mass of the vehicle, the first estimated road adhesion coefficient and the vehicle sensor information by the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient. The tire model and the seven-degree-of-freedom vehicle dynamics model are introduced in the adhesion coefficient determination module, combined with the maximum mixed correlation entropy criterion square root cubage Kalman filter algorithm (MMCC+SCKF), which effectively suppresses the interference of non-Gaussian noise on the road adhesion coefficient estimation, and improves the accuracy and robustness of the road adhesion coefficient estimation.

[0094] Figure 2 The flowchart of the first embodiment of the vehicle state estimation method provided by the application is shown, which is an example and is not limited to the application. As shown in the figure, the method can be applied to the vehicle state estimation device described above. As shown in the figure, the method can include: Figure 2 301, obtaining vehicle sensor information and vehicle front road image; In order to improve the accuracy and robustness of vehicle state estimation, the vehicle state estimation device obtains vehicle sensor information through vehicle IMU sensor or vehicle wheel speed sensor. That is, the vehicle state estimation device obtains the left front wheel speed , right front wheel speed , left rear wheel speed , right rear wheel speed , longitudinal acceleration , lateral acceleration , vertical acceleration , jerk of longitudinal acceleration , jerk of vertical acceleration , yaw rate , pitch rate, front wheel steering angle , longitudinal vehicle speed , lateral vehicle speed , roll rate, longitudinal acceleration at the center of mass , lateral acceleration at the center of mass , yaw rate , yaw moment , engine torque , engine speed n, left front wheel tire longitudinal force , right front wheel tire longitudinal force , left front wheel tire lateral force , right front wheel tire lateral force , left rear wheel tire longitudinal force , right rear wheel tire longitudinal force , left rear wheel tire lateral force , and right rear wheel tire lateral force ​The vehicle state estimation device also obtains a vehicle front road image through a camera arranged at a driving position of the vehicle; that is, the vehicle state estimation device mainly processes picture information from a camera sensor picture information through an information layer in an automatic driving domain, and performs picture cutting along a future wheel track. The vehicle sensor information can be subjected to low-pass filtering, denoising and time synchronization, so as to ensure that the vehicle state parameters output by the information layer to the state estimation layer have high precision, low time delay and consistency, and provide data for a subsequent state estimation module.

[0095] 302, inputting the vehicle sensor information into a BiLSTM model to obtain vehicle estimation quality; After obtaining the vehicle sensor information, the vehicle state estimation device inputs the vehicle sensor information into a BiLSTM model to obtain vehicle estimation quality.

[0096] 303, determining a road category at a current time through processing of the vehicle front road image, and obtaining a first estimated road adhesion coefficient according to a mapping relationship between the road category and the road adhesion coefficient; After obtaining the vehicle front road image, the vehicle state estimation device determines a road category at a current time through processing of the vehicle front road image, and obtains a first estimated road adhesion coefficient according to a mapping relationship between the road category and the road adhesion coefficient. After obtaining the vehicle front road image, the vehicle state estimation device performs road adhesion coefficient range estimation and image classification confidence calculation based on an EffcientNetV2.

[0097] 304, processing the vehicle estimation quality, the first estimated road adhesion coefficient and the vehicle sensor information through an MMCC+SCKF algorithm to obtain an estimated road adhesion coefficient.

[0098] After obtaining the vehicle estimation quality, obtaining the first estimated road adhesion coefficient and obtaining the vehicle sensor information, the vehicle state estimation device processes the vehicle estimation quality, the first estimated road adhesion coefficient and the vehicle sensor information through an MMCC+SCKF (Maximum Mixed Covariance-Square Root Cubature Kalman Filter) algorithm to obtain an estimated road adhesion coefficient.

[0099] That is, the vehicle estimation quality obtained by the quality estimation module , the first estimated road adhesion coefficient obtained by the image adhesion coefficient estimation module and the vehicle sensor information obtained by the information layer are taken as inputs to construct a vehicle state estimation module based on the maximum mixed covariance-square root cubature Kalman filter, to estimate the road adhesion coefficient and obtain an estimated road adhesion coefficient. The estimated road adhesion coefficient includes a left wheel road adhesion coefficient and a right wheel road adhesion coefficient .

[0100] 305, processing the vehicle estimation mass, the vehicle sensor information and the road adhesion estimation coefficient by the MMCC+SCKF algorithm to obtain vehicle state estimation information.

[0101] The vehicle state estimation information can be obtained by simulating calculation through a vehicle state estimation device; As an embodiment, the vehicle state estimation information is obtained by processing the vehicle estimation mass, the vehicle sensor information and the road adhesion estimation coefficient by the MMCC+SCKF algorithm, comprising: K1, establishing a third seven-degree-of-freedom dynamic model according to the vehicle estimation mass and the vehicle sensor information; The vehicle state estimation device establishes a third seven-degree-of-freedom dynamic model according to the vehicle estimation mass and the vehicle sensor information after determining the vehicle estimation mass and obtaining the vehicle sensor information; The estimation mass obtained by the mass estimation module As a known parameter, an accurate seven-degree-of-freedom vehicle double-track dynamic model is established, including seven degrees of freedom of longitudinal, lateral, yaw and four wheel rolling of the vehicle. The dynamic equations of longitudinal motion, lateral motion and yaw motion are: ; ; ; Wherein, is the moment of inertia of the whole vehicle around the Z axis; is the longitudinal acceleration at the center of mass; is the lateral acceleration at the center of mass; is the longitudinal velocity; is the lateral velocity is the longitudinal acceleration; is the lateral acceleration; is the yaw angular velocity, is the yaw moment.

[0102] Wherein, the vehicle longitudinal acceleration , the lateral acceleration , the yaw moment , the tire longitudinal force and the tire lateral force between them are: ; ; ; In the formula, This is the distance from the vehicle's center of gravity to the front axle. This is the distance from the vehicle's center of gravity to the rear axle. The track width of the front axle; The track width of the rear axle; Input the front wheel steering angle. The longitudinal force on the left front tire; The longitudinal force on the right front tire; The lateral force on the left front tire; The lateral force on the right front tire; The longitudinal force on the left rear tire; The longitudinal force on the right rear tire; The lateral force on the left rear tire; This refers to the lateral force on the right rear tire.

[0103] K2, based on the estimated vehicle mass, the estimated road surface adhesion coefficient, and the vehicle sensor information, determines the third system state equation, the third system measurement equation, and the third variable information of the third system measurement equation for the vehicle state estimation module based on the MMCC+SCKF algorithm; The vehicle condition estimation device determines the estimated vehicle mass. After obtaining the road surface adhesion estimation coefficient and vehicle sensor information, the third system state equation, third system measurement equation, and third system measurement equation information of the vehicle state estimation module based on the MMCC+SCKF algorithm are determined according to the estimated vehicle mass, the road surface adhesion estimation coefficient, and the vehicle sensor information. The third state variable includes longitudinal velocity. lateral velocity , centroid side slip angle Longitudinal acceleration lateral acceleration and yaw rate The third observed variable includes longitudinal acceleration. lateral acceleration and centroid sideslip angle The third input variable includes the front wheel steering angle. Left front wheel speed Right front wheel speed Left rear wheel speed Right rear wheel speed Left wheel road adhesion coefficient and the road surface adhesion coefficient of the right wheel .

[0104] Based on this third-degree-of-freedom vehicle dual-track dynamics model, SCKF is used to estimate the road gradient. The state equation of the third system in the nonlinear system is: ; wherein, is the state variable at time k; is the state variable at time k-1; is the input variable at time k-1; is the process noise at time k-1.

[0105] wherein, the third system measurement equation is: ; wherein the third state variable is: ; wherein the third observation variable is: ; wherein the third input variable is: ; wherein, is the road adhesion coefficient of the left wheel; is the road adhesion coefficient of the right wheel; is the mass center side slip angle; is the longitudinal acceleration; is the lateral acceleration; is the yaw rate.

[0106] wherein, is the front wheel steering angle, is the left front wheel speed, is the right front wheel speed, is the left rear wheel speed, is the right rear wheel speed.

[0107] K3, according to the MMCC+SCKF algorithm, iterates the third seven-DOF dynamics model, the third system state equation, the third system measurement equation and the third variable information to obtain vehicle state estimation information.

[0108] After the third system state equation, the third system measurement equation and the third variable information are determined, according to the MMCC+SCKF algorithm, the third seven-DOF dynamics model, the third system state equation, the third system measurement equation and the third variable information are iterated to obtain vehicle state estimation information. That is, the maximum mixed covariance-square root cubature Kalman filter algorithm is constructed to iterate to obtain vehicle state estimation information. The vehicle state estimation information includes mass center side slip angle , longitudinal acceleration , lateral acceleration and yaw rate .

[0109] wherein, the maximum mixed correlation entropy criterion in the MMCC+SCKF algorithm is .

[0110] As an implementation, the third seven-degree-of-freedom dynamics model, the third system state equation, the third system measurement equation and the third variable information are iterated according to the MMCC+SCKF algorithm to obtain vehicle state estimation information, which can include: L1, the third state variable is initialized, the initial value of the third process noise and the initial value of the third measurement noise covariance matrix are set, and the square root factor of the third initial error covariance matrix is calculated; After the vehicle state estimation device determines the third system state equation, the third system measurement equation and the third variable information, the third state variable is initialized, the initial value of the third process noise and the third measurement noise covariance matrix Q, R is set, and the square root factor of the third state error covariance matrix is calculated ; In the formula, Chol(·) is the Cholesky decomposition function.

[0111] L2, the time is updated, and the third state prediction value and the square root factor of the third error covariance matrix are obtained according to the updated third seven-degree-of-freedom dynamics model, the updated third system state equation, the updated third system measurement equation and the updated third variable information; After the vehicle state estimation device initializes the third state variable, the vehicle state estimation device updates the time. When the vehicle state estimation device updates the time, the volume point is calculated and input to the state transfer equation, such as: ; ; In the formula, n is the dimension of the state variable, is the i-th column of the volume point weight matrix , and In is an n-order unit matrix.

[0112] The third state prediction value is calculated; ; Wherein, the weighted central moment is defined as ; The square root factor of the third error covariance matrix is further calculated ; Wherein, Tria(·) is the trace of the matrix.

[0113] L3, updating the measurement, according to the updated third seven-degree-of-freedom dynamic model, the updated third system state equation, the updated third system measurement equation, the updated third variable information, and the third state prediction value and the third error covariance matrix square root factor, obtaining the third measurement prediction value and the third Kalman gain; The vehicle state estimation device, after initializing the third state variable, updates the measurement, according to the updated third seven-degree-of-freedom dynamic model, the updated third system state equation, the updated third system measurement equation, the updated third variable information, and the third state prediction value and the third error covariance matrix square root factor, obtaining the third measurement prediction value and the third Kalman gain.

[0114] When updating the measurement, first, update the volume points and input them to the measurement equation: ; ; Secondly, calculate the third measurement prediction value ; ; Wherein, the weighted central moment is: ; Thirdly, calculate the square root factor of the third innovation covariance matrix: ; Fourthly, calculate the third measurement covariance matrix and the third cross-covariance matrix: The third measurement covariance matrix ; The third cross-covariance matrix ; Wherein, the weighted central moment is: ; Finally, calculate the third Kalman gain from the third measurement covariance matrix and the third cross-covariance matrix: .

[0115] L4, according to the third state prediction value, the third measurement prediction value, the third measurement value and the third Kalman gain, obtaining the vehicle state estimation information.

[0116] The vehicle state estimation device, after obtaining the third state prediction value, the third measurement prediction value, the third measurement value and the third Kalman gain, according to the third state prediction value, the third measurement prediction value, the third measurement value and the third Kalman gain, obtaining the vehicle state estimation information.

[0117] The vehicle state estimation device obtains a third state prediction value after updating time and a third measurement prediction value after updating measurement and a third Kalman gain and obtains a third measurement value at time k After that, the vehicle state information is estimated to obtain vehicle state estimation information.

[0118] That is, the third state variable of the posteriori estimation can be obtained according to the third state prediction value , the third measurement prediction value , the third measurement value and the third Kalman gain , and the specific expression is: ; The last updated posteriori third error covariance square root factor is: .

[0119] Wherein, is the third prediction value of the state at time k-1, is the Kalman gain matrix at time k, is the true value of the observation at time k, is the third prediction value of the observation variable at time k-1.

[0120] Therefore, the third state variable can be obtained through the adaptive covariance matrix of the last time. The vehicle state estimation information and the third process noise covariance matrix are obtained. The vehicle state estimation information includes the mass center side slip angle , the longitudinal acceleration , the lateral acceleration and the yaw rate

[0121] As an implementation form, updating the measurement, according to the updated third seven-degree-of-freedom dynamics model, the updated third system state equation, the updated third system measurement equation, the updated third variable information, and the third state prediction value and the third error covariance matrix square root factor, to obtain the third measurement prediction value and the third Kalman gain, can include: updating the measurement, according to the updated third seven-degree-of-freedom dynamics model, the updated third system state equation, the updated third system measurement equation, the updated third variable information, and the third state prediction value and the third error covariance matrix square root factor, to obtain the third measurement prediction value and the third innovation covariance matrix square root factor; according to the third innovation covariance matrix square root factor, obtaining the third measurement covariance matrix and the third cross-covariance matrix; according to the third measurement covariance matrix and the third cross-covariance matrix, obtaining the third Kalman gain.

[0122] Based on the multi-source sensor data obtained from the information layer, the application realizes accurate estimation of the vehicle mass by using the BiLSTM network, processes the front road surface image by using the EfficientNetV2 network, thereby obtaining the prior estimation value of the road surface adhesion coefficient, and provides reliable initial information for subsequent state decoupling. The tire model and the seven-degree-of-freedom vehicle dynamics model are introduced at the model end, and the maximum mixed correlation entropy criterion square root cubature Kalman filtering algorithm (MMCC+SCKF) is combined, which effectively suppresses the interference of non-Gaussian noise on the estimation of the road surface adhesion coefficient, and significantly improves the robustness and convergence of the algorithm under complex working conditions. Through the design of the space-time synchronization and fusion strategy, the estimation range of the prospective road surface adhesion coefficient obtained in the data-driven module is dynamically fused with the adhesion coefficient estimation value based on the dynamics model in the model-driven module, to realize stable and reliable estimation of the road surface state.

[0123] Based on the estimated vehicle mass and adhesion coefficient information, the application jointly estimates the key dynamic state parameters such as the vehicle longitudinal speed, lateral speed, body roll angle, and mass center side slip angle by using the Kalman filtering model, to ensure that the system still has good stability and reliability under multiple working conditions and strong disturbance environment. The hybrid-driven software architecture effectively improves the precision, stability, and adaptability of the vehicle-road state parameter joint estimation, and provides solid data support and safety guarantee for the intelligent chassis control system and high-level automatic driving decision-making.

[0124] The application also provides a terminal device, which comprises at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in the road surface adhesion coefficient estimation method embodiment when executing the computer program, or the processor implements the steps in the vehicle state estimation method embodiment when executing the computer program.

[0125] The embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to execute part or all of steps of any one of the road adhesion coefficient estimation methods described in the above method embodiments, or causes the computer to execute part or all of steps of any one of the vehicle state estimation methods described in the above method embodiments.

[0126] The embodiment of the present application further provides a computer program product, which comprises a non-transitory computer-readable storage medium storing a computer program, and the computer program causes a computer to execute part or all of steps of any one of the road adhesion coefficient estimation methods described in the above method embodiments, or causes the computer to execute part or all of steps of any one of the vehicle state estimation methods described in the above method embodiments.

Claims

1. A method of estimating a road adhesion coefficient, characterized by, The method comprises the following steps: acquiring vehicle sensor information and a road image in front of the vehicle; inputting the vehicle sensor information into a BiLSTM model to obtain a vehicle estimated mass; determining a road category at a current time by processing the road image in front of the vehicle, and obtaining a first estimated road adhesion coefficient according to a mapping relationship between the road category and the road adhesion coefficient; processing the vehicle estimated mass, the first estimated road adhesion coefficient and the vehicle sensor information by an MMCC+SCKF algorithm to obtain an estimated road adhesion coefficient.

2. The road adhesion coefficient estimation method according to claim 1, characterized by, The method of processing the vehicle estimated mass, the first estimated road adhesion coefficient and the vehicle sensor information by the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient comprises the following steps: processing the vehicle estimated mass and the vehicle sensor information by the MMCC+SCKF algorithm to obtain a second estimated road adhesion coefficient; fusing the first estimated road adhesion coefficient and the second estimated road adhesion coefficient to obtain a road adhesion estimation coefficient.

3. The road adhesion coefficient estimation method according to claim 2, characterized by, The method of processing the vehicle estimated mass and the vehicle sensor information by the MMCC+SCKF algorithm to obtain the second estimated road adhesion coefficient comprises the following steps: establishing a first seven-degree-of-freedom dynamic model according to the vehicle estimated mass and the vehicle sensor information; determining a first system state equation, a first system measurement equation and first variable information of the first system measurement equation of a road adhesion coefficient estimation module based on the MMCC+SCKF algorithm according to the vehicle estimated mass and the vehicle sensor information; iterating the first seven-degree-of-freedom dynamic model, the first system state equation, the first system measurement equation and the first variable information according to the MMCC+SCKF algorithm to obtain the second estimated road adhesion coefficient.

4. The road adhesion coefficient estimation method according to claim 3, characterized by, The first estimated road adhesion coefficient comprises a current frame adhesion coefficient recognition result, an image category adhesion coefficient upper limit, an image category adhesion coefficient lower limit and an image confidence level, and the second estimated road adhesion coefficient comprises a left wheel road adhesion coefficient, a right wheel road adhesion coefficient, a longitudinal acceleration and a lateral acceleration; the method of fusing the first estimated road adhesion coefficient and the second estimated road adhesion coefficient to obtain the road adhesion estimation coefficient comprises the following steps: obtaining an excitation level according to the longitudinal acceleration and the lateral acceleration; obtaining the road adhesion estimation coefficient according to the current frame adhesion coefficient recognition result, the image category adhesion coefficient upper limit, the image category adhesion coefficient lower limit, the image confidence level, the left wheel road adhesion coefficient, the right wheel road adhesion coefficient and the excitation level.

5. The road adhesion coefficient estimation method according to claim 1, characterized by, The method of processing the vehicle estimated mass, the first estimated road adhesion coefficient and the vehicle sensor information by the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient further comprises the following steps: establishing a second seven-degree-of-freedom vehicle double-track dynamic model according to the vehicle estimated mass and the vehicle sensor information; determining a second system state equation, a second system measurement equation of a road adhesion coefficient estimation module based on an MMCC+SCKF algorithm and second variable information of the second system measurement equation according to the vehicle estimated mass and the vehicle sensor information; taking the first estimated road adhesion coefficient as an initial state estimation, iteratively processing the second seven-degree-of-freedom dynamic model, the second system state equation, the second system measurement equation and the second variable information according to the MMCC+SCKF algorithm to obtain the estimated road adhesion coefficient.

6. The road adhesion coefficient estimation method according to any one of claims 1 to 4, wherein the vehicle speed information and the vehicle dynamic information are input into the BiLSTM model to obtain the vehicle estimated mass, and the method comprises: obtaining the vehicle speed from the vehicle sensor information; performing correlation coefficient analysis on the vehicle speed, the vehicle speed information and the vehicle dynamic information to determine a feature input vector; inputting the feature input vector into a full connection layer of the BiLSTM model to obtain the vehicle estimated mass.

7. The road adhesion coefficient estimation method according to any one of claims 1 to 4, characterized by, The first estimated road adhesion coefficient is determined by processing the road image in front of the vehicle to determine the road category at the current time, and according to the mapping relationship between the road category and the road adhesion coefficient, and the method comprises: obtaining an adhesion mapping relationship between the road type and the road adhesion coefficient range; using the time sequence correlation in the road image in front of the vehicle, introducing a Gaussian weighted sliding window mechanism on the Softmax output probability vector of the last layer of the EfficientNetV2 network to determine a road type probability vector and an image confidence; processing the road type probability vector based on the confidence adaptive Gaussian weighted sliding window to obtain a final output category at the current time; determining the first estimated road adhesion coefficient corresponding to the final output category at the current time in the adhesion mapping relationship according to the final output category at the current time.

8. A vehicle state estimation method characterized by comprising: The vehicle state estimation method further comprises: processing the vehicle estimated mass, the vehicle sensor information and the road adhesion estimation coefficient by the MMCC+SCKF algorithm to obtain vehicle state estimation information.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the road adhesion coefficient estimation method according to any one of claims 1 to 7, or the processor executes the computer program to implement the vehicle state estimation method according to claim 8.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The processor executes the computer program to implement the road adhesion coefficient estimation method according to any one of claims 1 to 7, or the processor executes the computer program to implement the vehicle state estimation method according to claim 8.

Images