Road surface adhesion coefficient estimation method and device, vehicle and storage medium

Abstract

The application discloses a road surface adhesion coefficient estimation method and device, a vehicle and a storage medium, wherein the method comprises the following steps: collecting image information of a current road surface; identifying an actual road surface type of the current road surface based on the image information, and matching a corresponding road surface adhesion coefficient range based on the actual road surface type; estimating the road surface adhesion coefficient of the current road surface through a pre-constructed vehicle dynamics model, and obtaining a final road surface adhesion coefficient estimation result of the current road surface in combination with the road surface adhesion coefficient range and a fusion result of the road surface adhesion coefficient. Thus, the technical problem that in the prior art, the road surface adhesion coefficient needs to be further obtained by installing a sensor, it is difficult to be applied to complex working conditions, and the calculation amount of the object detection network structure is huge, and it is difficult to meet the real-time detection requirement of automobile vision is solved.

Description

Technical Field

[0001] This application relates to the field of vehicle dynamics control technology, and in particular to a method, device, vehicle, and storage medium for estimating road surface adhesion coefficient. Background Technology

[0002] With the rapid development of society and economy, people's demand for transportation is increasing, and the number of motor vehicles is also increasing. While enjoying the convenience that cars bring to work and life, people are also demanding higher and higher active safety from vehicles. As a result, ADAS (Advanced Driving Assistance System) has emerged. Its products include AEB (Autonomous Emergency Braking), ACC (Adaptive Cruise Control), and ABS (Anti-skid Braking System). The road surface adhesion coefficient is an important parameter in vehicle dynamics control. Many ADAS products require the road surface adhesion coefficient as a parameter for calculation. Therefore, obtaining the road surface adhesion coefficient quickly and accurately is of great significance to the safety performance of vehicles.

[0003] The different road surface adhesion coefficients directly affect the vehicle's power, braking performance, and handling stability. The main road surface identification methods in related technologies are Cause-Based and Effect-Based. The Cause-Based method can identify the road surface adhesion coefficient by measuring the road surface roughness and the road surface slipperiness, while the Effect-Based method uses the vehicle's response when driving on different road surfaces and estimates it through the vehicle dynamics model.

[0004] However, in related technologies, whether it is the Cause-Based or Effect-Based method, due to the inherent characteristics of sensors, the method of further obtaining the road adhesion coefficient by installing sensors has drawbacks. Some sensors are difficult to adapt to complex and changing working conditions, and the use of backbones, such as VGGNet and ResNet object detection network structures, has a huge amount of network computation, which is difficult to meet the real-time detection requirements of intelligent vehicle vision and needs to be improved. Summary of the Invention

[0005] This application provides a method, device, vehicle, and storage medium for estimating road surface adhesion coefficient, in order to solve the technical problems in related technologies, which require the installation of sensors to further obtain the road surface adhesion coefficient, making it difficult to apply to complex working conditions, and the computational load of object detection network structures is huge, making it difficult to meet the real-time detection requirements of automotive vision.

[0006] The first aspect of this application provides a method for estimating the road surface adhesion coefficient, comprising the following steps: acquiring image information of the current road surface; identifying the actual road surface type of the current road surface based on the image information, and matching the corresponding range of road surface adhesion coefficients based on the actual road surface type; estimating the road surface adhesion coefficient of the current road surface through a pre-built vehicle dynamics model, and obtaining the final road surface adhesion coefficient estimation result of the current road surface by combining the fusion result of the range of road surface adhesion coefficients and the road surface adhesion coefficients.

[0007] Optionally, in one embodiment of this application, the step of identifying the actual road surface type of the current road surface based on the image information and matching the corresponding range of road surface adhesion coefficients based on the actual road surface type includes: identifying the actual road surface type of the current road surface based on the image information and estimating the range of road surface adhesion coefficients based on a preset lightweight convolutional neural network.

[0008] Optionally, in one embodiment of this application, estimating the road adhesion coefficient of the current road surface using a pre-built vehicle dynamics model includes: collecting the actual vehicle speed and actual acceleration; and estimating the road adhesion coefficient based on the actual vehicle speed and actual acceleration using a preset QP (Quadratic Programming) algorithm with a forgetting factor.

[0009] Optionally, in one embodiment of this application, the step of combining the range of the road surface adhesion coefficient and the fusion result of the road surface adhesion coefficient to obtain the final road surface adhesion coefficient estimation result of the current road surface includes: based on the extended Kalman filter algorithm, using the range of the road surface adhesion coefficient and the road surface adhesion coefficient as the predicted value and the observed value respectively, establishing a closed-loop feedback model of prediction and observation to obtain the final road surface adhesion coefficient estimation result.

[0010] Optionally, in one embodiment of this application, after collecting the actual vehicle speed and actual acceleration, the method further includes: determining the difference between the actual vehicle speed at the current moment and the actual vehicle speed at the previous moment; if the difference is greater than a preset threshold, determining that the vehicle is skidding, and estimating the acceleration and actual speed at the current moment based on the Kalman filter at the previous moment.

[0011] A second aspect of this application provides a road surface adhesion coefficient estimation device, comprising: an acquisition module for acquiring image information of the current road surface; an identification module for identifying the actual road surface type of the current road surface based on the image information, and matching the corresponding range of road surface adhesion coefficients based on the actual road surface type; and an estimation module for estimating the road surface adhesion coefficient of the current road surface through a pre-built vehicle dynamics model, and combining the fusion result of the range of road surface adhesion coefficients and the road surface adhesion coefficients to obtain the final road surface adhesion coefficient estimation result of the current road surface.

[0012] Optionally, in one embodiment of this application, the identification module includes: an identification unit, used to identify the actual road surface type of the current road surface based on a preset lightweight convolutional neural network, and to estimate the range to which the road surface adhesion coefficient belongs.

[0013] Optionally, in one embodiment of this application, the estimation module includes: a data acquisition unit for acquiring the actual vehicle speed and actual acceleration; and a first estimation unit for estimating the road surface adhesion coefficient based on the actual vehicle speed and actual acceleration using a preset QP algorithm with a forgetting factor.

[0014] Optionally, in one embodiment of this application, the estimation module includes: a modeling unit, used to establish a closed-loop feedback model of prediction and observation based on the extended Kalman filter algorithm, using the range to which the road surface adhesion coefficient belongs and the road surface adhesion coefficient as the predicted value and the observed value, respectively, to obtain the final road surface adhesion coefficient estimation result.

[0015] Optionally, in one embodiment of this application, the estimation module further includes: a judgment unit, used to judge the difference between the actual vehicle speed at the current moment and the actual vehicle speed at the previous moment; and a second estimation unit, used to determine that the vehicle is skidding when the difference is greater than a preset threshold, and to estimate the acceleration and actual speed at the current moment based on the Kalman filter of the previous moment.

[0016] A third aspect of this application provides a vehicle including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the road adhesion coefficient estimation method as described in the above embodiments.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the road surface adhesion coefficient estimation method described above.

[0018] This application embodiment can acquire images of the road ahead, identify the actual road surface type, obtain the corresponding range of road surface adhesion coefficients, and within the target road's road surface adhesion coefficient range, obtain a fusion result with the road surface adhesion coefficient estimated by the vehicle dynamics model. This results in an accurate and effective final road surface adhesion coefficient estimate, meeting the requirements of real-time vehicle vision detection. It achieves road surface adhesion coefficient estimation combining vision and dynamics, is applicable to complex road conditions, and has a moderate computational load. Therefore, it solves the technical problems in related technologies that require further acquisition of road surface adhesion coefficients through sensor installation, making them unsuitable for complex working conditions, and that the computational load of object detection network structures is enormous, making it difficult to meet the real-time detection requirements of vehicle vision.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0021] Figure 1 This is a flowchart of a road surface adhesion coefficient estimation method provided according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the MobileNet structure for a road surface adhesion coefficient estimation method according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the adhesion rate-slip rate curve of a road adhesion coefficient estimation method according to an embodiment of this application;

[0024] Figure 4 This is a schematic diagram illustrating the temporal and spatial asynchrony of a road surface adhesion coefficient estimation method according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram illustrating the principle of a road surface adhesion coefficient estimation method according to an embodiment of this application;

[0026] Figure 6 This is a flowchart of a road surface adhesion coefficient estimation method according to an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of a road surface adhesion coefficient estimation device provided according to an embodiment of this application;

[0028] Figure 8 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0030] The following description, with reference to the accompanying drawings, outlines a method, apparatus, vehicle, and storage medium for estimating the road surface adhesion coefficient according to embodiments of this application. Addressing the technical problems mentioned in the background section, where the road surface adhesion coefficient requires the installation of sensors, making it unsuitable for complex working conditions, and where the computational load of object detection network structures is enormous, hindering real-time detection requirements for automotive vision, this application provides a method for estimating the road surface adhesion coefficient. This method involves acquiring an image of the road ahead, identifying the actual road surface type, obtaining the corresponding range of road surface adhesion coefficients, and fusing the road surface adhesion coefficient estimated by the vehicle dynamics model within the target road's range. This results in an accurate and effective final road surface adhesion coefficient estimate, meeting the requirements of real-time automotive vision detection. This method combines visual and dynamic road surface adhesion coefficient estimation, is applicable to complex road conditions, and has a moderate computational load. Therefore, it solves the technical problems in related technologies where the road surface adhesion coefficient requires the installation of sensors, making it unsuitable for complex working conditions, and where the computational load of object detection network structures is enormous, hindering real-time detection requirements for automotive vision.

[0031] Specifically, Figure 1 This is a flowchart illustrating a method for estimating the road surface adhesion coefficient provided in an embodiment of this application.

[0032] like Figure 1 As shown, the method for estimating the road surface adhesion coefficient includes the following steps:

[0033] In step S101, image information of the current road surface is acquired.

[0034] In actual implementation, the embodiments of this application can collect image information of the road surface on which the vehicle is currently traveling based on vehicle-mounted image acquisition devices, such as vehicle-mounted cameras, so as to identify the current road surface type in the future.

[0035] In step S102, the actual road surface type of the current road surface is identified based on the image information, and the corresponding range of road surface adhesion coefficient is matched based on the actual road surface type.

[0036] As one possible approach, embodiments of this application can identify the actual road surface type, such as rigid road surface, flexible road surface, and semi-rigid road surface, based on the collected image information. Based on the actual road surface type, the corresponding range of road surface adhesion coefficients can be matched through methods such as database query, big data estimation, and construction of relevant models.

[0037] Optionally, in one embodiment of this application, identifying the actual road surface type of the current road surface based on image information and matching the corresponding range of road surface adhesion coefficients based on the actual road surface type includes: identifying the actual road surface type of the current road surface based on image information and estimating the range of road surface adhesion coefficients based on a pre-constructed lightweight convolutional neural network.

[0038] In some embodiments, considering the limited computing power of intelligent vehicle platforms, the embodiments of this application may select a lightweight convolutional neural network model such as MobileNet v2 as the basic structure of the image classification network, so as to identify the actual road surface type of the current road surface and estimate the range of the road surface adhesion coefficient based on the image information of the pre-constructed lightweight convolutional neural network.

[0039] MobileNet v2 evolved from MobileNet v1. The core of MobileNet is depthwise separable convolution, which decomposes the standard convolution into a depthwise convolution and a 1×1 convolution to combine the output of all the depthwise convolutions.

[0040] The specific structure of MobileNet can be as follows: Figure 2 As shown, this is a layer-by-layer stacked network. Dw represents a depthwise decomposition convolution, which is followed by a 1×1 convolution. s2 represents a convolution with a stride of 2, which can reduce the size of the feature map. At the end of the network, a global average pooling layer can be used, which is then fed into a fully connected layer and Softmax for road classification prediction. In addition, MobileNet v1 also sets two hyperparameters: the width multiplier is used to control the number of channels in the feature map, and the resolution multiplier is used to control the size of the feature map.

[0041] MobileNet v2 can replace the original convolution stacking method with residual structures, and proposes an InvertedResidual Block structure and a Linear Bottleneck concept.

[0042] In actual implementation, the process of identifying road types using the MobileNet v2 structure in this embodiment of the application can be as follows:

[0043] S1. Based on the collected image information, the video is processed into image format to obtain a preliminary image dataset;

[0044] S2. To avoid overfitting due to a small number of samples, use OpenCV to expand the image dataset, such as mirror transformation, rotation, brightness adjustment, and exposure adjustment.

[0045] S3. Set up the annotation of the road surface image dataset, which can be 8 types, namely ice film, compacted snow, loose snow, wet cement, dry cement, wet asphalt, dry asphalt and brick paving.

[0046] S4. The DeepLab network was trained using the open-source dataset UAS-UESTC All-Day Scenery to perform semantic segmentation on the image dataset, extract the road surface area, and obtain an image dataset containing only the road surface area, thus eliminating the influence of other information on road type recognition.

[0047] S5. Create training and validation sets, build the MobileNet v2 network model, and train the network to identify road surface types and obtain the range of road surface adhesion coefficients.

[0048] This application uses the lightweight convolutional neural network MobileNet v2 to identify road types, thereby enabling vehicles to predict road surface changes. As long as the training dataset is large enough and covers a wide range of working conditions, it can predict the road surface adhesion coefficient of the road ahead in various weather scenarios.

[0049] In step S103, the road adhesion coefficient of the current road surface is estimated by using a pre-built vehicle dynamics model, and the final road adhesion coefficient estimation result of the current road surface is obtained by combining the range of the road adhesion coefficient and the fusion result of the road adhesion coefficient.

[0050] As one possible approach, embodiments of this application can pre-build a vehicle dynamics model and estimate the road adhesion coefficient of the current road surface based on the vehicle dynamics model. The range of the road adhesion coefficient of the current road surface obtained in the above steps is combined with the estimated road adhesion coefficient of the current road surface to obtain a fusion result, thereby obtaining the final road adhesion coefficient estimation result of the current road surface. This fully leverages the advantages of the estimation algorithm, such as accuracy, real-time performance, and predictability, while ensuring the real-time performance of the estimation.

[0051] It should be noted that the pre-built vehicle dynamics model will be described below.

[0052] Optionally, in one embodiment of this application, after collecting the actual vehicle speed and actual acceleration, the method further includes: determining the difference between the actual vehicle speed at the current moment and the actual vehicle speed at the previous moment; if the difference is greater than a preset threshold, determining that the vehicle is skidding, and estimating the acceleration and actual speed at the current moment based on the Kalman filter at the previous moment.

[0053] It is understandable that, due to the complex and variable noise environment of vehicle operation, the actual vehicle acceleration measured by the IMU (Inertial Measurement Unit) sensor and the actual vehicle speed measured by the GPS (Global Positioning System) are not accurate. In this embodiment, wheel speed sensor data can be used to calculate the vehicle speed.

[0054]

[0055] Where FLSpeed ​​is the speed of the vehicle's left front wheel. tyre FRSpeed ​​is the speed of the left front wheel of the vehicle, and FRSpeed ​​is the speed of the right front wheel of the vehicle. tyre RLSpeed ​​is the speed of the front wheels of the vehicle, and RLSpeed ​​is the speed of the left rear wheel of the vehicle. tyre RRSpeed ​​is the speed of the left rear wheel of the vehicle, and RRSpeed ​​is the speed of the right rear wheel of the vehicle. tyre R is the speed of the right rear wheel of the vehicle, and r is the wheel radius.

[0056] Since the above vehicle speed does not consider the driving and braking slippage conditions, the embodiments of this application need to determine wheel slippage. According to engineering experience, those skilled in the art know that if the wheel speed of a certain wheel at a certain moment is more than 1.5 times the wheel speed at the previous moment, it can be considered that the wheel is slipping at that moment. In this case, the acceleration estimated by Kalman filtering at the previous moment and the speed at the previous moment can be used to replace the speed at this moment.

[0057]

[0058] Conversely, if there is no slippage, the speeds of the four wheels need to be sorted. If the vehicle is accelerating at this moment, the vehicle speed at this moment can be considered as the speed calculated by the third largest wheel speed in this embodiment of the application; if the vehicle is braking at this moment, the vehicle speed at this moment can be considered as the speed calculated by the second largest wheel speed in this embodiment of the application.

[0059] The Kalman filter algorithm is a recursive algorithm. Each recursion mainly includes two steps: first, a predicted value is calculated; then, the predicted value and the measured value are weighted and summed to obtain the optimal estimate. The processed vehicle speed and acceleration are used as the state vector of the Kalman filter algorithm, and the initial values ​​of each variable are set. The algorithm steps are as follows:

[0060] Calculate the predicted value at time k based on the optimal estimate at time k-1:

[0061] x k+1 =Ax k +Buk ,

[0062] Calculate the error of the predicted value at time k based on the error of the optimal estimate at time k-1:

[0063] P k+1 =AP k A T +Q,

[0064] Calculate the Kalman gain K at time k based on the error of the predicted value at time k and the error of the measured value at time k:

[0065] K k =P k H T HP k H T +R) -1 ,

[0066] Calculate the optimal estimate at time k based on the predicted value, the measured value, and the Kalman gain:

[0067] x k =x k +K k (z k -Hx k ),

[0068] The error of the final estimate at time k is calculated based on the error of the predicted value at time k and the Kalman gain at time k:

[0069] P k =(IK k H)P k .

[0070] Considering that noise Q and R change accordingly with changes in road surface and vehicle speed during vehicle operation, this embodiment of the application can use the Sage-Husa method to introduce a hyperparameter to adjust the update parameter d, where b is the forgetting factor:

[0071]

[0072] Then, in this embodiment, the observation error R can be adjusted based on d, and a weighted summation method can be used so that the observation error R tends to stabilize over time.

[0073] The Kalman filter algorithm described above can re-estimate vehicle speed and acceleration using onboard sensor parameters, and the estimated results can be used as known conditions for estimating the road adhesion coefficient.

[0074] Optionally, in one embodiment of this application, the road adhesion coefficient of the current road surface is estimated by means of a pre-built vehicle dynamics model, including: collecting the actual vehicle speed and actual acceleration; and estimating the road adhesion coefficient based on the actual vehicle speed and actual acceleration using a preset QP algorithm with a forgetting factor.

[0075] Specifically, the embodiments of this application can collect parameters such as the actual speed, actual acceleration, wheel speed, and torque of the vehicle based on the vehicle controller or on-board sensors, and then use a preset QP algorithm with a forgetting factor to estimate the road adhesion coefficient.

[0076] Furthermore, embodiments of this application may use the QP algorithm with a forgetting factor to estimate the road surface adhesion coefficient.

[0077] First, consider the situation when the vehicle is stationary. Due to environmental noise, sensors will inevitably be disturbed, resulting in data noise. The following formula shows that even a small amount of noise can have a very large impact on the slip ratio calculation:

[0078]

[0079] Based on engineering experience, those skilled in the art will know that if the vehicle speed is less than 0.1 m / s at a certain moment and the vehicle speed at the previous moment is also less than 0.1 m / s, the embodiments of this application can be considered as the vehicle being stationary at that moment, and the vehicle speed is set to 0.

[0080] The QP problem is a quadratic programming problem, and its standard form includes an objective function and constraints:

[0081]

[0082] stBx≤d,

[0083] Based on the vehicle's longitudinal travel equation:

[0084]

[0085] The embodiments of this application can simplify the above formula by ignoring ramp resistance and air resistance, resulting in:

[0086]

[0087] The embodiments of this application can linearize the longitudinal adhesion coefficient curve. Assuming that the road adhesion coefficient is at its maximum when the slip ratio is 20%, two linear functions are obtained:

[0088]

[0089] The meanings of μ0, k1, and k2 in the above formula can be understood as follows: Figure 3 As shown.

[0090] The embodiments of this application can transform the above into the standard form of a QP problem:

[0091] AX = F,

[0092] Let the state vector X = [k1m k2m m μ0m] T Categorical discussion:

[0093] When s < 0.2, we have:

[0094]

[0095] When s > 0.2, we have:

[0096]

[0097] The objective function is to minimize (AX-F), that is:

[0098]

[0099] The above equation can be rewritten in the standard form of the QP algorithm as follows:

[0100]

[0101] Where H = A T A, f T =-F T A, thus we obtain the standard QP form objective function.

[0102] Based on engineering experience, those skilled in the art will know that constraints can be designed in the embodiments of this application:

[0103]

[0104] This enables the estimation of road surface adhesion coefficient using the QP algorithm with a forgetting factor.

[0105] Optionally, in one embodiment of this application, the final road surface adhesion coefficient estimation result of the current road surface is obtained by combining the fusion result of the road surface adhesion coefficient range and the road surface adhesion coefficient, including: based on the extended Kalman filter algorithm, using the road surface adhesion coefficient range and the road surface adhesion coefficient as the predicted value and the observed value respectively, establishing a closed-loop feedback model of prediction and observation to obtain the final road surface adhesion coefficient estimation result.

[0106] In actual implementation, the embodiments of this application can combine visual imagery and vehicle dynamics methods to obtain accurate final road surface adhesion coefficient estimation results.

[0107] It is understandable that the information detected by the vehicle camera is the road information ahead, while the vehicle sensor estimates the road information on which the vehicle is traveling, which can easily cause the two methods to be out of sync in time and space.

[0108] like Figure 4 As shown, this application embodiment can use an approximate method to handle this problem:

[0109]

[0110] In this embodiment, the spatial distance between the two methods can be approximated as the straight-line distance from the center point of the road surface section detected by the camera to the center of mass of the vehicle. Dividing this distance by the vehicle speed and then by the sampling time yields the lag time of the vehicle sensor relative to the vehicle camera, thus solving the problem of time and space asynchrony between the two methods.

[0111] Based on engineering experience, those skilled in the art can establish a table comparing road surface types with road surface adhesion coefficients, as shown in Table 1.

[0112] Table 1

[0113] Road surface type Coefficient of adhesion (low speed) Coefficient of adhesion (high speed) Dry asphalt 0.5-0.8 0.45-0.75 wet asphalt 0.45-0.7 0.4-0.6 dry cement 0.55-0.8 0.5-0.75 wet cement 0.45-0.75 0.45-0.7 Brick paving 0.5-0.8 0.45-0.75 Songxue 0.2-0.4 0.2-0.3 Compacted snow 0.1-0.20 0.08-0.2 ice film 0.08-0.2 0.08-0.15

[0114] This application embodiment can set up vision-based road type recognition to estimate the adhesion coefficient of the target road surface in the range of [f]. min ,f max The road adhesion coefficient estimation result based on vehicle dynamics is f. x .

[0115] This application embodiment can consider using the Extended Kalman Filter algorithm to solve the data fusion problem. The Extended Kalman Filter process is as follows:

[0116] The dynamic process of a discrete nonlinear system can be represented as:

[0117] x k+1 =f(k,x k )+g k ω k ,

[0118] z k =h(k,x k )+v k ,

[0119] Where, x k Let z be the state vector. k Let f(g) be the measurement vector, and h(g) be the nonlinear state function and the observation function, respectively, and ω be the measurement vector. k and v k It is uncorrelated Gaussian white noise with zero mean and covariance Q. k and Rk .

[0120] For the nonlinear state function f(g) in x k Performing a first-order Taylor expansion at the given point, we get:

[0121]

[0122] in, Let be a higher-order infinitesimal term, let The above expression can then be simplified to:

[0123]

[0124] Then the one-step state prediction can be:

[0125]

[0126] The one-step prediction covariance is:

[0127]

[0128] One-step state prediction for the nonlinear observation function h(g) Performing a first-order Taylor expansion at the given location yields:

[0129]

[0130] in, Let be a higher-order infinitesimal term, let The above formula can then be simplified to:

[0131]

[0132] Then the prediction for the first observation step is:

[0133]

[0134] The covariance matrix of the observation prediction error is:

[0135]

[0136] The covariance matrix between state and measurement is:

[0137]

[0138] The Kalman gain matrix is ​​then:

[0139]

[0140] Then the state estimate at time k+1 is:

[0141]

[0142] The state estimation error covariance matrix is:

[0143]

[0144] In this embodiment, the extended Kalman filter method can be used to estimate the road surface adhesion coefficient. A road surface adhesion coefficient estimation method based on vehicle dynamics is used as the predicted value, and a road surface adhesion coefficient estimation method based on vision is used as the observed value. A closed-loop feedback model of prediction and observation is established to obtain the road surface adhesion coefficient.

[0145] The following is combined Figures 2 to 6 As shown, the working principle of the road surface adhesion coefficient estimation method of this application is explained in detail with an embodiment.

[0146] in Figure 5 This describes the working principle of the road surface adhesion coefficient estimation method in the embodiments of this application.

[0147] Combination Figure 5 and Figure 6 As shown, embodiments of this application may include the following steps:

[0148] Step S601: Use the vision-based lightweight convolutional neural network MobileNet v2 to identify road types and estimate the range of the current road surface's adhesion coefficient.

[0149] In some embodiments, considering the limited computing power of intelligent vehicle platforms, the embodiments of this application may select a lightweight convolutional neural network model such as MobileNetv2 as the basic structure of the image classification network, so as to identify the actual road surface type of the current road surface and estimate the range of the road surface adhesion coefficient based on the image information of the pre-constructed lightweight convolutional neural network.

[0150] MobileNet v2 evolved from MobileNet v1. The core of MobileNet is depthwise separable convolution, which decomposes the standard convolution into a depthwise convolution and a 1×1 convolution to combine the output of all the depthwise convolutions.

[0151] The specific structure of MobileNet can be as follows: Figure 2 As shown, this is a layer-by-layer stacked network. Dw represents a depthwise decomposition convolution, which is followed by a 1×1 convolution. s2 represents a convolution with a stride of 2, which can reduce the size of the feature map. At the end of the network, a global average pooling layer can be used, which is then fed into a fully connected layer and Softmax for road classification prediction. In addition, MobileNet v1 also sets two hyperparameters: the width multiplier is used to control the number of channels in the feature map, and the resolution multiplier is used to control the size of the feature map.

[0152] MobileNet v2 can replace the original convolution stacking method with residual structures, and proposes an InvertedResidual Block structure and a Linear Bottleneck concept.

[0153] In actual implementation, the process of identifying road types using the MobileNet v2 structure in this embodiment of the application can be as follows:

[0154] S1. Based on the collected image information, the video is processed into image format to obtain a preliminary image dataset;

[0155] S2. To avoid overfitting due to a small number of samples, use OpenCV to expand the image dataset, such as mirror transformation, rotation, brightness adjustment, and exposure adjustment.

[0156] S3. Set up the annotation of the road surface image dataset, which can be 8 types, namely ice film, compacted snow, loose snow, wet cement, dry cement, wet asphalt, dry asphalt and brick paving.

[0157] S4. The DeepLab network was trained using the open-source dataset UAS-UESTC All-Day Scenery to perform semantic segmentation on the image dataset, extract the road surface area, and obtain an image dataset containing only the road surface area, thus eliminating the influence of other information on road type recognition.

[0158] S5. Create training and validation sets, build the MobileNet v2 network model, and train the network to identify road surface types and obtain the range of road surface adhesion coefficients.

[0159] This application uses the lightweight convolutional neural network MobileNet v2 to identify road types, thereby enabling vehicles to predict road surface changes. As long as the training dataset is large enough and covers a wide range of working conditions, it can predict the road surface adhesion coefficient of the road ahead in various weather scenarios.

[0160] Step S602: Estimation of road surface adhesion coefficient based on vehicle dynamics, using the QP algorithm with forgetting factor to estimate the road surface adhesion coefficient.

[0161] Specifically, the embodiments of this application can collect parameters such as the actual speed, actual acceleration, wheel speed, and torque of the vehicle based on the vehicle controller or on-board sensors, and then use a preset QP algorithm with a forgetting factor to estimate the road adhesion coefficient.

[0162] It is understandable that, due to the complex and variable noise environment of vehicle operation, the actual vehicle acceleration measured by the IMU (Inertial Measurement Unit) sensor and the actual vehicle speed measured by the GPS (Global Positioning System) are not accurate. In this embodiment, wheel speed sensor data can be used to calculate the vehicle speed.

[0163]

[0164] Where FLSpeed ​​is the speed of the vehicle's left front wheel. tyre FRSpeed ​​is the speed of the left front wheel of the vehicle, and FRSpeed ​​is the speed of the right front wheel of the vehicle. tyre RLSpeed ​​is the speed of the front wheels of the vehicle, and RLSpeed ​​is the speed of the left rear wheel of the vehicle. tyre RRSpeed ​​is the speed of the left rear wheel of the vehicle, and RRSpeed ​​is the speed of the right rear wheel of the vehicle. tyre R is the speed of the right rear wheel of the vehicle, and r is the wheel radius.

[0165] Since the above vehicle speed does not consider the driving and braking slippage conditions, the embodiments of this application need to determine wheel slippage. According to engineering experience, those skilled in the art know that if the wheel speed of a certain wheel at a certain moment is more than 1.5 times the wheel speed at the previous moment, it can be considered that the wheel is slipping at that moment. In this case, the acceleration estimated by Kalman filtering at the previous moment and the speed at the previous moment can be used to replace the speed at this moment.

[0166]

[0167] Conversely, if there is no slippage, the speeds of the four wheels need to be sorted. If the vehicle is accelerating at this moment, the vehicle speed at this moment can be considered as the speed calculated by the third largest wheel speed in this embodiment of the application; if the vehicle is braking at this moment, the vehicle speed at this moment can be considered as the speed calculated by the second largest wheel speed in this embodiment of the application.

[0168] The Kalman filter algorithm is a recursive algorithm. Each recursion mainly includes two steps: first, a predicted value is calculated; then, the predicted value and the measured value are weighted and summed to obtain the optimal estimate. The processed vehicle speed and acceleration are used as the state vector of the Kalman filter algorithm, and the initial values ​​of each variable are set. The algorithm steps are as follows:

[0169] Calculate the predicted value at time k based on the optimal estimate at time k-1:

[0170] x k+1 =Ax k +Bu k ,

[0171] Calculate the error of the predicted value at time k based on the error of the optimal estimate at time k-1:

[0172] P k+1 =AP k A T +Q,

[0173] Calculate the Kalman gain K at time k based on the error of the predicted value at time k and the error of the measured value at time k:

[0174] K k =P k H T HP k H T +R) -1 ,

[0175] Calculate the optimal estimate at time k based on the predicted value, the measured value, and the Kalman gain:

[0176] x k =x k +K k (z k -Hx k ),

[0177] The error of the final estimate at time k is calculated based on the error of the predicted value at time k and the Kalman gain at time k:

[0178] P k =(IK k H)P k .

[0179] Considering that noise Q and R change accordingly with changes in road surface and vehicle speed during vehicle operation, this embodiment of the application can use the Sage-Husa method to introduce a hyperparameter to adjust the update parameter d, where b is the forgetting factor:

[0180]

[0181] Then, in this embodiment, the observation error R can be adjusted based on d, and a weighted summation method can be used so that the observation error R tends to stabilize over time.

[0182] The Kalman filter algorithm described above can re-estimate vehicle speed and acceleration using onboard sensor parameters, and the estimated results can be used as known conditions for estimating the road adhesion coefficient.

[0183] Furthermore, embodiments of this application may use the QP algorithm with a forgetting factor to estimate the road surface adhesion coefficient.

[0184] First, consider the situation when the vehicle is stationary. Due to environmental noise, sensors will inevitably be disturbed, resulting in data noise. The following formula shows that even a small amount of noise can have a very large impact on the slip ratio calculation:

[0185]

[0186] Based on engineering experience, those skilled in the art will know that if the vehicle speed is less than 0.1 m / s at a certain moment and the vehicle speed at the previous moment is also less than 0.1 m / s, the embodiments of this application can be considered as the vehicle being stationary at that moment, and the vehicle speed is set to 0.

[0187] The QP problem is a quadratic programming problem, and its standard form includes an objective function and constraints:

[0188]

[0189] stBx≤d,

[0190] Based on the vehicle's longitudinal travel equation:

[0191]

[0192] The embodiments of this application can simplify the above formula by ignoring ramp resistance and air resistance, resulting in:

[0193]

[0194] The embodiments of this application can linearize the longitudinal adhesion coefficient curve. Assuming that the road adhesion coefficient is at its maximum when the slip ratio is 20%, two linear functions are obtained:

[0195]

[0196] The meanings of μ0, k1, and k2 in the above formula can be understood as follows: Figure 3 As shown.

[0197] The embodiments of this application can transform the above into the standard form of a QP problem:

[0198] AX = F,

[0199] Let the state vector X = [k1m k2m m μ0m] T Categorical discussion:

[0200] When s < 0.2, we have:

[0201]

[0202] When s > 0.2, we have:

[0203]

[0204] The objective function is to minimize (AX-F), that is:

[0205]

[0206] The above equation can be rewritten in the standard form of the QP algorithm as follows:

[0207]

[0208] Where H = A T A, f T =-F T A, thus we obtain the standard QP form objective function.

[0209] Based on engineering experience, those skilled in the art will know that constraints can be designed in the embodiments of this application:

[0210]

[0211] This enables the estimation of road surface adhesion coefficient using the QP algorithm with a forgetting factor.

[0212] Step S603: Combine the visual image and vehicle dynamics methods to obtain an accurate estimate of the final road adhesion coefficient of the current road surface.

[0213] In actual implementation, the embodiments of this application can combine visual imagery and vehicle dynamics methods to obtain accurate final road surface adhesion coefficient estimation results.

[0214] It is understandable that the information detected by the vehicle camera is the road information ahead, while the vehicle sensor estimates the road information on which the vehicle is traveling, which can easily cause the two methods to be out of sync in time and space.

[0215] like Figure 4 As shown, this application embodiment can use an approximate method to handle this problem:

[0216]

[0217] In this embodiment, the spatial distance between the two methods can be approximated as the straight-line distance from the center point of the road surface section detected by the camera to the center of mass of the vehicle. Dividing this distance by the vehicle speed and then by the sampling time yields the lag time of the vehicle sensor relative to the vehicle camera, thus solving the problem of time and space asynchrony between the two methods.

[0218] Based on engineering experience, those skilled in the art can establish a table comparing road surface types with road surface adhesion coefficients, as shown in Table 1.

[0219] This application embodiment can set up vision-based road type recognition to estimate the adhesion coefficient of the target road surface in the range of [f]. min ,f max The road adhesion coefficient estimation result based on vehicle dynamics is f. x .

[0220] This application embodiment can consider using the Extended Kalman Filter algorithm to solve the data fusion problem. The Extended Kalman Filter process is as follows:

[0221] The dynamic process of a discrete nonlinear system can be represented as:

[0222] x k+1 =f(k,x k )+g k ω k ,

[0223] z k =h(k,x k )+v k ,

[0224] Where, x k Let z be the state vector. k Let f(g) be the measurement vector, and h(g) be the nonlinear state function and the observation function, respectively, and ω be the measurement vector. k and v k It is uncorrelated Gaussian white noise with zero mean and covariance Q. k and R k .

[0225] For the nonlinear state function f(g) in x k Performing a first-order Taylor expansion at the given point, we get:

[0226]

[0227] in, Let be a higher-order infinitesimal term, let The above expression can then be simplified to:

[0228]

[0229] Then the one-step state prediction can be:

[0230]

[0231] The one-step prediction covariance is:

[0232]

[0233] One-step state prediction for the nonlinear observation function h(g) Performing a first-order Taylor expansion at the given location yields:

[0234]

[0235] in, Let be a higher-order infinitesimal term, let The above formula can then be simplified to:

[0236]

[0237] Then the prediction for the first observation step is:

[0238]

[0239] The covariance matrix of the observation prediction error is:

[0240]

[0241] The covariance matrix between state and measurement is:

[0242]

[0243] The Kalman gain matrix is ​​then:

[0244]

[0245] Then the state estimate at time k+1 is:

[0246]

[0247] The state estimation error covariance matrix is:

[0248]

[0249] In this embodiment, the extended Kalman filter method can be used to estimate the road surface adhesion coefficient. A road surface adhesion coefficient estimation method based on vehicle dynamics is used as the predicted value, and a road surface adhesion coefficient estimation method based on vision is used as the observed value. A closed-loop feedback model of prediction and observation is established to obtain the road surface adhesion coefficient.

[0250] The road surface adhesion coefficient estimation method proposed in this application can acquire images of the road ahead, identify the actual road surface type, obtain the corresponding range of road surface adhesion coefficients, and, within the target road surface adhesion coefficient range, obtain a fusion result with the road surface adhesion coefficient estimated by the vehicle dynamics model. This yields an accurate and effective final road surface adhesion coefficient estimation result, meeting the requirements of real-time vehicle vision detection. It achieves road surface adhesion coefficient estimation combining vision and dynamics, is applicable to complex road conditions, and has a moderate computational load. Therefore, it solves the technical problems in related technologies that require the installation of sensors to further obtain the road surface adhesion coefficient, making it difficult to apply to complex working conditions, and that the computational load of the object detection network structure is enormous, making it difficult to meet the real-time detection requirements of vehicle vision.

[0251] Next, the road surface adhesion coefficient estimation device proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0252] Figure 7 This is a block diagram of a road surface adhesion coefficient estimation device according to an embodiment of this application.

[0253] like Figure 7 As shown, the road surface adhesion coefficient estimation device 10 includes: a data acquisition module 100, an identification module 200, and an estimation module 300.

[0254] Specifically, the acquisition module 100 is used to acquire image information of the current road surface.

[0255] The identification module 200 is used to identify the actual road surface type of the current road surface based on image information, and to match the corresponding range of road surface adhesion coefficients based on the actual road surface type.

[0256] The estimation module 300 is used to estimate the road adhesion coefficient of the current road surface through a pre-built vehicle dynamics model, and to obtain the final road adhesion coefficient estimation result of the current road surface by combining the range of the road adhesion coefficient and the fusion result of the road adhesion coefficient.

[0257] Optionally, in one embodiment of this application, the identification module 200 includes an identification unit.

[0258] The identification unit is used to identify the actual road surface type of the current road surface and estimate the range of the road surface adhesion coefficient based on the image information of a pre-constructed lightweight convolutional neural network.

[0259] Optionally, in one embodiment of this application, the estimation module 300 includes: a data acquisition unit and a first estimation unit.

[0260] The data acquisition unit is used to collect the vehicle's actual speed and actual acceleration.

[0261] The first estimation unit is used to estimate the road surface adhesion coefficient based on the actual vehicle speed and actual acceleration using a preset QP algorithm with a forgetting factor.

[0262] Optionally, in one embodiment of this application, the estimation module 300 includes a modeling unit.

[0263] The modeling unit is used to establish a closed-loop feedback model for prediction and observation based on the extended Kalman filter algorithm, using the range of the road surface adhesion coefficient and the road surface adhesion coefficient as the predicted and observed values, respectively, to obtain the final road surface adhesion coefficient estimation result.

[0264] Optionally, in one embodiment of this application, the estimation module 300 further includes: a judgment unit and a second estimation unit.

[0265] The judgment unit is used to determine the difference between the actual vehicle speed at the current moment and the actual vehicle speed at the previous moment.

[0266] The second estimation unit is used to determine that the vehicle is skidding when the difference is greater than a preset threshold, and to estimate the acceleration and actual speed at the current moment based on the Kalman filter of the previous moment.

[0267] It should be noted that the foregoing explanation of the road surface adhesion coefficient estimation method embodiment also applies to the road surface adhesion coefficient estimation device of this embodiment, and will not be repeated here.

[0268] The road surface adhesion coefficient estimation device proposed in this application can acquire images of the road ahead, identify the actual road surface type, obtain the corresponding range of road surface adhesion coefficients, and, within the target road surface adhesion coefficient range, obtain a fusion result with the road surface adhesion coefficient estimated by the vehicle dynamics model. This results in an accurate and effective final road surface adhesion coefficient estimation result, meeting the requirements of real-time vehicle vision detection. It achieves road surface adhesion coefficient estimation combining vision and dynamics, is applicable to complex road conditions, and has a moderate computational load. Therefore, it solves the technical problems in related technologies that require the installation of sensors to further obtain the road surface adhesion coefficient, making it difficult to apply to complex working conditions, and that the computational load of the object detection network structure is enormous, making it difficult to meet the real-time detection requirements of vehicle vision.

[0269] Figure 8 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0270] The memory 801, the processor 802, and the computer program stored on the memory 801 and capable of running on the processor 802.

[0271] When the processor 802 executes the program, it implements the road surface adhesion coefficient estimation method provided in the above embodiments.

[0272] Furthermore, the vehicle also includes:

[0273] Communication interface 803 is used for communication between memory 801 and processor 802.

[0274] The memory 801 is used to store computer programs that can run on the processor 802.

[0275] The memory 801 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0276] If the memory 801, processor 802, and communication interface 803 are implemented independently, then the communication interface 803, memory 801, and processor 802 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0277] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0278] The processor 802 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0279] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for estimating the road surface adhesion coefficient.

[0280] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0281] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0282] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0283] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0284] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0285] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0286] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0287] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for estimating the road surface adhesion coefficient, characterized in that, Includes the following steps: Acquire image information of the current road surface; Based on the image information, the actual road surface type of the current road surface is identified, and the corresponding range of road surface adhesion coefficients is matched based on the actual road surface type. This includes: using a pre-constructed lightweight convolutional neural network, identifying the actual road surface type of the current road surface from the image information, and estimating the range of the road surface adhesion coefficients; wherein the MobileNet v2 lightweight convolutional neural network model is selected as the basic structure of the image classification network, based on the pre-constructed lightweight convolutional neural network; and The road surface adhesion coefficient of the current road surface is estimated by using a pre-built vehicle dynamics model. Then, by combining the range of the road surface adhesion coefficient with the fusion result of the road surface adhesion coefficient, a final road surface adhesion coefficient estimation result is obtained for the current road surface. Based on the extended Kalman filter algorithm, a closed-loop feedback model for prediction and observation is established, using the range of the road surface adhesion coefficient and the road surface adhesion coefficient as the predicted and observed values, respectively, to obtain the final road surface adhesion coefficient estimation result. By acquiring images of the road ahead, identifying the actual road surface type, and obtaining the corresponding range of road surface adhesion coefficients, the fusion result of the road surface adhesion coefficient estimated by the vehicle dynamics model is obtained within the target road surface adhesion coefficient range. This leads to an accurate and effective final road surface adhesion coefficient estimation result, thus meeting the requirements of real-time vehicle vision detection and realizing road surface adhesion coefficient estimation that combines vision and dynamics. The estimation of the road adhesion coefficient of the current road surface using a pre-built vehicle dynamics model includes: Collect the vehicle's actual speed and actual acceleration; Based on the actual vehicle speed and actual acceleration, the road surface adhesion coefficient is estimated using a preset QP algorithm with a forgetting factor. The QP problem is a quadratic programming problem, and its standard form includes an objective function and constraints. , Linearization of the longitudinal adhesion coefficient curve yields two linear functions: ; The process involves collecting image information of the current road surface and processing the video into image format to initially obtain an image dataset. The image dataset was augmented using OpenCV, which performed mirror transformations, rotations, brightness adjustments, and exposure adjustments. A DeepLab network was trained using the open-source dataset UAS-UESTC All-Day Scenery to perform semantic segmentation on the image dataset, extracting the road surface region, resulting in an image dataset containing only the road surface region.

2. The method according to claim 1, characterized in that, After collecting the actual vehicle speed and actual acceleration, the following is also included: Determine the difference between the current vehicle speed and the previous vehicle speed; If the difference is greater than a preset threshold, the vehicle is determined to be skidding, and the acceleration and actual speed at the current moment are estimated based on the Kalman filter of the previous moment.

3. A road surface adhesion coefficient estimation device, characterized in that, include: The acquisition module is used to acquire image information of the current road surface; A recognition module is used to identify the actual road surface type of the current road surface based on the image information, and to match the corresponding range of road surface adhesion coefficients based on the actual road surface type. The recognition module includes: A recognition unit is used to identify the actual road surface type of the current road surface and estimate the range of the road surface adhesion coefficient based on the image information, using a pre-constructed lightweight convolutional neural network. The MobileNet v2 lightweight convolutional neural network model is selected as the basic structure of the image classification network. The estimation module is used to estimate the road adhesion coefficient of the current road surface using a pre-built vehicle dynamics model, and to obtain the final road adhesion coefficient estimation result of the current road surface by combining the range of the road adhesion coefficient and the fusion result of the road adhesion coefficient. Based on the extended Kalman filter algorithm, a closed-loop feedback model for prediction and observation is established, using the range of the road surface adhesion coefficient and the road surface adhesion coefficient as the predicted and observed values, respectively, to obtain the final road surface adhesion coefficient estimation result. By acquiring images of the road ahead, identifying the actual road surface type, and obtaining the corresponding range of road surface adhesion coefficients, the fusion result of the road surface adhesion coefficient estimated by the vehicle dynamics model is obtained within the target road surface adhesion coefficient range. This leads to an accurate and effective final road surface adhesion coefficient estimation result, thus meeting the requirements of real-time vehicle vision detection and realizing road surface adhesion coefficient estimation that combines vision and dynamics. The estimation module includes: The data acquisition unit is used to collect the vehicle's actual speed and actual acceleration. The estimation unit is used to estimate the road surface adhesion coefficient based on the actual vehicle speed and actual acceleration using a preset QP algorithm with a forgetting factor. The QP problem is a quadratic programming problem, and its standard form includes an objective function and constraints. , Linearization of the longitudinal adhesion coefficient curve yields two linear functions: ; The process involves collecting image information of the current road surface and processing the video into image format to initially obtain an image dataset. The image dataset was augmented using OpenCV, which performed mirror transformations, rotations, brightness adjustments, and exposure adjustments. A DeepLab network was trained using the open-source dataset UAS-UESTC All-Day Scenery to perform semantic segmentation on the image dataset, extracting the road surface region, resulting in an image dataset containing only the road surface region.

4. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the road surface adhesion coefficient estimation method as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the road surface adhesion coefficient estimation method as described in any one of claims 1-2.

Images