A road surface friction coefficient estimation method and system based on visual recognition and dynamics fusion

Abstract

The application discloses a road surface friction coefficient estimation method and system based on visual identification and dynamics fusion, relates to the technical field of autonomous control of wheeled unmanned platforms, and comprises the following steps: acquiring road surface images and motion state information in the driving process of the wheeled unmanned platform; performing classification and smoothing processing on the road surface images, determining a visual classification probability, a visual prior interval and a center value; estimating wheel longitudinal tire force, vertical tire force and slip ratio based on the motion state information; determining a slip slope fitting interval based on the center value of the visual prior interval and the visual classification probability, identifying the slip slope by using a recursive least square method and calculating a dynamics adhesion coefficient estimation value; respectively calculating visual confidence and dynamics confidence, and performing hierarchical decision fusion to output a final road surface friction coefficient estimation value. The application can improve the accuracy and robustness of TRFC estimation without a tire model.

Description

Technical Field

[0001] This application relates to the field of autonomous control technology for wheeled unmanned platforms, and in particular to a method and system for estimating road surface friction coefficient based on visual recognition and dynamics fusion. Background Technology

[0002] During operation, the tire-road friction coefficient (TRFC) of wheeled unmanned platforms directly affects their driving, braking, and steering capabilities, making it a key parameter determining their driving safety and autonomous control stability. TRFC is used as a basis for tire force constraints or control in model predictive control, traction control, and stability assessment under extreme conditions. Therefore, accurately obtaining TRFC is of great significance for the driving safety and autonomous control stability of wheeled unmanned platforms.

[0003] However, TRFC cannot be directly measured by onboard sensors and can only be estimated indirectly through physical quantities such as wheel slip ratio, tire force, and vehicle motion. Currently, TRFC estimation methods are mainly divided into three categories: dynamic model-based methods, vision-based methods, and vision-dynamic fusion methods, but all of them have significant shortcomings.

[0004] Dynamic model-based methods must embed parametric tire models such as the HSRI tire model, Dugoff model, or magic formula, and also require the establishment of complex whole-vehicle dynamic models. For wheeled unmanned platforms that widely use non-pneumatic tires, tire model parameters are difficult to obtain, leading to large estimation errors or even failure. In addition, such methods have low signal-to-noise ratios and drastic fluctuations in estimated values ​​under low slip and low speed conditions.

[0005] Vision-based methods identify road surface types from images, but can only output empirical ranges of adhesion coefficients corresponding to road surface types. They cannot provide continuous and accurate values ​​and are easily affected by environmental factors such as lighting, shadows, water stains, and ice and snow. In visually deceptive scenarios such as black ice and strong reflections, the classification accuracy drops sharply.

[0006] In existing vision-dynamics fusion methods, the dynamics component still relies on parametric tire models, with visual information only providing auxiliary reference at the decision-making level and failing to actively guide the dynamics identification process. Furthermore, the fusion weights are often fixed values ​​or rely on a single criterion, lacking a joint measure of the instantaneous reliability of the two heterogeneous information sources, and thus unable to maintain stability under sensor anomalies or drastic changes in operating conditions. Therefore, existing methods struggle to achieve continuous, accurate, and stable estimation of TRFC without a tire model. Summary of the Invention

[0007] The purpose of this application is to provide a method and system for estimating road friction coefficient based on visual recognition and dynamics fusion, which can improve the accuracy and robustness of TRFC estimation without the need for a tire model.

[0008] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for estimating the road surface friction coefficient based on the fusion of visual recognition and dynamics, including: Acquire road surface images and motion state information during the driving process of the wheeled unmanned platform; the motion state information includes at least the longitudinal velocity of the center of gravity of the wheeled unmanned platform, the yaw rate, the angular velocity of each wheel, the longitudinal acceleration, and the lateral acceleration.

[0009] The road surface image is input into a pre-trained classification network, which outputs a road surface category probability vector. The road surface category probability vector is then smoothed to determine the visual classification probability, the visual prior interval, and the center value of the visual prior interval.

[0010] Based on the motion state information, the longitudinal tire force and vertical tire force of the wheel are estimated, and the wheel slip ratio is calculated.

[0011] Based on the center value of the visual prior interval and the visual classification probability, the sliding slope fitting interval is determined.

[0012] If the wheel slip ratio is within the slip slope fitting range, the slip slope estimate is updated using RLS; otherwise, the slip slope estimate remains unchanged.

[0013] Based on the pre-calibrated reference road surface slip slope, reference adhesion coefficient, and the estimated slip slope value, the estimated value of the dynamic adhesion coefficient is calculated.

[0014] Based on the visual classification probability, a piecewise function is used to calculate the visual confidence level.

[0015] The dynamic confidence level is calculated based on the motion state information, the wheel slip ratio, and the slip slope fitting interval.

[0016] The estimated dynamic adhesion coefficient is compared with the visual prior interval, and a hierarchical decision fusion is performed by combining the visual confidence and the dynamic confidence to output the final road surface friction coefficient estimate.

[0017] Secondly, this application provides a road surface friction coefficient estimation system based on the fusion of visual recognition and dynamics, comprising: The visual classification module is used to acquire road surface images during the driving process of the wheeled unmanned platform; it is also used to input the road surface images into a pre-trained classification network, output a road surface category probability vector, and smooth the road surface category probability vector to determine the visual classification probability, visual prior interval, and visual prior interval center value.

[0018] The dynamic identification module is used to acquire motion state information of the wheeled unmanned platform during its operation; it is also used to estimate the longitudinal and vertical tire forces of the wheels based on the motion state information, and calculate the wheel slip ratio; it is also used to determine the slip slope fitting interval based on the visual prior interval center value and the visual classification probability; if the wheel slip ratio is within the slip slope fitting interval, the slip slope estimate is updated through RLS; otherwise, the slip slope estimate remains unchanged; it is also used to calculate the dynamic adhesion coefficient estimate based on the pre-calibrated reference road slip slope, reference adhesion coefficient, and the slip slope estimate.

[0019] The visual-dynamic fusion module is used to calculate the visual confidence score based on the visual classification probability using a piecewise function; it is also used to calculate the dynamic confidence score based on the motion state information, the wheel slip ratio, and the slip slope fitting interval; it is also used to compare the estimated dynamic adhesion coefficient with the visual prior interval, and combine the visual confidence score and the dynamic confidence score to perform hierarchical decision fusion, and output the final road surface friction coefficient estimate.

[0020] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method and system for estimating the road friction coefficient (TRFC) based on visual recognition and dynamics fusion. The method acquires road surface images and outputs road surface category probability vectors from a pre-trained classification network, which are then smoothed to determine the visual classification probability, visual prior interval, and its center value. Simultaneously, it estimates the longitudinal and vertical tire forces based on motion state information and calculates the wheel slip ratio. This allows for the acquisition of road surface prior information and force-slip baseline data even without a tire model, solving the problem of estimation failure caused by existing methods relying on parameterized tire models and achieving a continuous estimation basis for TRFC. The method determines the slip slope fitting interval based on the center value of the visual prior interval and the visual classification probability. When the wheel slip ratio falls within this interval, the slip slope estimate is updated using RLS. Finally, based on the reference road surface slip slope, reference adhesion coefficient, and the slip slope estimate... The method calculates the dynamic adhesion coefficient estimate, allowing the fitting interval to adaptively adjust with the reliability of visual information. This solves the problem of unreasonable information utilization when the fixed fitting interval changes, improving the accuracy and stability of dynamic estimation. Visual confidence is calculated using a piecewise function based on visual classification probability, while dynamic confidence is calculated based on the fitting interval using motion state information, wheel slip rate, and slip slope. The dynamic adhesion coefficient estimate is compared with the visual prior interval, and hierarchical decision fusion is performed by combining visual and dynamic confidence to output the final road friction coefficient estimate. This allows the fusion strategy to dynamically adjust with the instantaneous reliability of the two information sources, solving the problem of poor robustness of existing fusion methods under extreme conditions such as visual deception, low-speed weak excitation, and sudden road changes. Therefore, it can improve the accuracy and robustness of TRFC estimation without the need for a tire model. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is an application environment diagram of a road friction coefficient estimation method based on visual recognition and dynamics fusion in one embodiment of this application; Figure 2 A flowchart illustrating a method for estimating road surface friction coefficient based on visual recognition and dynamic fusion, provided in an embodiment of this application; Figure 3 A schematic diagram of the processing flow of a visual classification module based on Efficientnet-B0 provided for another embodiment of this application; Figure 4This is a schematic diagram of the processing flow of the dynamic identification module provided in another embodiment of this application; Figure 5 This is a schematic diagram of the processing flow of the visual-dynamic dual-confidence fusion decision module provided in another embodiment of this application; Figure 6 A schematic diagram of the functional modules of a road surface friction coefficient estimation system based on visual recognition and dynamic fusion provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] The road friction coefficient estimation method based on visual recognition and dynamics fusion provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on other servers. Terminal 102 can send road surface images and motion state information during the wheeled unmanned platform's operation to server 104. After receiving the road surface images and motion state information, server 104 inputs the road surface images into a pre-trained classification network, outputs a road surface category probability vector, and smooths the road surface category probability vector to determine the visual classification probability, visual prior interval, and visual prior interval center value. Based on the motion state information, it estimates the longitudinal and vertical tire forces of the wheels and calculates the wheel slip ratio. Based on the visual prior interval center value and visual prior interval center value... The classification probability is used to determine the slip slope fitting interval. If the wheel slip ratio falls within the slip slope fitting interval, the slip slope estimate is updated using RLS; otherwise, the slip slope estimate remains unchanged. Based on the pre-calibrated reference road slip slope, reference adhesion coefficient, and slip slope estimate, the dynamic adhesion coefficient estimate is calculated. Based on the visual classification probability, a piecewise function is used to calculate the visual confidence score. Based on motion state information, wheel slip ratio, and slip slope fitting interval, the dynamic confidence score is calculated. The dynamic adhesion coefficient estimate is compared with the visual prior interval, and the visual confidence score and dynamic confidence score are combined for hierarchical decision fusion to output the final road friction coefficient estimate. Server 104 can feed back the obtained final road friction coefficient estimate to terminal 102. Furthermore, in some embodiments, the road friction coefficient estimation method based on visual recognition and dynamics fusion can also be implemented separately by the server 104 or the terminal 102. For example, the terminal 102 can directly perform road friction coefficient estimation processing based on visual recognition and dynamics fusion on the road surface images and motion state information during the driving process of the wheeled unmanned platform. Alternatively, the server 104 can obtain the road surface images and motion state information during the driving process of the wheeled unmanned platform from the data storage system and perform road friction coefficient estimation processing based on visual recognition and dynamics fusion on the road surface images and motion state information.

[0026] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0027] In one exemplary embodiment, such as Figure 2 As shown, a method for estimating road surface friction coefficient based on visual recognition and dynamic fusion is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps A1 to A9. Wherein: Step A1: Obtain road surface images and motion state information during the driving process of the wheeled unmanned platform; the motion state information includes at least the longitudinal velocity of the center of mass of the wheeled unmanned platform, the yaw rate, the angular velocity of each wheel, the longitudinal acceleration, and the lateral acceleration.

[0028] Before step A1 in this embodiment, sensor configuration, calibration, and coordinate system establishment are performed. Specifically, a front-facing camera (binocular or monocular), an inertial measurement unit (IMU), and motor encoders are fixedly installed on the wheeled unmanned platform. The front-facing camera's intrinsic parameters are calibrated to determine its focal length, principal point coordinates, and distortion coefficients; the extrinsic parameters (translation vector and rotation matrix) between the front-facing camera and the wheeled unmanned platform's center of mass are calibrated. The platform's original coordinate system (origin at the wheeled unmanned platform's center of mass, x-axis forward, y-axis left, z-axis upward), the platform's real-time coordinate system (reflecting real-time roll angle, pitch angle, and yaw angle changes with the wheeled unmanned platform's attitude), the camera's original coordinate system, and the camera's real-time coordinate system are established. The inertial measurement unit measures the three-axis angular velocity and three-axis acceleration of the wheeled unmanned platform's center of mass; the motor encoders include wheel speed encoders and joint angle encoders. The wheel speed encoders measure the angular velocity of each wheel, and the joint angle encoders measure the angles and angular velocities of each joint. The timestamps of all sensors are synchronized.

[0029] Step A2: Input the road surface image into the pre-trained classification network, output the road surface category probability vector, and smooth the road surface category probability vector to determine the visual classification probability, visual prior interval, and visual prior interval center value.

[0030] Step A3: Based on the motion state information, estimate the longitudinal tire force and vertical tire force of the wheel, and calculate the wheel slip ratio.

[0031] Step A4: Determine the sliding slope fitting interval based on the visual prior interval center value and the visual classification probability.

[0032] Step A5: If the wheel slip ratio is within the slip slope fitting interval, then update the slip slope estimate using RLS; otherwise, keep the slip slope estimate unchanged. In this embodiment, RLS refers to Recursive Least Squares. The RLS algorithm is an adaptive filtering algorithm that continuously updates the filter coefficients recursively to minimize the weighted sum of squares of the error signal.

[0033] Step A6: Calculate the dynamic adhesion coefficient estimate based on the pre-calibrated reference road surface slip slope, reference adhesion coefficient, and the slip slope estimate.

[0034] Step A7: Based on the visual classification probability, calculate the visual confidence level using a piecewise function.

[0035] Step A8: Calculate the dynamic confidence level based on the motion state information, the wheel slip ratio, and the slip slope fitting interval.

[0036] Step A9: Compare the estimated dynamic adhesion coefficient with the visual prior interval, and perform hierarchical decision fusion by combining the visual confidence and the dynamic confidence to output the final road surface friction coefficient estimate.

[0037] By implementing steps A1 to A9 above, this embodiment of the application can achieve continuous, accurate, and stable estimation of the road friction coefficient without relying on any parameterized tire model. By introducing visual classification information to actively guide the fitting interval of the slip slope using recursive least squares, the problem of low signal-to-noise ratio and large estimation fluctuations in traditional methods under low slip and low excitation conditions is effectively solved. At the same time, by independently quantifying visual confidence and dynamic confidence, and realizing adaptive variable weight fusion based on hierarchical decision logic, high robustness and estimation accuracy can still be maintained under extreme conditions such as visual deception, sudden road surface changes, and sensor anomalies, providing real-time and reliable input for the autonomous operation and safety control of wheeled unmanned platforms.

[0038] In another exemplary embodiment of this application, the pre-trained classification network is an EfficientNet-B0 classification network. Since single-frame road surface images are easily affected by environmental factors such as changes in illumination, shadow occlusion, and water stain reflection, directly using single-frame classification results to determine the road surface type can lead to frequent jumps in visual prior information, reducing the stability of subsequent fusion estimation. Therefore, it is necessary to perform temporal smoothing on the classification results of multiple consecutive frames to suppress the impact of single-frame misjudgments on the reliability of visual prior information. Thus, the step A2 above, "smoothing the road surface category probability vector to determine the visual classification probability, visual prior interval, and the center value of the visual prior interval," is replaced by the following steps B1 to B3: Step B1: Using fixed-window Gaussian smoothing, the probability vectors of the road surface categories in multiple consecutive frames are weighted and averaged to obtain the smoothed probability vector.

[0039] Step B2: Take the maximum value in the smoothed probability vector as the visual classification probability, and determine the category corresponding to the visual classification probability as the current road surface type.

[0040] Step B3: Based on the current road surface type, query the pre-stored road surface type-TRFC interval lookup table to determine the visual prior interval and the center value of the visual prior interval.

[0041] During the operation of the wheeled unmanned platform, the front-mounted camera captures road surface images at a fixed frame rate. For example... Figure 3 As shown, the acquired RGB images are scaled to 224×224×3 pixels and fed into a pre-trained Efficientnet-B0 classification network. This network, based on the Mobile Inverted Bottleneck Convolution module, balances network depth, width, and input resolution through a composite scaling method. While maintaining classification accuracy, it requires only 5.3M parameters and 0.39 GFLOPs of computation, meeting the real-time requirements of automotive embedded platforms. The Efficientnet-B0 classification network was trained on the RSCD road image dataset, covering... N For typical road surfaces, output probability vectors for each category. ,in, Indicates the first n The predicted probability of a typical road surface at time t. .

[0042] To avoid random fluctuations in single-frame classification results, the probability vector is smoothed using a fixed-window Gaussian smoothing. Let the half-window size be... (Offset) Gaussian weights are ,in, This means that the standard deviation of the Gaussian kernel is set to one-third of the half-window length, causing the weights at the window boundaries to decay to approximately 1% of the center weights, thus ensuring smoothness while avoiding the introduction of outdated information. After smoothing, the... n The probability of a class is equal to the weighted sum of the predicted probabilities of each frame and their corresponding Gaussian weights, divided by the total sum of the weights. Therefore, the road type at the current moment is determined as the class with the highest smoothed probability, corresponding to the visual classification probability. This represents the maximum value in the smoothed probability vector. The visual prior interval is obtained based on the pre-stored road surface type-TRFC interval lookup table. and its central value .

[0043] In another exemplary embodiment of this application, to further improve the continuity and accuracy of road surface friction coefficient estimation, dynamic information is introduced for fusion based on the above visual classification. Therefore, step A3 is replaced by steps C1 to C4: Step C1: Based on the wheel angular velocity and combined with the wheel rotation dynamics equation, estimate the longitudinal tire force of each wheel.

[0044] Step C2: Based on the longitudinal acceleration and the lateral acceleration, and in conjunction with the vehicle dynamics load transfer formula, calculate the vertical tire force of each wheel.

[0045] Step C3: Calculate the longitudinal velocity of each wheel hinge point based on the longitudinal velocity of the center of mass and the yaw rate.

[0046] Step C4: Calculate the wheel slip ratio based on the wheel angular velocity and the longitudinal velocity, combined with the wheel's effective rolling radius.

[0047] This application's embodiments use tire dynamics to estimate longitudinal tire force. The equation for the rotational dynamics of a wheel is: ,in, For wheel drive or braking torque, Let be the derivative of the wheel's angular velocity. For the moment of inertia of the wheel, This is the effective rolling radius of the wheel.

[0048] For wheeled unmanned platforms, the vertical tire force of each wheel The load transfer formula is used for vehicle dynamics calculations, without relying on any tire model. It is derived from the longitudinal acceleration of the entire vehicle. Lateral acceleration Vehicle body quality Wheelbase Distance from the front and rear axles to the center of gravity of the wheeled unmanned platform and Wheelbase Center of mass height The vertical tire force of each wheel is calculated using the following formula, taking into account parameters such as: ; ; ; ; in, and Indicates static axle load. This indicates the load transfer caused by longitudinal acceleration. This indicates load transfer caused by lateral acceleration. This indicates the longitudinal and lateral coupling correction term. Represents the left front wheel. Represents the right front wheel. Represents the left rear wheel. Represents the right rear wheel. This refers to the vertical tire force corresponding to the left front wheel. This represents the vertical tire force corresponding to the right front wheel. The vertical tire force corresponding to the left rear wheel. This represents the vertical tire force corresponding to the right rear wheel. This represents gravitational acceleration. All parameters are known structural parameters of the wheeled unmanned platform or measured values ​​from the IMU. For platforms with active suspension or deformable chassis, the vertical tire forces of each wheel must also consider the additional loads caused by the movement of the suspension or leg mechanism.

[0049] The longitudinal velocity of the center of mass of the wheeled unmanned platform and yaw rate (Obtained by fusing inertial measurement unit and wheel speed information), combined with the geometric dimensions of the wheeled unmanned platform, the longitudinal velocity of each wheel hinge point is calculated.

[0050] ; in, Indicates the front and rear axles. Indicate left and right sides. The left wheel is represented by a negative sign, and the right wheel by a positive sign.

[0051] Wheel angular velocity measured by wheel speed encoder Combined with the effective rolling radius of the wheel Wheel slip ratio Defined as the linear velocity of the wheel ( The difference between the longitudinal velocity of the wheel and the longitudinal velocity of the wheel hinge point is divided by the larger of the two values.

[0052] ; in, Indicates the front and rear axles. Indicates left and right sides.

[0053] In another exemplary embodiment of this application, considering the significant differences in the linear range of tire force-slip ratio for different adhesion coefficients of the road surface, the linear range of a high-adhesion road surface can extend to a large slip ratio, while the linear range of a low-adhesion road surface enters the nonlinear saturation region at a small slip ratio. Therefore, it is necessary to dynamically adjust the slip slope fitting interval based on the visual classification probability and the visual prior interval center value so that RLS can identify the current road surface type within the effective linear range. Thus, step A4 above is replaced by steps D1 to D4: Step D1: Calculate the optimal slip ratio using a pre-calibrated empirical mapping curve based on the center value of the visual prior interval.

[0054] Step D2: When the visual classification probability is greater than or equal to a preset threshold, the optimal slip ratio is used as the upper limit of the slip slope fitting interval; otherwise, a preset conservative upper bound is used as the upper limit of the slip slope fitting interval.

[0055] Step D3: Use the preset minimum slip ratio threshold as the lower limit of the slip slope fitting interval.

[0056] Step D4: Determine the sliding slope fitting interval based on the upper limit and the lower limit of the sliding slope fitting interval.

[0057] The visual prior interval obtained from the aforementioned steps central value and visual classification probability The optimal slip ratio is used to determine the fitting interval for the adaptive slip slope of the visual guidance. .

[0058] Traditional RLS methods use a fixed conservative fitting interval. Data outside this range is discarded, resulting in insufficient information utilization. This application's embodiments utilize visual classification probability. Dynamically adjust the upper bound of the fitted interval. Based on the center value of the visual prior interval. The optimal slip ratio is calculated using an empirical mapping curve. Where a, b, and c are pre-calibrated empirical coefficients. When the visual classification probability... Greater than or equal to the preset threshold When the value is 0.9, trusting the visual prior, the upper bound of the sliding slope fitting interval is extended to... Otherwise, revert to the conservative upper limit. The lower limit of the sliding slope fitting interval remains unchanged. If it remains unchanged, then we have the following equation: .

[0059] In another exemplary embodiment of this application, to further improve the convergence speed and real-time performance of the slip slope estimation, recursive least squares with a forgetting factor is introduced for online parameter identification based on the slip slope fitting interval determined by the above visual guidance. Therefore, the step A5 above, "If the wheel slip ratio is within the slip slope fitting interval, then update the slip slope estimate using RLS," is replaced by steps E1 to E4: Step E1: Initialize the estimated covariance and slip slope values.

[0060] Step E2: At each valid time step, calculate the gain based on the estimated covariance of the previous time step, the wheel slip rate at the current time step, and the forgetting factor.

[0061] Step E3: Update the estimated slip slope at the current moment based on the gain, the longitudinal force coefficient at the current moment, the estimated slip slope at the previous moment, and the wheel slip ratio at the current moment; the longitudinal force coefficient at the current moment is the ratio of the longitudinal tire force at the current moment to the vertical tire force at the current moment.

[0062] Step E4: Update the estimated covariance at the current time based on the gain, the estimated slip slope at the current time, and the estimated covariance at the previous time.

[0063] In this embodiment of the application, only when the wheel slip ratio satisfy Only when the condition is met should an RLS update be performed; otherwise, the slip slope estimate should be kept unchanged.

[0064] Initialize covariance estimation Slip slope estimate .

[0065] At each effective moment, execute the process with a forgetting factor. The recursive update begins with calculating the gain. : .

[0066] Then update the slip slope estimate. : .

[0067] in, This represents the longitudinal force coefficient at the current moment.

[0068] Last update estimates covariance : .

[0069] Forgetting factor By controlling the decay rate of historical data, the estimator can track time-varying road conditions.

[0070] In another exemplary embodiment of this application, a calibration experiment is conducted on a dry asphalt reference pavement to obtain a reference slip slope. Let the reference adhesion coefficient of dry asphalt be... Then the estimated value of the dynamic adhesion coefficient of the current road surface equal Divide by Multiply by the current estimated slip slope. ,Right now .

[0071] The above steps demonstrate the specific workflow of the dynamics identification module, such as... Figure 4 As shown, the vehicle motion state information is collected in real time through IMU and motor encoder, and the visual prior interval and visual classification probability output by the visual classification module are received simultaneously. The slip slope fitting interval is dynamically updated based on the visual information. The normalized longitudinal force and slip ratio data obtained by longitudinal force estimation, vertical force estimation and slip ratio calculation are fitted using the least squares method to identify the slip slope online, providing a basis for subsequent dynamic adhesion coefficient estimation.

[0072] Then, a visual-dynamic fusion process is performed, such as... Figure 5 As shown below, the process will be described in detail.

[0073] In another exemplary embodiment of this application, the piecewise function is a piecewise tanh-shift function; to further quantify the reliability of the visual classification results and provide a basis for subsequent visual-dynamic fusion decisions, the visual confidence level is calculated through the visual classification probability, and the above step A7 is replaced by the following steps F1~F2: Step F1: When the visual classification probability is greater than or equal to the first preset threshold, the visual confidence level is set to 1.

[0074] Step F2: When the visual classification probability is less than the first preset threshold, the visual confidence value is [value to be filled in]. ;in, This is an adjustable coefficient. P For visual classification probability.

[0075] This application embodiment is based on the visual classification probability obtained above. Visual confidence was calculated using a piecewise tanh-shift function. The calculation formula is: .

[0076] when P When the visual confidence level is greater than or equal to 0.9, the visual confidence level is set to 1; when... P When the visual confidence score is less than 0.9, it equals .

[0077] In another exemplary embodiment of this application, to further quantify the reliability of the dynamic estimation results and provide a basis for subsequent vision-dynamic fusion decision-making, the dynamic confidence level is calculated based on the vehicle motion state information, wheel slip ratio, and slip slope fitting interval. Therefore, step A8 is replaced by steps G1 to G4: Step G1: Determine the center of the fitting interval based on the slip slope fitting interval, and use the center of the fitting interval as a reference, combined with the wheel slip ratio, to calculate the slip ratio confidence factor using the Cauchy distribution function.

[0078] Step G2: Calculate the velocity factor based on the longitudinal velocity of the centroid.

[0079] Step G3: Calculate the acceleration-slip joint factor based on the longitudinal acceleration and the slip ratio confidence factor.

[0080] Step G4: Multiply the velocity factor by the acceleration-slip joint factor to calculate the dynamic confidence level.

[0081] The dynamic confidence level of the embodiments in this application The degree of fit between the vehicle speed, acceleration, wheel slip ratio, and slip slope fitting intervals is considered. The center of the fitting interval is defined. half width ,in The scaling factor is greater than zero. The Cauchy distribution is used to characterize the slip ratio confidence level, and the slip ratio confidence factor is... equal ,in The amplitude coefficient is between 0 and 1. Wheel slip ratio. Speed ​​factor. Acceleration-slip joint factor Ultimately, the dynamic confidence level... It equals the velocity factor multiplied by the acceleration-slip joint factor, i.e. Multiply ,in, , This is a positive, adjustable parameter. The product structure ensures that the dynamic confidence level is high only when the wheeled unmanned platform has a certain speed, sufficient acceleration, and the wheel slip ratio is close to the center of the slip slope fitting interval. Dynamic Confidence Level The calculation formula is: .

[0082] In another exemplary embodiment of this application, in order to achieve adaptive fusion of visual and dynamic information, and to select the optimal estimation result or a smooth transition value between the two under different visual and dynamic reliability conditions, thereby improving the robustness and continuity of the road friction coefficient estimation, the above step A9 is replaced by the following steps H1 to H4: Step H1: At each valid time, compare the current dynamic adhesion coefficient estimate with the visual prior interval. If the current dynamic adhesion coefficient estimate is within the visual prior interval, output the current dynamic adhesion coefficient estimate as the final road friction coefficient estimate.

[0083] Step H2: If the estimated value of the dynamic adhesion coefficient at the current moment is not within the visual prior interval, and the dynamic confidence level is greater than the preset dynamic confidence level threshold, then the estimated value of the dynamic adhesion coefficient at the current moment is output as the final estimated value of the road surface friction coefficient.

[0084] Step H3: If the current estimated value of the dynamic adhesion coefficient is not within the visual prior interval, the dynamic confidence is less than or equal to the preset dynamic confidence threshold, and the visual confidence is greater than the preset visual confidence threshold, then a first-order recursive method is used to smoothly transition to the center value of the visual prior interval, and the transition value is output as the final estimated value of the road surface friction coefficient.

[0085] Step H4: If the current estimated value of the dynamic adhesion coefficient is not within the visual prior interval, the dynamic confidence level is less than or equal to the preset dynamic confidence level threshold, and the visual confidence level is less than or equal to the preset visual confidence level threshold, then the final road surface friction coefficient estimate output at the previous moment remains unchanged and is used as the final road surface friction coefficient estimate.

[0086] In this embodiment, the estimated dynamic adhesion coefficient is compared at each time step. visual prior interval ,refer to and The following four types of decisions are made based on the threshold: The first type is the consistent case: if the estimated dynamic adhesion coefficient is within the visual prior interval, then the final estimated road surface friction coefficient is directly output. This is equal to the estimated value of the dynamic adhesion coefficient. At this point, the visual and dynamic results corroborate each other, resulting in the highest confidence level.

[0087] The second category is dynamically reliable but deviates from visual estimation: if the estimated dynamic adhesion coefficient is not within the visual prior interval but the dynamic confidence level is greater than 0.7, the final road surface friction coefficient estimate will still be output. It equals the estimated value of the dynamic adhesion coefficient because the dynamic confidence level is high and can cover the visual prior.

[0088] The third category is unreliable dynamics but reliable visuals: If the estimated dynamic adhesion coefficient is not within the visual prior interval, the dynamic confidence level is less than or equal to 0.7, and the visual confidence level is greater than 0.5, then a first-order recursive approach is used to smoothly transition to the visual center, and the final estimated road surface friction coefficient is obtained. The smoothing coefficient Between 0 and 1, This represents the estimated final road surface friction coefficient at the previous moment.

[0089] The fourth category is unreliable for both sources: if the estimated value of the dynamic adhesion coefficient is not within the visual prior interval, the dynamic confidence is less than or equal to 0.7 and the visual confidence is less than or equal to 0.5, then the output of the previous time step remains unchanged.

[0090] In another exemplary embodiment of this application, prior to step A9, the method further includes: The visual prior information and dynamic information are spatiotemporally aligned; the visual prior information includes the visual prior interval, the road surface type corresponding to the visual prior interval, the center value of the visual prior interval, the visual classification probability, and the visual confidence level; the dynamic information includes the estimated value of the dynamic adhesion coefficient and the dynamic confidence level.

[0091] The spatiotemporal alignment specifically includes: The travel time corresponding to the aiming distance is calculated based on the longitudinal velocity of the center of mass and the longitudinal acceleration.

[0092] The visual prior information at the current moment is delayed by the travel time and then aligned with the dynamic information so that the visual prior information and the dynamic information are for the same road surface area.

[0093] Specifically, when the absolute value of the longitudinal acceleration is greater than or equal to a preset acceleration threshold, a uniform acceleration model is used to calculate the travel time; when the absolute value of the longitudinal acceleration is less than the preset acceleration threshold, a uniform speed model is used to calculate the travel time.

[0094] In this embodiment, at each fusion moment, the visual dynamics fusion module receives two sets of spatiotemporally aligned inputs: one set is delay-compensated visual information, including road surface type and visual prior interval. and its central value Visual classification probability and its confidence level The other set is the estimated dynamic adhesion coefficient at the current moment. and its confidence level Since the two sets of information have been aligned in time and space, they describe the same road surface, making fusion decision-making meaningful.

[0095] For front-facing cameras, the travel time from the moment of shooting to the vehicle's arrival at that road surface area needs to be considered. This is based on the current longitudinal velocity of the center of gravity. and longitudinal acceleration Calculate the aiming distance Corresponding travel time When the absolute value of longitudinal acceleration is greater than or equal to a preset acceleration threshold, a uniform acceleration model is used to calculate the travel time; otherwise, a uniform velocity model is used. Visual prior information is then combined with... The dynamic information is then aligned to ensure that both refer to the same physical road block.

[0096] Travel time The calculation formula is as follows: .

[0097] in, This is a preset acceleration threshold.

[0098] Finally, TRFC The system outputs data in real time to the upper-level controller (such as model predictive control or traction control system) at a frequency of not less than 10Hz, while simultaneously recording intermediate quantities such as visual classification probability, dynamic confidence level, and slip slope estimate for system monitoring and subsequent analysis.

[0099] This application also provides an application scenario in which the aforementioned method for estimating the road surface friction coefficient based on the fusion of visual recognition and dynamics is applied. Specifically, the method for estimating the road surface friction coefficient based on the fusion of visual recognition and dynamics provided in this application embodiment can be applied to the autonomous driving scenario of a wheeled unmanned platform. This autonomous driving scenario includes an environmental perception stage, a decision-making and planning stage, and a motion control stage. During the driving process, the wheeled unmanned platform collects road surface images and motion state information in real time through onboard sensors, estimates the current road surface friction coefficient in real time using the method described in this application embodiment, and outputs the estimation result to the decision-making and planning stage and the motion control stage. The method for estimating the road surface friction coefficient based on the fusion of visual recognition and dynamics provided in this application embodiment belongs to the key parameter estimation part of the environmental perception stage. Specifically, during the autonomous driving process of the wheeled unmanned platform, the road surface friction coefficient can be obtained in real time based on the fusion of visual recognition and dynamics, providing an accurate reference for road surface adhesion capability for the upper-level decision-making and control module.

[0100] Based on the same inventive concept, this application also provides a road friction coefficient estimation system based on visual recognition and dynamics fusion to implement the road friction coefficient estimation method based on visual recognition and dynamics fusion described above. The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more road friction coefficient estimation system embodiments based on visual recognition and dynamics fusion provided below can be found in the limitations of the road friction coefficient estimation method based on visual recognition and dynamics fusion described above, and will not be repeated here.

[0101] In one exemplary embodiment, such as Figure 6 As shown, a road surface friction coefficient estimation system based on the fusion of visual recognition and dynamics is provided, including: The visual classification module is used to acquire road surface images during the operation of the wheeled unmanned platform; it is also used to input the road surface images into a pre-trained classification network, output a road surface category probability vector, and smooth the road surface category probability vector to determine the visual classification probability, the visual prior interval, and the center value of the visual prior interval. This embodiment outputs the probabilities of eight road surface types.

[0102] The dynamic identification module is used to acquire motion state information of the wheeled unmanned platform during its operation; it is also used to estimate the longitudinal and vertical tire forces of the wheels based on the motion state information, and calculate the wheel slip ratio; it is also used to determine the slip slope fitting interval based on the visual prior interval center value and the visual classification probability; if the wheel slip ratio is within the slip slope fitting interval, the slip slope estimate is updated through RLS; otherwise, the slip slope estimate remains unchanged; it is also used to calculate the dynamic adhesion coefficient estimate based on the pre-calibrated reference road slip slope, reference adhesion coefficient, and the slip slope estimate.

[0103] The visual-dynamic fusion module is used to calculate the visual confidence score based on the visual classification probability using a piecewise function; it is also used to calculate the dynamic confidence score based on the motion state information, the wheel slip ratio, and the slip slope fitting interval; it is also used to compare the estimated dynamic adhesion coefficient with the visual prior interval, and combine the visual confidence score and the dynamic confidence score to perform hierarchical decision fusion, and output the final road surface friction coefficient estimate.

[0104] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 7As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores road surface images and motion status information during the operation of the wheeled unmanned platform. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a method for estimating the road surface friction coefficient based on visual recognition and dynamic fusion.

[0105] Figure 7 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0106] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0107] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations and be authorized by the owner of the corresponding device.

[0109] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0110] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0111] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for estimating the road surface friction coefficient based on visual recognition and dynamic fusion, characterized in that, The method includes: Acquire road surface images and motion state information during the driving process of the wheeled unmanned platform; the motion state information includes at least the longitudinal velocity of the center of gravity of the wheeled unmanned platform, the yaw rate, the angular velocity of each wheel, the longitudinal acceleration, and the lateral acceleration. The road surface image is input into a pre-trained classification network, which outputs a road surface category probability vector. The road surface category probability vector is then smoothed to determine the visual classification probability, the visual prior interval, and the center value of the visual prior interval. Based on the motion state information, the longitudinal tire force and vertical tire force of the wheel are estimated, and the wheel slip ratio is calculated. Based on the visual prior interval center value and the visual classification probability, determine the sliding slope fitting interval; If the wheel slip ratio is within the slip slope fitting range, then the slip slope estimate is updated using RLS; otherwise, the slip slope estimate remains unchanged. Based on the pre-calibrated reference road surface slip slope, reference adhesion coefficient, and the estimated slip slope value, the estimated value of the dynamic adhesion coefficient is calculated; Based on the visual classification probability, a piecewise function is used to calculate the visual confidence score; The dynamic confidence level is calculated based on the motion state information, the wheel slip ratio, and the slip slope fitting interval; The estimated dynamic adhesion coefficient is compared with the visual prior interval, and a hierarchical decision fusion is performed by combining the visual confidence and the dynamic confidence to output the final road surface friction coefficient estimate.

2. The method for estimating road surface friction coefficient based on visual recognition and dynamic fusion according to claim 1, characterized in that, The pre-trained classification network is the EfficientNet-B0 classification network; the road surface category probability vector is smoothed to determine the visual classification probability, visual prior interval, and visual prior interval center value, specifically including: A fixed-window Gaussian smoothing method is used to perform a weighted average of the road surface category probability vectors in multiple consecutive frames to obtain a smoothed probability vector. The maximum value in the smoothed probability vector is taken as the visual classification probability, and the category corresponding to the visual classification probability is determined as the current road surface type. Based on the current road surface type, the pre-stored road surface type-TRFC interval lookup table is queried to determine the visual prior interval and the center value of the visual prior interval.

3. The method for estimating road surface friction coefficient based on visual recognition and dynamic fusion according to claim 1, characterized in that, Based on the aforementioned motion state information, the longitudinal and vertical tire forces of the wheel are estimated, and the wheel slip ratio is calculated, specifically including: Based on the wheel angular velocity and combined with the wheel rotation dynamics equation, the longitudinal tire force of each wheel is estimated. Based on the longitudinal acceleration and the lateral acceleration, and combined with the vehicle dynamics load transfer formula, the vertical tire force of each wheel is calculated; Calculate the longitudinal velocity of each wheel hinge point based on the longitudinal velocity of the center of mass and the yaw rate; The wheel slip ratio is calculated based on the wheel angular velocity and the longitudinal velocity, combined with the wheel's effective rolling radius.

4. The method for estimating road surface friction coefficient based on visual recognition and dynamic fusion according to claim 1, characterized in that, Based on the visual prior interval center value and the visual classification probability, the sliding slope fitting interval is determined, specifically including: The optimal slip ratio is calculated using the pre-calibrated empirical mapping curve based on the center value of the visual prior interval. When the visual classification probability is greater than or equal to a preset threshold, the optimal slip ratio is used as the upper limit of the slip slope fitting interval; otherwise, a preset conservative upper bound is used as the upper limit of the slip slope fitting interval. The preset minimum slip ratio threshold is used as the lower limit of the slip slope fitting interval; The slip slope fitting interval is determined based on the upper limit and the lower limit of the slip slope fitting interval.

5. The method for estimating road surface friction coefficient based on visual recognition and dynamic fusion according to claim 1, characterized in that, If the wheel slip ratio is within the slip slope fitting interval, the slip slope estimate is updated using RLS, specifically including: Initialize and estimate the covariance and slip slope; At each effective time step, the gain is calculated based on the estimated covariance of the previous time step, the wheel slip rate at the current time step, and the forgetting factor. The slip slope estimate at the current moment is updated based on the gain, the longitudinal force coefficient at the current moment, the slip slope estimate at the previous moment, and the wheel slip ratio at the current moment; the longitudinal force coefficient at the current moment is the ratio of the longitudinal tire force at the current moment to the vertical tire force at the current moment. The estimated covariance at the current time is updated based on the gain, the estimated slip slope at the current time, and the estimated covariance at the previous time.

6. The method for estimating road surface friction coefficient based on visual recognition and dynamic fusion according to claim 1, characterized in that, The piecewise function is a piecewise tanh-shift function; Based on the visual classification probability, a piecewise function is used to calculate the visual confidence score, specifically including: When the visual classification probability is greater than or equal to the first preset threshold, the visual confidence level is 1. When the visual classification probability is less than the first preset threshold, the visual confidence score is set to [value]. ;in, is an adjustable coefficient, and P is the visual classification probability.

7. The method for estimating road surface friction coefficient based on visual recognition and dynamic fusion according to claim 1, characterized in that, The dynamic confidence score is calculated based on the motion state information, the wheel slip ratio, and the slip slope fitting interval, specifically including: The center of the fitting interval is determined based on the slip slope fitting interval, and the center of the fitting interval is used as a reference. Combined with the wheel slip ratio, the Cauchy distribution function is used to calculate the slip ratio confidence factor. Calculate the velocity factor based on the longitudinal velocity of the centroid; Calculate the acceleration-slip joint factor based on the longitudinal acceleration and the slip ratio confidence factor; The dynamic confidence level is calculated by multiplying the velocity factor by the acceleration-slip joint factor.

8. The method for estimating road surface friction coefficient based on visual recognition and dynamic fusion according to claim 1, characterized in that, The estimated dynamic adhesion coefficient is compared with the visual prior interval, and a hierarchical decision fusion is performed by combining the visual confidence score and the dynamic confidence score to output the final estimated road surface friction coefficient, specifically including: At each valid moment, the estimated value of the dynamic adhesion coefficient at the current moment is compared with the visual prior interval. If the estimated value of the dynamic adhesion coefficient at the current moment is within the visual prior interval, the estimated value of the dynamic adhesion coefficient at the current moment is output as the final estimated value of the road surface friction coefficient. If the current estimated value of the dynamic adhesion coefficient is not within the visual prior interval, and the dynamic confidence level is greater than the preset dynamic confidence level threshold, then the current estimated value of the dynamic adhesion coefficient is output as the final estimated value of the road surface friction coefficient. If the current estimated value of the dynamic adhesion coefficient is not within the visual prior interval, the dynamic confidence is less than or equal to the preset dynamic confidence threshold, and the visual confidence is greater than the preset visual confidence threshold, then a first-order recursive method is used to smoothly transition to the center value of the visual prior interval, and the transition value is output as the final estimated value of the road surface friction coefficient. If the current estimated value of the dynamic adhesion coefficient is not within the visual prior interval, the dynamic confidence level is less than or equal to the preset dynamic confidence level threshold, and the visual confidence level is less than or equal to the preset visual confidence level threshold, then the final road surface friction coefficient estimate output at the previous moment remains unchanged and is used as the final road surface friction coefficient estimate.

9. The method for estimating road surface friction coefficient based on visual recognition and dynamic fusion according to claim 1, characterized in that, Before comparing the estimated dynamic adhesion coefficient with the visual prior interval, the method further includes: The visual prior information and dynamic information are spatiotemporally aligned; the visual prior information includes the visual prior interval, the road surface type corresponding to the visual prior interval, the center value of the visual prior interval, the visual classification probability, and the visual confidence level; the dynamic information includes the estimated value of the dynamic adhesion coefficient and the dynamic confidence level. The spatiotemporal alignment specifically includes: Calculate the travel time corresponding to the aiming distance based on the longitudinal velocity of the center of mass and the longitudinal acceleration; The visual prior information at the current moment is delayed by the driving time and then aligned with the dynamic information so that the visual prior information and the dynamic information are for the same road surface area; Specifically, when the absolute value of the longitudinal acceleration is greater than or equal to a preset acceleration threshold, a uniform acceleration model is used to calculate the travel time; when the absolute value of the longitudinal acceleration is less than the preset acceleration threshold, a uniform speed model is used to calculate the travel time.

10. A road surface friction coefficient estimation system based on visual recognition and dynamics fusion, characterized in that, The system includes: The visual classification module is used to acquire road surface images during the driving process of the wheeled unmanned platform; it is also used to input the road surface images into a pre-trained classification network, output a road surface category probability vector, and smooth the road surface category probability vector to determine the visual classification probability, visual prior interval, and visual prior interval center value. The dynamic identification module is used to acquire motion state information of the wheeled unmanned platform during its operation; it is also used to estimate the longitudinal and vertical tire forces of the wheels based on the motion state information, and calculate the wheel slip ratio; it is also used to determine the slip slope fitting interval based on the visual prior interval center value and the visual classification probability; if the wheel slip ratio is within the slip slope fitting interval, the slip slope estimate is updated through RLS; otherwise, the slip slope estimate remains unchanged; it is also used to calculate the dynamic adhesion coefficient estimate based on the pre-calibrated reference road slip slope, reference adhesion coefficient, and the slip slope estimate. The visual-dynamic fusion module is used to calculate the visual confidence score based on the visual classification probability using a piecewise function; it is also used to calculate the dynamic confidence score based on the motion state information, the wheel slip ratio, and the slip slope fitting interval; it is also used to compare the estimated dynamic adhesion coefficient with the visual prior interval, and combine the visual confidence score and the dynamic confidence score to perform hierarchical decision fusion, and output the final road surface friction coefficient estimate.

Images