Multi-source fusion estimation method and system for side slip angle and road adhesion coefficient

Abstract

The invention relates to the technical field of active safety control of intelligent automobiles, in particular to a multi-source fusion estimation method and system for a side slip angle and a road adhesion coefficient, and aims to realize step-by-step estimation of the side slip angle and the road adhesion coefficient through a cascade nonlinear observer architecture, solve the problem of joint estimation caused by dynamic coupling of the side slip angle and the road adhesion coefficient, and improve the safety of the side slip angle and the road adhesion coefficient. Based on an estimation result of a cascade nonlinear observer, a joint estimation method for the side slip angle and the road adhesion coefficient based on an exogenous Kalman filtering framework is provided, the problem that stability and optimality / suboptimum are difficult to consider in a traditional method is solved, lane line information is recognized through visual recognition, and a vehicle kinematics model is combined, so that the road adhesion coefficient is estimated. According to the method, a Kalman filter of the vehicle side slip angle is constructed to output expectation and variance, so that the uncertainty of visual estimation of the side slip angle is quantified, meanwhile, after visual-based road surface classification, the expectation and variance of a visual estimation road surface adhesion coefficient are obtained by adopting a Bayesian method, and subsequent multi-source information fusion is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent automobile active safety control, and particularly relates to a mass center side slip angle and road adhesion coefficient multi-source fusion estimation method and system. BACKGROUND

[0002] With the rapid development of intelligent automobile active safety control technology, the advanced driving assistance system (ADAS) puts forward higher requirements for the control accuracy of vehicle dynamics, and the mass center side slip angle and the road adhesion coefficient as core state parameters directly affect the lateral stability control (such as ESC), the trajectory tracking performance and the emergency obstacle avoidance ability.

[0003] At present, the estimation methods of the mass center side slip angle can be mainly divided into: a method based on a kinematic model, a method based on a dynamic model and a method based on multi-source information fusion. The method based on the kinematic model generally directly integrates the kinematic equation and the vehicle state information, or designs an observer to estimate the mass center side slip angle based on the model, but it is too dependent on the sensor accuracy and cannot avoid the integral cumulative error. The general idea of the method based on the dynamic model is to establish an observer to estimate the mass center side slip angle based on the vehicle dynamics model and the tire model, which depends on the modeling and sensor accuracy. In addition, some existing methods based on the fusion of dynamics and kinematics, and the fusion with visual information lack the fusion of uncertainties and are difficult to improve the robustness to road adhesion conditions.

[0004] The current mainstream road adhesion coefficient estimation methods can be divided into Cause-based and Effect-based two categories, both of which have significant limitations. The former mainly identifies the road state through the camera information of the vehicle to distinguish dry, wet, snow and ice road surfaces, but cannot provide accurate road adhesion coefficient and is easily affected by weather conditions and environment. The latter generally estimates the road adhesion coefficient based on vehicle dynamics, but the estimation accuracy is affected by driving excitation, and generally the greater the excitation, the higher the estimation accuracy.

[0005] The current joint estimation of the mass center side slip angle and the road peak adhesion coefficient mainly exists in two technical routes of series and parallel. The series strategy estimates in steps to reduce the design complexity, estimates the mass center side slip angle based on the kinematic / dynamic model fusion first, and then estimates the road adhesion coefficient combined with the tire force. Since the series strategy cannot realize the synchronous estimation of the mass center side slip angle and the peak adhesion coefficient, some scholars also use the parallel strategy of the nonlinear adaptive observer or the improved double extended Kalman filter to estimate the mass center side slip angle and the road peak adhesion coefficient synchronously. Since the series strategy introduces time lag due to step-by-step estimation, and the parallel strategy is limited by the strong coupling between the mass center side slip angle and the road adhesion coefficient, the existing joint estimation research generally has the bottleneck that the estimation accuracy and convergence speed are difficult to balance. SUMMARY

[0006] To solve the above technical problems in the prior art, the application provides a mass center side slip angle and road adhesion coefficient multi-source fusion estimation method and system.

[0007] To solve the above technical problems, the application provides the following technical scheme: a mass center side slip angle and road adhesion coefficient multi-source fusion estimation method and system, the estimation method specifically includes the following steps:

[0008] S1, estimating the mass center side slip angle and the road adhesion coefficient based on dynamics, outputting the estimated value and variance of the vehicle mass center side slip angle estimated by dynamics and the estimated value and variance of the road adhesion coefficient;

[0009] S2, estimating the mass center side slip angle and the road adhesion coefficient based on visual information, outputting the estimated value and variance of the mass center side slip angle estimated by vision, the estimated value and variance of the road adhesion coefficient and the classification confidence;

[0010] S3, adjusting the fusion estimation framework by considering the observation noise, and outputting the estimated value and variance of the mass center side slip angle and the estimated value and variance of the road adhesion coefficient after fusion of the dynamics estimation and the vision estimation, which can be used as input parameters of a vehicle advanced control system.

[0011] Preferably, the step S1 specifically includes the following contents:

[0012] S11, analyzing the steering and lateral motion characteristics of the vehicle through a vehicle single-track model;

[0013] S12, describing the tire side stiffness and the road adhesion coefficient through a brush model, capturing the dynamic response of the tire force changing with the adhesion condition, and accurately predicting the tire force, the tire lateral force based on the brush model is For:

[0014]

[0015] In the formula, represents the road adhesion coefficient, represents the vertical force, represents the tire side stiffness, represents the tire side slip angle, , , the segment point , represents the vertical force, represents a normalized parameter determined by the tire stiffness, the vertical load and the road adhesion coefficient, represents the equivalent lateral slip amount of the tire;

[0016] S13, designing a cascaded nonlinear observer to preliminarily estimate the mass center side slip angle and the road adhesion coefficient, and taking the estimated result as a pseudo-measurement value;

[0017] S14 linearizes the vehicle lateral dynamics model and measurement model, and uses Kalman filtering to make optimal estimates of the centroid sideslip angle and road adhesion coefficient.

[0018] Preferably, in step S1, when analyzing the vehicle's steering and lateral motion characteristics, the vehicle's lateral and yaw motion equations are as follows:

[0019]

[0020]

[0021] in,

[0022]

[0023]

[0024]

[0025] In the formula, , These represent the front and rear axle lateral forces calculated based on the brush tire model. , These represent the distances from the center of mass to the front and rear axles, respectively. Indicates the overall vehicle weight. Indicates the whole vehicle is around Moment of inertia in the axial direction, Indicates yaw rate. The derivative of the yaw rate. , Indicates the longitudinal and lateral velocities at the center of mass. Indicates the road surface adhesion coefficient. Indicates the front wheel steering angle. , Indicates the front and rear axle slip angles.

[0026] Preferably, in step S13, the sideslip angle of the vehicle's center of gravity is the angle between the lateral velocity and the longitudinal velocity at the center of gravity. for:

[0027]

[0028] In the formula, This represents the estimated longitudinal velocity. Indicates the estimated lateral speed;

[0029] Let the longitudinal velocity be... It is known that the estimated value is obtained through adaptive adjustment. Value estimation observer estimates lateral vehicle speed ,when Proximity measurement The specific observers are as follows:

[0030]

[0031]

[0032] In the formula: This represents the lateral force when the vehicle's road surface adhesion coefficient is 1. This represents the adaptive parameter, which ranges from [0.05 to 1.2].

[0033] Based on the estimated lateral velocity The designed road surface adhesion coefficient observer is as follows:

[0034]

[0035]

[0036]

[0037] In the formula: , This represents the lateral tire forces on the front and rear axles calculated based on the vehicle's monorail model. Indicates based on The front and rear axle slip angles calculated from the lateral velocity. This indicates that the estimated values ​​of the lateral forces on the front and rear axles are calculated based on the skid angle using a brush model. Indicates the gain of the observer. This represents the estimated value of the road surface adhesion coefficient.

[0038] Preferably, step S14 is based on the equations of lateral and yaw motion, when When = 0, the system state equation after Euler discretization is:

[0039]

[0040]

[0041] in, , , , ;

[0042] The state equations obtained above are in Linearization yields:

[0043]

[0044]

[0045] The expected centroid sideslip angle is estimated using the standard formula of Kalman filtering and then output by the dynamic estimation module. and variance Expected value of road surface adhesion coefficient and variance This result will serve as one of the inputs to the information fusion module, and together with the visual estimation result, complete the fusion estimation.

[0046] Preferably, step S2 specifically includes the following:

[0047] S21. The front-facing camera acquires lane line information. The camera is monocular. The image information is preprocessed using OpenCV, including image reading, resizing, image enhancement, and normalization. The preprocessed image is then input into the semantic segmentation network DeepLabV3. DeepLabV3 performs road segmentation on the image information. DeepLabV3 has been trained offline on a labeled dataset containing various road scene scenarios.

[0048] S22, DeepLabV3 outputs segmentation results, in which the road surface area and non-road surface area are effectively distinguished, and lane line information is identified. Using the camera preview point as a bridge, a geometric kinematic model between the lane lines and the vehicle is established, and a centroid sideslip angle estimator is built.

[0049] The state equation for the vehicle's sideslip angle in this vehicle center of gravity sideslip angle estimator is:

[0050]

[0051]

[0052] In the formula, This represents the centroid sideslip angle to be estimated. The derivative of Indicates the vehicle's yaw angle. This indicates the lateral distance between the camera's preview point and the lane line. The derivative of Indicates the longitudinal speed of the vehicle. This indicates the distance from the vehicle's pre-installed camera to its target point. Indicates lateral acceleration. Indicates yaw rate;

[0053] Let the above formula be , , Discretizing the above state equations yields:

[0054]

[0055]

[0056] In the formula, , , State variables Control variables , Let these represent system noise and measurement noise, respectively. They are uncorrelated and have a mean of 0. Represents the identity matrix. Indicates the sampling frequency;

[0057] The final value of the vehicle's sideslip angle is then estimated using the Kalman filter method. and variance .

[0058] S23: After identifying lane line information, the road surface area image is preserved. Preprocessing is performed using OpenCV, including image reading, resizing, and image enhancement. The region of interest is cropped and input into the lightweight classification network ShuffleNetV2. ShuffleNetV2 outputs road surface category labels and classification confidence scores through a softmax layer. The road surface is then classified based on ShuffleNetV2, and the corresponding road surface adhesion coefficient range and classification confidence scores are output.

[0059] Based on the pavement type and adhesion coefficient comparison table, the category labels are converted into pavement adhesion coefficient ranges, and the pavement adhesion coefficient intervals and corresponding classification confidence scores are output. It is assumed that for each pavement category... Below, road surface adhesion coefficient It follows a uniform distribution.

[0060] The overall expectation is:

[0061]

[0062] in, Indicates the road surface category. Indicates the number of road surface categories. Represent each category The range of adhesion coefficients, Indicates the given input The model outputs the road surface category. The confidence level is below.

[0063] Preferably, in step S3, since lane detection, semantic segmentation, and image classification algorithms involve complex image processing and computation processes, there is a delay between the output results and image acquisition, i.e., the image processing time of the visual estimation centroid side slip angle module. Image processing time of the visual estimation road adhesion coefficient module Furthermore, since the road area captured by the camera is located in front of the vehicle, the fusion of road surface adhesion coefficient estimation needs to take into account the time synchronization between visual and dynamic results;

[0064] Assuming the distance from the camera's preview point to the vehicle's front axle is l, the time it takes for the vehicle to reach that road segment is... Considering the complex curvature of the road, using integration along the path length, the total delay of the visual estimation module for the road surface adhesion coefficient is:

[0065]

[0066] Multi-source information fusion based on particle filtering: that is, using particle filtering to perform real-time fusion estimation of the centroid side slip angle and road surface adhesion coefficient from dynamic estimation and visual estimation, so as to obtain the posterior distribution and output the expectation and variance.

[0067] The state-space equations are:

[0068]

[0069]

[0070] in, , , , ;

[0071] Considering the poor observability of the dynamic model under low excitation conditions and the possibility of visual misjudgment, a state transition equation is obtained based on the vehicle's kinematic equations, and the sideslip angle of the center of mass is estimated using a direct integration method based on the lateral acceleration and yaw rate signals.

[0072]

[0073] Assuming that the road surface adhesion coefficient changes slowly over a short period of time, the fusion estimation result of the road surface adhesion coefficient at the previous moment is used as the state transition.

[0074] Therefore, the state transition equation is

[0075]

[0076]

[0077] In the formula, Sampling time; , This is process noise.

[0078] Preferably, the specific steps for multi-source information fusion based on particle filtering in step S3 are as follows:

[0079] a. Initialize particle, system noise, and observation noise;

[0080] b. State transition: Based on the vehicle kinematic equations and the fusion estimation results of the road adhesion coefficient at the previous moment, the prior estimates of the centroid sideslip angle and the road adhesion coefficient can be obtained.

[0081] c. Observation update particle weights: The prior observations are obtained from the prior estimation of the state. The error between the prior observations and the measured values ​​is calculated. The ratio of the posterior probability density function to the importance function is used as the particle weights, and the particle weights are normalized.

[0082] d. Resample to obtain a new set of particles and their weights, substitute them into the next loop, and repeat steps b-d.

[0083] Preferably, in step c within the particle filter state estimation framework, to improve robustness, an adaptive observation variance mechanism is introduced, which requires adjusting the observation variance based on the excitation level and visual classification.

[0084] Under low-excitation conditions, the dynamic model loses observability due to weak tire force, and the estimation error increases significantly. Or, when the visual confidence is low, the reliability of visual perception decreases. In this case, the observation variance is increased. Under special circumstances such as low-excitation conditions and harsh environments and weather, when the dynamic or visual estimation is unavailable, this observation is not updated. The baseline state estimate of the centroid sideslip angle and the road adhesion coefficient is provided by the fusion estimate of kinematics and the road adhesion coefficient of the previous moment.

[0085] Normalizing the lateral excitation level of the vehicle, we have:

[0086]

[0087] In the formula, Indicates the level of incentive. , , This indicates the reference lateral acceleration, yaw rate, and front wheel steering angle. , , Indicates the weight of each item;

[0088] The scaling function with respect to the excitation level is defined as:

[0089]

[0090] The scaling function for visual classification confidence is defined as:

[0091]

[0092] In the formula, , Represents the coefficient. This represents the maximum confidence level of the category output by the road surface classifier;

[0093] Therefore, the adjusted variance based on dynamics and visual estimation is:

[0094]

[0095]

[0096]

[0097]

[0098] when When this continues for a period of time, the particle filter only performs predictions, skipping the update step based on observations of dynamic estimates, and when the class discrimination is... When this continues for a period of time, the particle filter only performs predictions, skipping the update step based on visual estimates.

[0099] A multi-source fusion estimation system for centroid sideslip angle and road adhesion coefficient includes a dynamic estimation module, a visual estimation module, and an information fusion module. The dynamic estimation module and the visual estimation module receive information from sensors, such as vehicle yaw rate and lateral acceleration, as well as camera image information. The visual estimation module and the information fusion module are connected through the visually estimated centroid sideslip angle and road adhesion coefficient, estimation variance, and classification confidence. The dynamic estimation module and the information fusion module are also connected through the dynamically estimated centroid sideslip angle and road adhesion coefficient, as well as estimation variance. The information fusion module calculates the final estimation result by fusing information from the dynamic estimation module and the visual estimation module.

[0100] The dynamic estimation module, based on the vehicle lateral dynamics model and the brush tire model, combines multi-source signals from the inertial measurement unit, including lateral acceleration, yaw rate, front wheel steering angle, and vehicle speed, and uses an external Kalman filter framework to achieve joint estimation of the centroid sideslip angle and the road adhesion coefficient.

[0101] The visual estimation module, based on image information acquired by a camera mounted at the front of the vehicle, extracts road surface area information through the semantic segmentation neural network DeepLabV3. It then uses the lightweight classification network ShuffleNetV2 to identify the road surface type of the extracted area, classifying typical road surface conditions such as dry, wet, snow, and ice, and mapping them to the corresponding adhesion coefficient range. Based on lane line information and vehicle state parameters in the camera image, it constructs a geometric kinematic model of the lateral distance between the camera's aiming point and the lane line, as well as the distance from the pre-mounted camera on the vehicle to its aiming point. This model is then combined with the vehicle kinematic model to estimate the vehicle's center of gravity sideslip angle using Kalman filtering.

[0102] The information fusion module fuses the outputs of the dynamics estimation module and the visual estimation module. The fused centroid sideslip angle estimate and variance, as well as the road surface adhesion coefficient estimate and variance, can be used as input parameters for the vehicle's advanced control system. The particle filtering method is used to complete the state estimation fusion. In the prediction stage of the particle filtering, the kinematic model of the vehicle's centroid sideslip angle and the fused estimation result of the road surface adhesion coefficient at the previous time step are used to predict the state of the current particle. In the update stage, the particle weights are updated according to the expected state estimates and covariance of the two information sources. An adaptive adjustment mechanism for observation noise covariance is introduced to adaptively adjust the observation reliability under different road conditions and excitation conditions, thereby improving the stability and robustness of the fusion estimation.

[0103] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0104] 1. This invention achieves step-by-step estimation of centroid sideslip angle and road surface adhesion coefficient through a cascaded nonlinear observer architecture, solving the joint estimation problem caused by the dynamic coupling of the two. Based on the estimation results of the cascaded nonlinear observer, a method for estimating centroid sideslip angle and road surface adhesion coefficient based on an external Kalman filter framework is proposed, overcoming the problem that stability and optimality or suboptimality are difficult to balance in traditional methods.

[0105] 2. In this invention, lane line information is visually identified and combined with vehicle kinematics model to construct the Kalman filter output expectation and variance of vehicle centroid sideslip angle, thereby quantifying the uncertainty of visual estimation of centroid sideslip angle. At the same time, after visual road surface classification, Bayesian method is used to obtain the expectation and variance of visual estimation of road surface adhesion coefficient, which is beneficial to the subsequent fusion of multi-source information.

[0106] 3. By integrating dynamic and visual information, this invention achieves adaptive switching of the optimal estimation source under different working conditions, thereby ensuring rapid convergence of the centroid side slip angle and adhesion coefficient in multiple working conditions, improving estimation accuracy, and enhancing the robustness of the estimation results.

[0107] 4. This invention obtains the estimated values ​​and variances of the centroid sideslip angle and road surface adhesion coefficient based on vehicle dynamics and vision, respectively, and then constructs a fusion estimation framework that considers adaptive adjustment of observation noise. Through a dynamic covariance adjustment mechanism, it realizes the complementary advantages and collaborative estimation of multi-source information. Attached Figure Description

[0108] Figure 1 This is a schematic diagram of the overall architecture of an estimation method according to an embodiment of this application;

[0109] Figure 2 This is a schematic diagram of a monorail vehicle model;

[0110] Figure 3 This is an illustration of the relationship between road surface type and adhesion coefficient;

[0111] Figure 4 This is a schematic diagram of the adaptive mechanism for observation noise covariance. Detailed Implementation

[0112] The present invention will be further described below with reference to the accompanying drawings and embodiments, which illustrate the above and other technical features and advantages of the present invention. However, the following embodiments are merely preferred embodiments of the present invention and are not exhaustive.

[0113] Example:

[0114] like Figures 1-4 As shown, this invention provides a multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient. The estimation method specifically includes the following steps:

[0115] S1, based on the dynamic estimation of the vehicle's center of gravity sideslip angle and the road adhesion coefficient, outputs the estimated value and variance of the dynamic estimation of the vehicle's center of gravity sideslip angle, as well as the estimated value and variance of the road adhesion coefficient;

[0116] S2 estimates the centroid sideslip angle and road surface adhesion coefficient based on visual information, and outputs the visually estimated centroid sideslip angle estimate and variance, road surface adhesion coefficient estimate and variance, and classification confidence.

[0117] S3, considering the observation noise adjustment fusion estimation framework, the centroid sideslip angle estimate and variance after the fusion of the output dynamic estimation and visual estimation, and the road adhesion coefficient estimate and variance can be used as input parameters of the vehicle advanced control system.

[0118] In this embodiment, step S1 specifically includes the following:

[0119] S11, using a single-track vehicle model, analyzes the vehicle's steering and lateral motion characteristics, such as... Figure 2 As shown, the equations of motion for the vehicle's lateral and yaw movements are as follows:

[0120]

[0121]

[0122] in,

[0123]

[0124]

[0125]

[0126] In the formula, , These represent the front and rear axle lateral forces calculated based on the brush model, respectively. , These represent the distances from the center of mass to the front and rear axles, respectively. Indicates the overall vehicle weight. Indicates the whole vehicle is around Moment of inertia in the axial direction, Indicates yaw rate. The derivative of the yaw rate. , Indicates the longitudinal and lateral velocities at the center of mass. Indicates the road surface adhesion coefficient. Indicates the front wheel steering angle. , Indicates the front and rear axle slip angles;

[0127] S12 describes the tire's lateral stiffness and road adhesion coefficient using a brush model, capturing the dynamic response of tire force as adhesion conditions change, and accurately predicting tire force based on the brush model's tire lateral force. for:

[0128]

[0129] In the formula, Indicates the road surface adhesion coefficient. Indicates vertical force. Indicates tire lateral stiffness. Indicates the tire slip angle. , Segmentation point , Indicates vertical force. It is a normalized parameter determined by tire stiffness, vertical load, and road adhesion coefficient. This indicates the equivalent lateral slip of the tire.

[0130] S13, a cascaded nonlinear observer is designed to initially estimate the centroid sideslip angle and road adhesion coefficient. The estimated results are used as pseudo-measured values. The vehicle's centroid sideslip angle is the angle between the lateral velocity and the longitudinal velocity at the centroid. for:

[0131]

[0132] In the formula, This represents the estimated longitudinal velocity. Indicates the estimated lateral speed;

[0133] Let the longitudinal velocity be... It is known that the estimated value is obtained through adaptive adjustment. Value estimation observer estimates lateral vehicle speed ,when Proximity measurement The specific observers are as follows:

[0134]

[0135]

[0136] In the formula: This represents the lateral force when the vehicle's road surface adhesion coefficient is 1. This represents the adaptive parameters, which range from [0.05 to 1.2]. By setting reasonable parameters to make the observer converge, a good estimation effect can be guaranteed. Average observer gain.

[0137] Based on the estimated lateral velocity The designed road surface adhesion coefficient observer is as follows:

[0138]

[0139]

[0140]

[0141] In the formula: , This represents the lateral tire forces on the front and rear axles calculated based on the vehicle's monorail model. Indicates based on The front and rear axle slip angles calculated from the lateral velocity. This indicates that the estimated values ​​of the lateral forces on the front and rear axles are calculated based on the skid angle using a brush model. Indicates the gain of the observer. This represents the estimated value of the road surface adhesion coefficient;

[0142] S14 linearizes the vehicle lateral dynamics model and measurement model, and uses Kalman filtering to optimally estimate the centroid sideslip angle and road adhesion coefficient. Based on the lateral and yaw motion equations, when... When = 0, the system state equation after Euler discretization is:

[0143]

[0144]

[0145] in, , , , ;

[0146] The state equations obtained above are in Linearization yields:

[0147]

[0148]

[0149] in,

[0150]

[0151]

[0152]

[0153]

[0154]

[0155]

[0156] Finally, the standard formula of Kalman filtering is used for estimation, and the expected centroid sideslip angle is output by the dynamic estimation module. and variance Expected value of road surface adhesion coefficient and variance This result will serve as one of the inputs to the information fusion module, and together with the visual estimation result, complete the fusion estimation.

[0157] In this embodiment, step S2 specifically includes the following:

[0158] S21. The front-facing camera acquires lane line information. The camera is monocular. The image information is preprocessed using OpenCV, including image reading, resizing, image enhancement, and normalization. The preprocessed image is then input into the semantic segmentation network DeepLabV3. DeepLabV3 performs road segmentation on the image information. DeepLabV3 has been trained offline on a labeled dataset containing various road scene scenarios.

[0159] S22, DeepLabV3 outputs segmentation results, in which the road surface area and non-road surface area are effectively distinguished, and lane line information is identified. Using the camera preview point as a bridge, a geometric kinematic model between the lane lines and the vehicle is established, and a centroid sideslip angle estimator is built.

[0160] The state equation for the vehicle's sideslip angle in this vehicle center of gravity sideslip angle estimator is:

[0161]

[0162]

[0163] In the formula, This represents the centroid sideslip angle to be estimated. The derivative of Indicates the vehicle's yaw angle. This indicates the lateral distance between the camera's preview point and the lane line. The derivative of Indicates the longitudinal speed of the vehicle. This indicates the distance from the vehicle's pre-installed camera to its target point. Indicates lateral acceleration. Indicates yaw rate;

[0164] Let the above formula be , , Discretizing the above state equations yields:

[0165]

[0166]

[0167] In the formula, , , State variables Control variables , Let these represent system noise and measurement noise, respectively. They are uncorrelated and have a mean of 0. Represents the identity matrix. Indicates the sampling frequency;

[0168] The final value of the vehicle's sideslip angle is then estimated using the Kalman filter method. and variance .

[0169] S23. After identifying lane line information, the road surface area image is preserved and preprocessed using OpenCV, including image reading, resizing, and image enhancement. The Region of Interest (ROI) is then cropped and input into the lightweight classification network ShuffleNetV2. ShuffleNetV2 has been trained offline on datasets with multiple road surface conditions (dry, wet, snow, ice). ShuffleNetV2 outputs road surface category labels and classification confidence scores through a softmax layer. The road surface is then classified based on ShuffleNetV2, and the corresponding road surface adhesion coefficient range and classification confidence score are output.

[0170] in accordance with Figure 3 The table showing the relationship between road surface type and adhesion coefficient converts category labels into road surface adhesion coefficient ranges, outputting the road surface adhesion coefficient intervals and corresponding classification confidence scores. It is assumed that for each road surface category... Below, road surface adhesion coefficient It follows a uniform distribution.

[0171] The overall expectation is:

[0172]

[0173] in, Indicates the road surface category. Indicates the number of road surface categories. Represent each category The range of adhesion coefficients, Indicates the given input The model outputs the road surface category. The confidence level is below.

[0174] In this embodiment, in step S3, the image acquisition camera has a sampling frequency of 30Hz, and the vehicle status information has a sampling frequency of 100Hz. The two information cannot be directly calculated. It is necessary to perform time synchronization processing on the visual information and the vehicle status information to ensure data time alignment. A linear interpolation method is used to unify the visual information to the timestamp of the vehicle status information.

[0175] Because lane detection, semantic segmentation, and image classification algorithms involve complex image processing and computation, there is a delay between the output results and image acquisition, namely, the image processing time of the visual estimation centroid side slip angle module. Image processing time of the visual estimation road adhesion coefficient module Furthermore, since the road area captured by the camera is located in front of the vehicle, the fusion of road adhesion coefficient estimation needs to consider the time synchronization between visual and dynamic results. Assuming the distance from the camera preview point to the vehicle's front axle is... The time it takes for the vehicle to arrive at that section of road is Considering the complex curvature of the road, integration along the path length is used. Therefore, the total delay of the visual estimation module for the road surface adhesion coefficient is:

[0176]

[0177] Multi-source information fusion based on particle filtering: that is, using particle filtering to perform real-time fusion estimation of the centroid side slip angle and road surface adhesion coefficient from dynamic estimation and visual estimation, so as to obtain the posterior distribution and output the expectation and variance.

[0178] The state-space equations are:

[0179]

[0180]

[0181] in, , , , ;

[0182] Considering the poor observability of the dynamic model under low excitation conditions and the possibility of visual misjudgment, a state transition equation is obtained based on the vehicle's kinematic equations, and the sideslip angle of the center of mass is estimated using a direct integration method based on the lateral acceleration and yaw rate signals.

[0183]

[0184] Assuming that the road surface adhesion coefficient changes slowly over a short period of time, the fusion estimation result of the road surface adhesion coefficient at the previous moment is used as the state transition.

[0185] Therefore, the state transition equation is

[0186]

[0187]

[0188] In the formula, Sampling time; , This is process noise.

[0189] In this embodiment, the specific steps of multi-source information fusion based on particle filtering in step S3 are as follows:

[0190] a. Initialize particle, system noise, and observation noise;

[0191] b. State transition: Based on the vehicle kinematics equations and the fusion estimation results of the road adhesion coefficient at the previous moment, the prior estimates of the centroid sideslip angle and the road adhesion coefficient are obtained.

[0192] c. Observation update particle weights: The prior observations are obtained from the prior estimation of the state. The error between the prior observations and the measured values ​​is calculated. The ratio of the posterior probability density function to the importance function is used as the particle weights, and the particle weights are normalized.

[0193] d. Resample to obtain a new set of particles and their weights, substitute them into the next loop, and repeat steps b-d.

[0194] In this embodiment, step c is in the particle filter state estimation framework. The observation noise covariance is the core parameter for adjusting the fusion weight of multi-source information: the smaller the variance, the more reliable the observation value and the greater its contribution to the fusion result; otherwise, the contribution is reduced. In order to improve robustness, an observation variance adaptive mechanism is introduced, which needs to adjust the observation variance according to the excitation level and visual classification.

[0195] Under low-excitation conditions, the dynamic model loses observability due to weak tire force, and the estimation error increases significantly. Or, when the visual confidence is low, the reliability of visual perception decreases. In this case, the observation variance is increased. Under special circumstances such as low-excitation conditions and harsh environment and weather, when the dynamic estimation or visual estimation is unavailable, this observation is not updated. The baseline state estimate of the centroid sideslip angle and the road adhesion coefficient is provided by the fusion estimation results of kinematics and the road adhesion coefficient of the previous moment.

[0196] The adaptive mechanism for observation noise covariance is as follows: Figure 3 As shown, normalizing the lateral excitation level of the vehicle yields:

[0197]

[0198] In the formula, Indicates the level of incentive. , , This indicates the reference lateral acceleration, yaw rate, and front wheel steering angle. , , Indicates the weight of each item;

[0199] The scaling function with respect to the excitation level is defined as:

[0200]

[0201] The scaling function for visual classification confidence is defined as:

[0202]

[0203] In the formula, , Represents the coefficient. This represents the maximum confidence score of the class output by the road surface classifier; for example, when the classifier outputs 0.9 for dry road surface, 0.1 for wet road surface, 0.05 for snow road surface, and 0.05 for icy road surface. It is 0.9;

[0204] Therefore, the adjusted variance based on dynamics and visual estimation is:

[0205]

[0206]

[0207]

[0208]

[0209] when When this continues for a period of time, the particle filter only performs predictions, skipping the update step based on observations of dynamic estimates, and when the class discrimination is... When this continues for a period of time, the particle filter only performs predictions, skipping the update step based on visual estimates.

[0210] This adaptive mechanism ensures that the filter reduces the impact of unreliable observations when visual perception is impaired or dynamics are unobservable.

[0211] A multi-source fusion estimation system for centroid sideslip angle and road adhesion coefficient includes a dynamic estimation module, a visual estimation module, and an information fusion module. The dynamic estimation module and the visual estimation module receive vehicle yaw rate, lateral acceleration, and camera image information acquired by sensors. The visual estimation module and the information fusion module are connected through the visually estimated centroid sideslip angle and road adhesion coefficient, estimation variance, and classification confidence. The dynamic estimation module and the information fusion module are also connected through the dynamically estimated centroid sideslip angle and road adhesion coefficient, and estimation variance. The information fusion module calculates the final estimation result by fusing the information from the dynamic estimation module and the visual estimation module.

[0212] The dynamic estimation module, based on the vehicle lateral dynamics model and the brush tire model, combines multi-source signals from the inertial measurement unit (IMU) such as lateral acceleration, yaw rate, front wheel steering angle, and vehicle speed, and uses an external Kalman filter framework to achieve joint estimation of the centroid sideslip angle and the road adhesion coefficient.

[0213] The visual estimation module, based on image information acquired by a camera mounted at the front of the vehicle, extracts road surface area information through the semantic segmentation neural network DeepLabV3. It then uses the lightweight classification network ShuffleNetV2 to identify the road surface type of the extracted area, classifying typical road surface conditions such as dry, wet, snow, and ice, and mapping them to the corresponding adhesion coefficient range. Based on lane line information and vehicle state parameters in the camera image, it constructs a geometric kinematic model of the lateral distance between the camera's aiming point and the lane line, as well as the distance from the pre-mounted camera on the vehicle to its aiming point. This model is then combined with the vehicle kinematic model to estimate the vehicle's center of gravity sideslip angle using Kalman filtering.

[0214] The information fusion module fuses the outputs of the dynamics estimation module and the visual estimation module. The fused centroid sideslip angle estimate and variance, as well as the road surface adhesion coefficient estimate and variance, can be used as input parameters for the vehicle's advanced control system. The particle filtering method is used to complete the state estimation fusion. In the prediction stage of the particle filtering, the kinematic model of the vehicle's centroid sideslip angle and the fused estimation result of the road surface adhesion coefficient at the previous time step are used to predict the state of the current particle. In the update stage, the particle weights are updated according to the expected state estimates and covariance of the two information sources. An adaptive adjustment mechanism for observation noise covariance is introduced to adaptively adjust the observation reliability under different road conditions and excitation conditions, thereby improving the stability and robustness of the fusion estimation.

[0215] The above description is merely a preferred embodiment of the present invention and is illustrative rather than restrictive. Those skilled in the art will understand that many changes, modifications, and even equivalents can be made within the spirit and scope defined by the claims of the present invention, all of which will fall within the protection scope of the present invention.

Claims

1. A multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient, characterized in that, The estimation method specifically includes the following steps: S1, based on the dynamic estimation of the vehicle's center of gravity sideslip angle and the road adhesion coefficient, outputs the estimated value and variance of the dynamic estimation of the vehicle's center of gravity sideslip angle, as well as the estimated value and variance of the road adhesion coefficient; S2 estimates the centroid sideslip angle and road surface adhesion coefficient based on visual information, and outputs the visually estimated centroid sideslip angle estimate and variance, road surface adhesion coefficient estimate and variance, and classification confidence. S3, considering the observation noise adjustment fusion estimation framework, the estimated value and variance of the centroid sideslip angle after the fusion of the output dynamic estimation and visual estimation, and the estimated value and variance of the road adhesion coefficient can be used as input parameters of the vehicle advanced control system.

2. The multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in claim 1, characterized in that, The specific details of step S1 Includes the following: S11, analyze the steering and lateral motion characteristics of the vehicle using a single-track vehicle model; S12 describes the tire's lateral stiffness and road adhesion coefficient using a brush model, capturing the dynamic response of tire force as adhesion conditions change, and accurately predicting tire force based on the brush model's tire lateral force. for: In the formula, Indicates the road surface adhesion coefficient. Indicates vertical force. Indicates tire lateral stiffness. Indicates the tire slip angle. , Segmentation point , This represents the normalized parameter determined by tire stiffness, vertical load, and road adhesion coefficient. This indicates the equivalent lateral slip of the tire. S13, design a cascaded nonlinear observer to initially estimate the centroid sideslip angle and road surface adhesion coefficient, and use the estimated results as pseudo-measured values; S14 linearizes the vehicle lateral dynamics model and measurement model, and uses Kalman filtering to make optimal estimates of the centroid sideslip angle and road adhesion coefficient.

3. The multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in claim 2, characterized in that, In step S11, when analyzing the vehicle's steering and lateral motion characteristics, the vehicle's lateral and yaw motion equations are as follows: in, In the formula, , These represent the front and rear axle lateral forces calculated based on the brush model, respectively. , These represent the distances from the center of mass to the front and rear axles, respectively. Indicates the overall vehicle weight. Indicates the whole vehicle is around Moment of inertia in the axial direction, Indicates yaw rate. The derivative of the yaw rate. , Indicates the longitudinal and lateral velocities at the center of mass. Indicates the road surface adhesion coefficient. Indicates the front wheel steering angle. , Indicates the front and rear axle slip angles.

4. The multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in claim 2, characterized in that, In step S13, the vehicle's sideslip angle is the angle between the lateral velocity and the longitudinal velocity at the center of gravity. for: In the formula, This represents the estimated longitudinal velocity. Indicates the estimated lateral speed; Let the longitudinal velocity be... It is known that the estimated value is obtained through adaptive adjustment. Value estimation observer estimates lateral vehicle speed ,when Proximity measurement The specific observers are as follows: In the formula: This represents the lateral force when the vehicle's road surface adhesion coefficient is 1. This represents the adaptive parameter, which ranges from [0.05 to 1.2]. Based on the estimated lateral velocity The designed road surface adhesion coefficient observer is as follows: In the formula: , This represents the lateral tire forces on the front and rear axles calculated based on the vehicle's monorail model. Indicates based on The front and rear axle slip angles calculated from the lateral velocity. This indicates that the estimated values ​​of the lateral forces on the front and rear axles are calculated based on the skid angle using a brush model. Indicates the gain of the observer. This represents the estimated value of the road surface adhesion coefficient.

5. The multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in claim 2, characterized in that, Step S14 is based on the equations of lateral and yaw motion, when When = 0, the system state equation after Euler discretization is: in, , , , ; The state equations obtained above are in Linearization yields: The expected centroid sideslip angle is estimated using the standard formula of Kalman filtering and then output by the dynamic estimation module. and variance Expected value of road surface adhesion coefficient and variance This result will serve as one of the inputs to the information fusion module, and together with the visual estimation result, complete the fusion estimation.

6. The multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in claim 1, characterized in that, Step S2 specifically includes the following: S21. The front-facing camera acquires lane line information. The camera is monocular. The image information is preprocessed using OpenCV, including image reading, resizing, image enhancement, and normalization. The preprocessed image is then input into the semantic segmentation network DeepLabV3. DeepLabV3 performs road segmentation on the image information. DeepLabV3 has been trained offline on a labeled dataset containing various road scene scenarios. S22, DeepLabV3 outputs segmentation results, in which the road surface area and non-road surface area are effectively distinguished, and lane line information is identified. Using the camera preview point as a bridge, a geometric kinematic model between the lane lines and the vehicle is established, and a centroid sideslip angle estimator is built. The state equation for the vehicle's sideslip angle in this vehicle center of gravity sideslip angle estimator is: In the formula, This represents the centroid sideslip angle to be estimated. The derivative, Indicates the vehicle's yaw angle. This indicates the lateral distance between the camera's preview point and the lane line. The derivative, Indicates the longitudinal speed of the vehicle. This indicates the distance from the vehicle's pre-installed camera to its target point. Indicates lateral acceleration. Indicates yaw rate; Let the above formula be , , Discretizing the above state equations yields: In the formula, , , State variables Control variables , Let these represent system noise and measurement noise, respectively. They are uncorrelated and have a mean of 0. Represents the identity matrix. Indicates the sampling frequency; The final value of the vehicle's sideslip angle is then estimated using the Kalman filter method. and variance . S23: After identifying lane line information, the road surface area image is preserved. Preprocessing is performed using OpenCV, including image reading, resizing, and image enhancement. The region of interest is cropped and input into the lightweight classification network ShuffleNetV2. ShuffleNetV2 outputs road surface category labels and classification confidence scores through a softmax layer. The road surface is then classified based on ShuffleNetV2, and the corresponding road surface adhesion coefficient range and classification confidence scores are output. Based on the pavement type and adhesion coefficient comparison table, the category labels are converted into pavement adhesion coefficient ranges, and the pavement adhesion coefficient intervals and corresponding classification confidence scores are output. It is assumed that for each pavement category... Below, road surface adhesion coefficient It follows a uniform distribution. The overall expectation is: in, Indicates the road surface category. Indicates the number of road surface categories. Represent each category The range of adhesion coefficients, Indicates the given input The model outputs the road surface category. The confidence level is below.

7. The multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in claim 1, characterized in that, In step S3, because lane detection, semantic segmentation, and image classification algorithms involve complex image processing and computation, there is a delay between the output results and image acquisition, i.e., the image processing time of the visual estimation centroid side slip angle module. Image processing time of the visual estimation road adhesion coefficient module Furthermore, since the road area captured by the camera is located in front of the vehicle, the fusion of road surface adhesion coefficient estimation needs to take into account the time synchronization between visual and dynamic results; Assume the distance from the camera's preview point to the vehicle's front axle is... The time it takes for the vehicle to arrive at that section of road is Considering the complex curvature of the road, using integration along the path length, the total delay of the visual estimation module for the road surface adhesion coefficient is: Multi-source information fusion based on particle filtering: that is, using particle filtering to perform real-time fusion estimation of the centroid side slip angle and road surface adhesion coefficient from dynamic estimation and visual estimation, so as to obtain the posterior distribution and output the expectation and variance. The state-space equations are: in, , , , ; Considering the poor observability of the dynamic model under low excitation conditions and the possibility of visual misjudgment, a state transition equation is obtained based on the vehicle's kinematic equations, and the sideslip angle of the center of mass is estimated using a direct integration method based on the lateral acceleration and yaw rate signals. Assuming that the road surface adhesion coefficient changes slowly over a short period of time, the fusion estimation result of the road surface adhesion coefficient at the previous moment is used as the state transition. Therefore, the state transition equation is: In the formula, Sampling time; , This is process noise.

8. The multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in claim 1, characterized in that, The specific steps for multi-source information fusion based on particle filtering in step S3 are as follows: a. Initialize particle, system noise, and observation noise; b. State transition: Based on the vehicle kinematics equations and the fusion estimation results of the road adhesion coefficient at the previous moment, the prior estimates of the centroid sideslip angle and the road adhesion coefficient are obtained. c. Observation update particle weights: The prior observations are obtained from the prior estimation of the state. The error between the prior observations and the measured values ​​is calculated. The ratio of the posterior probability density function to the importance function is used as the particle weights, and the particle weights are normalized. d. Resample to obtain a new set of particles and their weights, substitute them into the next loop, and repeat steps b-d.

9. The multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in claim 8, characterized in that, In step c, within the particle filter state estimation framework, an adaptive observation variance mechanism is introduced to improve robustness. The observation variance needs to be adjusted according to the excitation level and visual classification. Under low-excitation conditions, the dynamic model loses observability due to weak tire force, resulting in increased estimation error. Alternatively, when visual confidence is low, the reliability of visual perception decreases. In such cases, the observation variance is increased. Under special circumstances such as low-excitation conditions and adverse environments and weather, when the dynamic or visual estimation is unavailable, this observation is not updated. Instead, the baseline estimate of the centroid sideslip angle and the road adhesion coefficient is provided by the fusion estimation results of kinematics and the previous moment's road adhesion coefficient. Normalizing the lateral excitation level of the vehicle, we have: In the formula, Indicates the level of incentive. , , This indicates the reference lateral acceleration, yaw rate, and front wheel steering angle. , , Indicates the weight of each item; The scaling function with respect to the excitation level is defined as: The scaling function for visual classification confidence is defined as: In the formula, , Represents the coefficient. This represents the maximum confidence level of the category output by the road surface classifier; Therefore, the adjusted variance based on dynamics and visual estimation is: when When this continues for a period of time, the particle filter only performs predictions, skipping the update step based on observations of dynamic estimates, and when the class discrimination is... When this continues for a period of time, the particle filter only performs predictions, skipping the update step based on visual estimates.

10. A multi-source fusion estimation system for centroid sideslip angle and road surface adhesion coefficient, used in the multi-source fusion estimation method for centroid sideslip angle and road surface adhesion coefficient as described in any one of claims 1-9, characterized in that, It includes a dynamics estimation module, a visual estimation module, and an information fusion module. The dynamics estimation module and the visual estimation module receive vehicle yaw rate, lateral acceleration, and camera image information acquired by sensors. The visual estimation module and the information fusion module are connected through the visually estimated centroid sideslip angle, road adhesion coefficient, estimation variance, and classification confidence. The dynamics estimation module and the information fusion module are connected through the dynamically estimated centroid sideslip angle, road adhesion coefficient, and estimation variance. The information fusion module calculates the final estimation result by fusing the information from the dynamics estimation module and the visual estimation module. The dynamic estimation module, based on the vehicle lateral dynamics model and the brush tire model, combines multi-source signals from the inertial measurement unit, including lateral acceleration, yaw rate, front wheel steering angle, and vehicle speed, and uses an external Kalman filter framework to achieve joint estimation of the centroid sideslip angle and the road adhesion coefficient. The visual estimation module, based on image information acquired by a camera mounted at the front of the vehicle, extracts road surface area information through the semantic segmentation neural network DeepLabV3. It then uses the lightweight classification network ShuffleNetV2 to identify the road surface type of the extracted area, classifying typical road surface conditions such as dry, wet, snow, and ice, and mapping them to the corresponding adhesion coefficient range. Based on lane line information and vehicle state parameters in the camera image, it constructs a geometric kinematic model of the lateral distance between the camera's aiming point and the lane line, as well as the distance from the pre-mounted camera on the vehicle to its aiming point. This model is then combined with the vehicle kinematic model to estimate the vehicle's center of gravity sideslip angle using Kalman filtering. The information fusion module fuses the outputs of the dynamics estimation module and the visual estimation module. The fused centroid sideslip angle estimate and variance, as well as the road surface adhesion coefficient estimate and variance, can be used as input parameters for the vehicle's advanced control system. The particle filtering method is used to complete the state estimation fusion. In the prediction stage of the particle filtering, the kinematic model of the vehicle's centroid sideslip angle and the fused estimation result of the road surface adhesion coefficient at the previous time step are used to predict the state of the current particle. In the update stage, the particle weights are updated according to the expected state estimates and covariance of the two information sources. An adaptive adjustment mechanism for observation noise covariance is introduced to adaptively adjust the observation reliability under different road conditions and excitation conditions, thereby improving the stability and robustness of the fusion estimation.

Images