An adaptive control method for magnetorheological suspension based on ihosmo-td3

Abstract

The application discloses an adaptive control method for a magneto-rheological semi-active suspension based on an IHOSMO-TD3, which comprises the following steps: firstly, a 1 / 4 vehicle suspension model containing a magneto-rheological damper is constructed; secondly, an improved high-order sliding mode observer is designed, and all states of the system can be quickly and accurately estimated only by measuring the body acceleration and the suspension dynamic deflection; finally, the estimated states are input into a controller designed based on a double-delay deep deterministic policy gradient algorithm, and the optimal damping force current value is output by the controller, so that the model-free adaptive optimal control of the suspension system is realized; the application does not need an accurate mathematical model, is strong in robustness to system uncertainty, needs only partial measurable states, solves the practical application bottleneck of DRL, and significantly improves the ride comfort and the steering stability of the vehicle.

Description

Technical Field

[0001] This invention belongs to the field of vehicle intelligent control technology, specifically relating to a magnetorheological suspension adaptive control method based on IHOSMO-TD3. Background Technology

[0002] With the rapid development of automotive intelligence and electrification technologies, vehicle suspension systems are evolving from traditional passive adjustment to intelligent control with adaptive perception and decision-making capabilities. As a key actuator connecting the vehicle body and wheels, the suspension system not only directly affects ride comfort but also relates to vehicle handling stability and driving safety. Magnetorheological suspension, due to its advantages of fast response, continuously adjustable damping, relatively simple structure, and low energy consumption, has become one of the mainstream technologies for intelligent suspension.

[0003] Existing research on vehicle suspension control can be broadly categorized into three types: classical control, observer-enhanced control, and learning-based control.

[0004] 1) Classical control methods mainly include robust control, fuzzy control, backstepping control and sliding mode control. These methods often rely on accurate system models and are difficult to balance ride comfort and handling stability at the same time. They also have limited ability to handle strong nonlinear and time-varying conditions.

[0005] 2) The observer-enhanced control method provides a path to solve the problems of unmeasurable state and unknown disturbances. It reconstructs the unmeasurable state through the state observer. However, traditional observers are prone to chattering or insufficient accuracy under strong disturbances and measurement noise.

[0006] 3) In recent years, deep reinforcement learning has provided a new research path for suspension control. This type of method can learn control strategies online by interacting with the environment and is applicable to continuous action space and complex nonlinear systems. However, this type of method often relies on the measurability of the whole state and is difficult to apply directly in engineering scenarios where the critical state is not measurable.

[0007] In summary, existing research has established three technical routes: classical robust control, observer compensation, and learning-based decision-making. However, a unified framework that can simultaneously consider state estimation accuracy, continuous control capability, physical constraint handling, and engineering feasibility is still lacking. Particularly for semi-active systems like magnetorheological suspensions, which are characterized by strong nonlinearity, strong disturbances, and strong constraints, traditional control methods often struggle to achieve a stable balance between comfort, handling stability, and actuator constraints. Pure reinforcement learning control methods, on the other hand, are limited by state measurement constraints and difficult to implement directly in engineering. Therefore, achieving high-performance adaptive control of magnetorheological suspensions under complex operating conditions without relying on high-precision models and full-state measurements has become a key research focus and challenge. Summary of the Invention

[0008] To overcome the shortcomings of the prior art, the purpose of this invention is to provide a magnetorheological suspension adaptive control method based on IHOSMO-TD3, which aims to solve the key problems of poor robustness caused by the reliance on accurate mathematical models in existing control strategies, and the difficulty in practical application due to the reliance on difficult-to-measure full-state information.

[0009] To achieve the above objectives, the technical solution adopted by this invention is: an adaptive control method for magnetorheological suspension based on IHOSMO-TD3, comprising the following steps:

[0010] Step 1: Establish a dynamic model of the quarter-vehicle magnetorheological suspension and parametrically describe the nonlinear output characteristics of the magnetorheological damper.

[0011] Step 2: Design an improved high-order sliding mode observer (IHOSMO) to reconstruct the unmeasurable critical states and equivalent disturbances of the magnetorheological suspension based on the output signals collected by the sensors.

[0012] Step 3: Design a dual-delay deep deterministic policy gradient (TD3) agent. Using the estimated state output by the improved high-order sliding mode observer as input, construct the state space, action space, and reward function adapted to the suspension control, and learn the optimal control strategy of the magnetorheological damper.

[0013] Step 4: The improved high-order sliding mode observer and the dual-delay deep deterministic policy gradient agent are jointly integrated to design the IHOSMO-TD3 magnetorheological suspension control method, realize the adaptive cooperative control of the magnetorheological suspension, and perform stability analysis on the joint closed-loop system.

[0014] Step 5: Build a quarter magnetorheological suspension experimental platform, compare the performance indicators of different control strategies, and verify the engineering effectiveness of the adaptive cooperative control method.

[0015] In step 1, the dynamic equation of the quarter vehicle magnetorheological suspension model is represented by equation (1);

[0016] (1)

[0017] In the formula, m s For the sprung mass, m u For unsprung mass, k s For the spring stiffness, k u For tire stiffness, F d The damping force output by the magnetorheological damper. and These represent the displacements of the vehicle body and the unsprung mass, respectively. Represents vehicle acceleration. Represents suspension dynamic deflection. This represents the tire's travel distance. Input for road surface displacement;

[0018] The output force of the magnetorheological damper adopts the modified Bingham model, and the output force of the magnetorheological damper is expressed by equation (2):

[0019]

[0020] in, To control the current The relevant yield strength, For friction compensation, and The vehicle speed and unsprung mass speed are respectively, sgn is the sign function, and a polynomial approximation (3) is used to describe the effect of current on yield force.

[0021] (3)

[0022] Parameters in the formula These are the polynomial fitting coefficients, which can be obtained through experimentation;

[0023] The system state variables are defined by equation (4):

[0024] (4)

[0025] The state-space form of the system is shown in equation (5):

[0026] (5)

[0027] In the formula, A is the system state matrix, and B is the system input matrix. For the system input vector, Here is the road surface displacement disturbance matrix. Input for road surface displacement disturbance;

[0028] The system output is defined by equation (6):

[0029] (6)

[0030] In step 2, the design of the improved high-order sliding mode observer (IHOSMO) includes: defining the observer state estimate and output error, the error of which is defined to satisfy equation (7):

[0031] (7)

[0032] in To estimate the error, This is the estimated value of the system state vector. For system output estimation, This is the output matrix;

[0033] The observer dynamics that integrate the saturated boundary layer and superhelical mechanism are constructed, and their expressions satisfy equations (8) and (9):

[0034] (8)

[0035] Where L is the linear observation gain. To assist in the integration state, For higher-order sliding mode correction term, e is the system output estimation error vector, i.e. ey as defined in equation (7), and f(·) is the system nominal dynamic function;

[0036] (9)

[0037] In the formula These are the estimated values ​​for the 1st to 4th system states, corresponding to the suspension dynamic deflection in turn. Vertical speed of the vehicle body Tire travel Unsprung mass vertical velocity The estimation results; These are the estimated values ​​for the first to fourth system states, respectively. This is an estimate of the output damping force of the magnetorheological damper. This is an estimate of the vertical input velocity of the road surface. The four elements of the linear observation gain matrix L correspond to the linear feedback correction gains for the four states, and are determined globally through the GQPSO optimization algorithm in step 2. For the superhelical sliding mode switching function that integrates saturated boundary layers, This is the gain coefficient of the higher-order sliding mode correction term, used to adjust the strength of the sliding mode correction. Auxiliary Integral Status We obtain the first derivative; The update gain coefficients for the auxiliary integral state are used to adjust the convergence speed of the perturbation estimation; It is a saturation function used to replace the sign function of traditional sliding mode, suppressing high-frequency chattering from the root. Its complete expression is shown in equation (12). The saturated boundary layer thickness is used to balance the convergence speed and chattering suppression effect of the sliding mode observer. The thicker the boundary layer, the smaller the chattering, but the steady-state error increases slightly.

[0038] in The superhelical term is adopted, and the expression of the superhelical term satisfies equations (10) and (11):

[0039] (10)

[0040] (11)

[0041] in This is the proportional gain coefficient for the superspiral term, used to adjust the approach speed of the sliding surface; The integral gain coefficient of the superspiral term is used to adjust the intensity of the integral correction; The integral state variable for the superspinning term is used to realize higher-order sliding model properties; For integration state variables The first derivative; The error in estimating the vertical acceleration of the vehicle body; Estimation error of vehicle body vertical acceleration The absolute value;

[0042] To reduce chattering, Replace it with a saturation function, the expression of which satisfies equation (12):

[0043] (12)

[0044] The estimation error dynamics satisfy equation (13):

[0045] (13)

[0046] in, ,in This represents the actual state of the suspension system. The state estimate output by the observer is defined in the same way as in equation (9); The estimation error of the vertical input velocity of the road surface is defined as the actual road surface velocity. Estimating road surface speed with the observer The difference, i.e. ; The estimation error of the output damping force of the magnetorheological damper is defined as the true damping force. The difference between the estimated damping force and the value of the damping force, if satisfying If the modeling error is bounded, then all unmodeled terms can be uniformly represented as disturbance terms. .

[0047] Step 2 further includes: optimizing the gain parameters of the improved high-order sliding mode observer (IHOSMO) using the GQPSO optimization algorithm, wherein the optimization objective function satisfies equation (15):

[0048] (15)

[0049] Where RMSE(·) is the root mean square error function, used to measure the overall steady-state accuracy of the state estimation; These are the weighting coefficients for each performance indicator, used to balance the importance of the vehicle body vertical velocity estimation accuracy, the unsprung mass vertical velocity estimation accuracy, and the peak acceleration error.

[0050] 5. The magnetorheological suspension adaptive control method based on IHOSMO-TD3 according to claim 1, characterized in that, in step 3, the design of the TD3 agent includes: state space design: constructing the input state of TD3 with the estimated state output by the improved high-order sliding mode observer (IHOSMO), wherein the input state vector satisfies equation (14):

[0051] (16)

[0052] in This represents the equivalent disturbance or auxiliary state output by the observer.

[0053] Action space design: The continuous output action of TD3 is mapped to the target damping force of the magnetorheological damper, and the mapping relationship satisfies equation (15):

[0054] (17)

[0055] in The raw, continuous action values ​​output by the TD3 agent network at time t; This represents the minimum damping force that the magnetorheological damper is allowed to output. This represents the maximum damping force that the magnetorheological damper can output.

[0056] Reward function design: A quadratic negative reward function is adopted, the expression of which satisfies equation (19):

[0057] (19)

[0058] in, As the weighting coefficient, the quadratic negative reward function encourages the system to reduce vehicle acceleration, suppress suspension deflection and tire dynamic deformation, while avoiding excessive control fluctuations; The target damping force of the magnetorheological damper at time t is used to limit the amplitude of the control input and prevent the damping force from exceeding the physical limit of the actuator. The rate of change of magnetorheological damping force at time t is used to suppress drastic fluctuations in control input, extend the service life of the magnetorheological damper, and reduce actuator energy consumption.

[0059] In step 3, the neural network structure of the TD3 agent includes two Critic value networks and one Actor policy network; the input of the Critic network is a concatenated vector of state and action; the output of the Actor network is mapped to the actual damping force after being activated by the tanh activation function; the TD3 agent adopts a mechanism of overestimation suppression of dual Critic networks, target policy smoothing, and soft update.

[0060] In step 4, the stability analysis of the IHOSMO-TD3 controller specifically involves: decomposing the joint closed-loop system into an observation error subsystem and a control subsystem, and performing stability analysis based on the input-to-state stability (ISS) theory; under the condition that the observation error, strategy approximation error, and external disturbances are bounded, proving that the joint closed-loop system satisfies input-to-state stability, and that the system state is uniformly bounded eventually; under ideal conditions, the system can achieve asymptotic stability.

[0061] In step 5, the experimental platform includes an excitation simulation module, a suspension body module, and a measurement and control system module; the experimental verification includes testing under bumpy road surface and random road surface conditions, and comparing the vehicle acceleration response of passive suspension and TD3 control.

[0062] Compared with existing technologies, the magnetorheological semi-active suspension adaptive control method based on IHOSMO-TD3 provided by this invention has the following significant advantages:

[0063] 1) It fundamentally eliminates the reliance on precise mathematical models, significantly enhancing robustness.

[0064] This invention employs a controller based on model-free reinforcement learning (TD3 algorithm) as the core decision-making unit. The TD3 agent autonomously learns the optimal control strategy through continuous interaction with the environment (vehicle suspension system), rather than relying on a pre-set, fixed mathematical model. Therefore, this method has a natural adaptability to the model uncertainties of the suspension system (such as changes in sprung mass and time-varying parameters) and the strong nonlinearity and hysteresis characteristics of the magnetorheological damper, fundamentally solving the technical problem of insufficient robustness of traditional model-based control methods in real-world complex environments.

[0065] 2) It achieves high-performance control under partially measurable conditions, solves the bottleneck of DRL applications, and has strong practicality.

[0066] This invention creatively combines an improved high-order sliding mode observer (IHOSMO) with the TD3 algorithm. IHOSMO can utilize readily measurable, finite signals (such as vehicle vertical acceleration and suspension dynamic deflection) to estimate all system state information required for the TD3 algorithm's decision-making in a high-precision, finite-time manner (especially the unsprung mass velocity, which is difficult to measure directly). This co-design breaks the dependence of existing DRL control methods on expensive or unreliable sets of physical sensors, enabling the economical and reliable deployment of high-performance intelligent control strategies in practical engineering, and solving a key bottleneck in the transition of model-free control from theory to practical application.

[0067] 3) Excellent vibration suppression effect, comprehensively improving vehicle dynamic performance.

[0068] Thanks to the powerful nonlinear approximation capability of the TD3 algorithm and the accurate state estimation provided by IHOSMO, this invention can calculate the optimal damping force in real time for complex and variable road excitations and driving conditions. Simulations and experiments show that, compared with passive suspension, traditional roof control, and the baseline TD3 control that relies on full-state information, this invention can effectively reduce the vehicle's vertical acceleration by 20%-40%, significantly improving ride comfort. At the same time, it can stabilize tire dynamic displacement and suspension dynamic deflection within a safe range, ensuring vehicle handling stability and driving safety, and achieving an optimal balance among several conflicting performance indicators.

[0069] 4) The observer has a fast convergence speed, high estimation accuracy, and strong anti-interference ability.

[0070] The IHOSMO adopted effectively suppresses the high-frequency chattering problem inherent in traditional high-order sliding mode observers through an improved reaching law. While ensuring fast convergence and finite-time stability, it provides a smooth and accurate state estimation curve, laying a reliable data foundation for the accurate decision-making of the upper-level controller and avoiding the degradation of control performance due to observation errors.

[0071] This invention addresses the engineering challenges of unmeasurable critical states in magnetorheological suspensions and the difficulty in ensuring the stability of observation-learning joint systems. It proposes an IHOSMO-TD3 joint control method based on the IHOSMO and TD3 algorithms. This method integrates saturated boundary layer and superhelical mechanisms, effectively suppressing traditional sliding mode chattering, achieving finite-time convergence, and reconstructing unmeasurable suspension states and equivalent disturbances with high precision, generating continuous adaptive damping control signals. Compared to traditional control methods and pure TD3 control, this scheme eliminates the need for full-state measurements, achieving adaptive control under complex operating conditions using only conventional sensors, and exhibiting both stronger robustness and theoretical reliability. Simulation and experimental verification demonstrate that this control method effectively reduces vehicle vibration, meets safety constraints, and balances ride comfort and driving safety, possessing significant engineering application value. Attached Figure Description

[0072] Figure 1 This is a flowchart of the method of the present invention.

[0073] Figure 2 This is a schematic diagram of the quarter magnetorheological suspension dynamics model of the present invention.

[0074] Figure 3 This is a schematic diagram of the IHOSMO structure of the present invention.

[0075] Figure 4 This is a comparison chart of the convergence of three optimization algorithms: GQPSO, QPSO, and PSO.

[0076] Figure 5 This is a schematic diagram of the neural network structure of the TD3 algorithm of the present invention.

[0077] Figure 6 This is a diagram of the IHOSMO-TD3 joint control framework of the present invention.

[0078] Figure 7 This is a simulation model diagram of the magnetorheological semi-active suspension control system of the present invention.

[0079] Figure 8 The reward convergence curves for three control strategies: DQN, TD3, and IHOSMO-TD3 are shown.

[0080] Figure 9 Figure 1 shows the displacement excitation curves of a convex pavement, where Figure (a) shows the displacement excitation curve of a single convex pavement and Figure (b) shows the displacement excitation curve of a double convex pavement.

[0081] Figure 10 Figure 1 shows a comparison of the performance indicators of various control strategies under low-speed conditions on bumpy roads. Figure 2 shows a comparison of the vehicle body acceleration of various control strategies, Figure 3 shows a comparison of the suspension dynamic deflection of various control strategies, Figure 4 shows a comparison of the tire dynamic travel of various control strategies, and Figure 5 shows a comparison of the vehicle body acceleration power spectral density of various control strategies.

[0082] Figure 11 Radar plots showing the RMS and MAX values ​​of various control strategies under low-speed conditions on uneven road surfaces.

[0083] Figure 12 Figure 1 shows a comparison of the performance indicators of various control strategies under high-speed conditions on bumpy roads. Figure 2 shows a comparison of the vehicle body acceleration of various control strategies, Figure 3 shows a comparison of the suspension dynamic deflection of various control strategies, Figure 4 shows a comparison of the tire dynamic travel of various control strategies, and Figure 5 shows a comparison of the vehicle body acceleration power spectral density of various control strategies.

[0084] Figure 13Radar graphs showing the RMS and MAX values ​​of various control strategies under high-speed conditions on uneven pavement.

[0085] Figure 14 This is a displacement excitation curve diagram for an ISO-C grade random road surface.

[0086] Figure 15 Figure 1 shows a comparison of the performance indicators of various control strategies under low-speed conditions on random road surfaces. Figure 2 shows a comparison of the vehicle body acceleration of various control strategies, Figure 3 shows a comparison of the suspension dynamic deflection of various control strategies, Figure 4 shows a comparison of the tire dynamic travel of various control strategies, and Figure 5 shows a comparison of the vehicle body acceleration power spectral density of various control strategies.

[0087] Figure 16 Radar plots of RMS and MAX values ​​for various control strategies under low-speed conditions on random road surfaces.

[0088] Figure 17 Figure 1 shows a comparison of the performance indicators of various control strategies under random road surface high-speed conditions. Figure 2 shows a comparison of the vehicle body acceleration of various control strategies, Figure 3 shows a comparison of the suspension dynamic deflection of various control strategies, Figure 4 shows a comparison of the tire dynamic travel of various control strategies, and Figure 5 shows a comparison of the vehicle body acceleration power spectral density of various control strategies.

[0089] Figure 18 Radar plots of RMS and MAX values ​​for various control strategies under random road surface high-speed conditions.

[0090] Figure 19 This is a schematic diagram of the magnetorheological suspension experimental platform of the present invention.

[0091] Figure 20 This is a schematic diagram illustrating the experimental principle of the present invention.

[0092] Figure 21 This is a schematic diagram of the mechanical structure of the quarter magnetorheological suspension experimental platform of the present invention.

[0093] Figure 22 Figure 1 shows a comparison of the vehicle acceleration response under the test condition of bumpy road surface. Figure 2 shows a comparison of the vehicle acceleration under different control strategies, and Figure 3 shows a comparison of the vehicle acceleration RMS under different control strategies.

[0094] Figure 23 Figure 1 shows a comparison of vehicle acceleration response under random road test conditions. Figure 2 shows a comparison of vehicle acceleration under different control strategies, and Figure 3 shows a comparison of vehicle acceleration RMS under different control strategies. Detailed Implementation

[0095] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0096] See Figure 1 This paper presents an adaptive control method for magnetorheological suspension based on IHOSMO-TD3. The core of this method is to construct a joint framework of IHOSMO and TD3 intelligent control to solve the control challenges of unpredictable magnetorheological suspension states and strong nonlinearity. (Reference...) Figure 1 As shown, it includes the following steps:

[0097] Step 1: Establish a dynamic model of the quarter-vehicle magnetorheological suspension and parametrically describe the nonlinear output characteristics of the magnetorheological damper.

[0098] Step 2: Design IHOSMO to reconstruct the unmeasurable critical states and equivalent disturbances of the magnetorheological suspension based on the output signals collected by the sensors.

[0099] Step 3: Design the TD3 agent, using the estimated state output by the improved high-order sliding mode observer as input, to construct the state space, action space, and reward function adapted to the suspension control, and learn the optimal control strategy of the magnetorheological damper.

[0100] Step 4: Integrate IHOSMO and TD3 agent to design IHOSMO-TD3 magnetorheological suspension control method, realize adaptive cooperative control of magnetorheological suspension, and perform stability analysis on joint closed-loop system.

[0101] Step 5: Build a quarter magnetorheological suspension test platform, collect experimental data under bumpy road surface and random road surface conditions, compare the performance indicators of different control strategies, and verify the engineering effectiveness of the adaptive cooperative control method.

[0102] See Figure 2 In step 1, the dynamic equation of the quarter-vehicle magnetorheological suspension model is as follows:

[0103] (1)

[0104] In the formula, m s For the sprung mass, m u For unsprung mass, k s For the spring stiffness, k u For tire stiffness, F d The damping force output by the magnetorheological damper. and These represent the displacements of the vehicle body and the unsprung mass, respectively. Represents vehicle acceleration. Represents suspension dynamic deflection. This represents the tire's travel distance. Input for road surface displacement;

[0105] Using the modified Bingham model, the output force of the magnetorheological damper can be expressed as:

[0106]

[0107] in, To control the current The relevant yield strength, For friction compensation, and Let be the vehicle body speed and the unsprung mass speed, respectively, and sgn be the sign function. To describe the moderating effect of current on yield force, a polynomial approximation is used:

[0108] (3)

[0109] Parameters in the formula These are the polynomial fitting coefficients, which can be obtained through experimentation;

[0110] Define the system state variables as follows:

[0111] (4)

[0112] The MR suspension system can then be rewritten in state-space form:

[0113] (5)

[0114] The system output is selected as follows:

[0115] (6)

[0116] In the formula Let be the first derivative of the state vector, A be the system state matrix, and B be the system input matrix. For the system input vector, Here is the road surface displacement disturbance matrix. Input for road surface displacement disturbance;

[0117] This means measuring the output related to the suspension dynamic deflection and vehicle speed. Since some states (such as unsprung mass velocity and road disturbance) cannot be directly measured, an observer is needed to reconstruct them.

[0118] Step 2, designing the improved high-order sliding mode observer (IHOSMO), specifically involves:

[0119] Starting with the sliding mode injection structure and nonlinear adjustment mechanism, the HOSMO was improved and redesigned. The observer structure is shown in [reference needed]. Figure 3 As shown,

[0120] Let the observer state estimate be The output error is defined as:

[0121] (7)

[0122] in To estimate the error, This is the estimated value of the system state vector. For system output estimation, This is the output matrix;

[0123] IHOSMO is designed as follows:

[0124] (8)

[0125] Where L is the linear observation gain. To assist in the integration state, For higher-order sliding mode correction terms; e is the system output estimation error vector, i.e., e as defined in equation (7). y f(·) is the nominal dynamic function of the system;

[0126] For magnetorheological suspension systems, it can be specifically written as follows:

[0127] (9)

[0128] In the formula These are the estimated values ​​for the 1st to 4th system states, corresponding to the suspension dynamic deflection in turn. Vertical speed of the vehicle body Tire travel Unsprung mass vertical velocity The estimation results; These are the estimated values ​​for the first to fourth system states, respectively. This is an estimate of the output damping force of the magnetorheological damper. This is an estimate of the vertical input velocity of the road surface. The four elements of the linear observation gain matrix L correspond to the linear feedback correction gains for the four states, and are determined globally through the GQPSO optimization algorithm in step 2. For the superhelical sliding mode switching function that integrates saturated boundary layers, This is the gain coefficient of the higher-order sliding mode correction term, used to adjust the strength of the sliding mode correction. Auxiliary Integral Status We obtain the first derivative; The update gain coefficients for the auxiliary integral state are used to adjust the convergence speed of the perturbation estimation; It is a saturation function used to replace the sign function of traditional sliding mode, suppressing high-frequency chattering from the root. Its complete expression is shown in equation (12). The saturated boundary layer thickness is used to balance the convergence speed and chattering suppression effect of the sliding mode observer. The thicker the boundary layer, the smaller the chattering, but the steady-state error increases slightly.

[0129] in Adopting a superspiral type term:

[0130]

[0131]

[0132] in This is the proportional gain coefficient for the superspiral term, used to adjust the approach speed of the sliding surface; The integral gain coefficient of the superspiral term is used to adjust the intensity of the integral correction; The integral state variable for the superspinning term is used to realize higher-order sliding model properties; For integration state variables The first derivative; The error in estimating the vertical acceleration of the vehicle body; Estimation error of vehicle body vertical acceleration The absolute value;

[0133] To reduce chattering, Replace it with a saturation function, the expression of which satisfies equation (12):

[0134] (12)

[0135] From equations (5) and (10)–(12), we can obtain the estimation error dynamics:

[0136] (13)

[0137] in, ,in This represents the actual state of the suspension system. The state estimate output by the observer is defined in the same way as in equation (9); The estimation error of the vertical input velocity of the road surface is defined as the actual road surface velocity. Estimating road surface speed with the observer The difference, i.e. ; The estimation error of the output damping force of the magnetorheological damper is defined as the true damping force. The difference between the estimated damping force and the value of the damping force, if satisfying If the modeling error is bounded, then all unmodeled terms can be uniformly represented as disturbance terms. .

[0138] To prove the convergence of observation errors, we construct the Lyapunov function:

[0139] (14)

[0140] P is a positive definite symmetric matrix, and is a Lyapunov equation. The solution (where A is the state matrix of the observation error system and Q is an arbitrarily selected positive definite symmetric matrix); this matrix guarantees the strict positive definiteness of the Lyapunov function V, and its elements can be calculated from the gain parameters of the observer.

[0141] It can be deduced that if the perturbation is bounded, the observation error is uniformly bounded eventually; if the perturbation is zero, the observation error asymptotically converges, and due to the properties of the superspiral term, the observation error can converge to the zero neighborhood in a finite time.

[0142] To improve estimation accuracy and address the local optima problem of the IHOSMO gain parameters, the GQPSO optimization method is employed to optimize the observer gain parameters. The objective function is defined as follows:

[0143] (15)

[0144] Where RMSE(·) is the root mean square error function, used to measure the overall steady-state accuracy of the state estimation; These are the weighting coefficients for each performance indicator, used to balance the importance of the vehicle body vertical velocity estimation accuracy, the unsprung mass vertical velocity estimation accuracy, and the peak acceleration error.

[0145] By leveraging the global search capability of the GQPSO algorithm, globally optimal tuning of the observer parameters is achieved. Comparative experiments show that GQPSO outperforms the traditional PSO and QPSO algorithms in both convergence speed and optimization accuracy. (See [link to relevant documentation]). Figure 4 As shown,

[0146] Step 3, the core of the coupling between the TD3 algorithm and the MR suspension, is the intelligent agent design, specifically implemented as follows:

[0147] 1) State-space design

[0148] Combining the MR suspension dynamics model, the IHOSMO estimated state is constructed as the TD3 input state:

[0149] (16)

[0150] in This represents the equivalent disturbance or auxiliary state output by the observer. This design allows TD3 to make control decisions not based on direct measurements of unmeasured speeds and road disturbances, but by utilizing "enhanced state information" provided by the observer.

[0151] 2) Motion space design

[0152] TD3 outputs continuous action And it is converted into the target current or target damping force of the damper through a mapping function:

[0153] (17)

[0154] (18)

[0155] in The raw, continuous action values ​​output by the TD3 agent network at time t; This represents the minimum damping force that the magnetorheological damper is allowed to output. This represents the maximum allowable damping force of the magnetorheological damper. The target damping force of the magnetorheological damper at time t is used to limit the amplitude of the control input and prevent the damping force from exceeding the physical limit of the actuator. This is a current-damping force mapping function. Since the magnetorheological damper has physical saturation, the output needs to be limited.

[0156] 3) Reward function design:

[0157] To balance comfort and safety, this invention employs a quadratic negative reward function: the higher the suspension performance index, the smaller the reward value. By maximizing the reward objective through the algorithm, adverse conditions are suppressed. The reward function is defined as follows:

[0158] (19)

[0159] in, As the weighting coefficient, the quadratic negative reward function encourages the system to reduce vehicle acceleration, suppress suspension deflection and tire dynamic deformation, while avoiding excessive control fluctuations; denoted as the rate of change of magnetorheological damping force at time t, this term is used to suppress drastic fluctuations in control input, extend the service life of the magnetorheological damper, and reduce actuator energy consumption.

[0160] 4) Neural network structure design

[0161] In the TD3 algorithm, its core neural network architecture consists of two Critic value networks and one Actor policy network. Its network structure is shown below. Figure 5 As shown:

[0162] All hidden layers of the network use the ReLU activation function to solve the gradient vanishing problem. The input of the Critic network is a concatenated vector of state and action with a dimension of 5+1=6. The output layer uses a linear activation method to output the Q value. The output layer of the Actor network uses the tanh function as the activation function to ensure that the output value is in the range of [-1,1], and then converts it into the actual damping force through mapping.

[0163] Meanwhile, the neural network structure of the TD3 agent includes two Critic value networks and one Actor policy network; the input of the Critic network is a concatenated vector of state and action; the output of the Actor network is mapped to the actual damping force after being activated by the tanh activation function; TD3 adopts the mechanism of overestimation suppression of dual Critic networks, smoothing of target policy, and soft update, which effectively solves the overestimation and convergence instability problems of the traditional DDPG algorithm.

[0164] Step 4, the design and stability analysis of the IHOSMO-TD3 joint controller, is specifically carried out as follows:

[0165] The joint closed-loop system is decomposed into an observation error subsystem and a control subsystem. Stability analysis is performed based on the input-to-state stability (ISS) theory. Under the condition that the observation error, policy approximation error and external disturbances are bounded, it is proved that the joint closed-loop system satisfies input-to-state stability and the system state is uniformly bounded. Under ideal conditions, the system can achieve asymptotic stability.

[0166] Unlike traditional control methods that rely on direct measurement of the state, the IHOSMO-TD3 joint controller fully utilizes the observer's ability to reconstruct unmeasurable and noise-contaminated states. This allows the TD3 controller to learn the optimal damping force adjustment strategy based on more complete and smoother state information. This joint design not only improves the quality of state perception but also enhances the robustness and feasibility of the control strategy under nonlinear and random disturbance conditions. The framework of the joint control is shown in [link to joint control framework]. Figure 6 As shown.

[0167] Consider system dynamics:

[0168] (20)

[0169] in This represents the deterministic control policy output by the TD3 agent Actor policy network, where θ represents the trainable weight parameters of the Actor neural network. To improve the system state estimate output by the high-order sliding mode observer (IHOSMO), The system has known dynamics, and g is the input matrix. The total disturbance term is composed of road surface excitation, unmodeled nonlinearity, and external disturbance.

[0170] By incorporating IHOSMO's nonlinear compensation, the system can be expanded to:

[0171] (twenty one)

[0172] in, For the augmented state vector, To enhance system dynamics, To control the input matrix, This represents the total uncertainty term;

[0173] because Therefore:

[0174] (twenty two)

[0175] Under local Lipschitz conditions, there exists The constant makes:

[0176] (twenty three)

[0177] in is the Euclidean norm of a vector, used to measure the magnitude of the vector; It is an ideal deterministic control strategy based on the real state of the system, that is, the output of the TD3 agent network when the real state x is the input.

[0178] Therefore, the observation error will be directly transmitted to the control input, and its effect is equivalent to an additional disturbance. Thus, the higher the estimation accuracy of IHOSMO, the closer TD3 control is to the ideal strategy.

[0179] Since the actual controller receives the estimated state rather than the actual state, the joint control input is expressed as:

[0180] (twenty four)

[0181] in The deterministic mapping function for the TD3 agent policy network, and the π function mentioned earlier. θ (·) represents the same function; here, μ is used to adapt to the augmented system notation. θ express. The estimated value of the augmented state vector is obtained from the output of the Improved High-Order Sliding Mode Observer (IHOSMO); Let the augmented state estimation error vector be defined as the difference between the true augmented state and the estimated augmented state, i.e. This includes the estimation error of the original system state and the estimation error of the observer-aided integral state;

[0182] For policy function exist Expanding to the first order within the neighborhood, we have:

[0183] (25)

[0184] in Let be the Jacobian matrix of the policy function with respect to the augmented state vector, which describes the gradient of the policy function output as a function of the augmented state. The higher-order infinitesimal remainder term in Taylor expansion represents the remainder when the observation error... When the value is sufficiently small, the magnitude of this term does not exceed the square of the magnitude of the observation error, and its dominant influence can be ignored in stability analysis.

[0185] From equations (21) and (24), the IHOSMO-TD3 closed-loop system can be expressed as:

[0186] (26)

[0187] Further breakdown into:

[0188] (27)

[0189] in The control input error term is the deviation between the actual control and the ideal control caused by the observation error. Its effect is equivalent to the additional disturbance acting on the control channel. The magnitude of this term directly determines the performance gap between the joint system and the ideal system.

[0190] Therefore, the estimation error introduced by IHOSMO is equivalent to an additional input disturbance in the control channel.

[0191] To ensure the effectiveness of the joint controller, this invention performs a stability analysis on the joint closed-loop system:

[0192] The joint closed-loop system is decomposed into an observation error subsystem and a control subsystem. Based on the ISS theory, under the assumptions of Lipschitz continuity of system dynamics, bounded disturbance, bounded observation error, Lipschitz continuity of policy network, and bounded policy approximation error, it is proved that the joint closed-loop system satisfies input-to-state stability with respect to external disturbance, observation error, and policy approximation error. That is, the system state is uniformly bounded, and the steady-state error is positively correlated with the upper bound of the three error sources.

[0193] Under ideal conditions, namely when the observation error converges to zero, the strategy approximation error disappears, and external disturbances are negligible, the joint closed-loop system can achieve asymptotic stability, which theoretically guarantees the reliability of the control method.

[0194] To verify the effectiveness of the proposed IHOSMO-TD3 joint control strategy, a simulation platform was first built in MATLAB / Simulink. (See attached diagram.) Figure 7 As shown, the control performance of passive suspension, PID, DQN, TD3 and the IHOSMO-TD3 of the present invention were compared for different working conditions of bumpy road surface and ISO-C grade random road surface.

[0195] The reward convergence curve during the training phase is shown in [reference needed]. Figure 8 As shown, the IHOSMO-TD3 algorithm of this invention has the fastest convergence speed, reaching convergence in 20 rounds, and the highest final reward value, reaching approximately 3600, which is significantly better than DQN and pure TD3 algorithms.

[0196] Under single / double bump road surface conditions at low speeds of 10m / s and high speeds of 20m / s, the IHOSMO-TD3 controller exhibits the best overall vibration reduction performance. At low speeds, its root mean square (RMS) acceleration is 26.6% lower than the passive suspension and 3.3% lower than the TD3 control; suspension dynamic deflection and tire travel peak values ​​are reduced by 26.3% and 26.5% respectively, with no significant residual oscillations. At high speeds, the RMS acceleration is 28.3% lower than the passive suspension and 5.1% lower than the TD3 control; the RMS suspension dynamic deflection is reduced by 4.9%, exhibiting the fastest vibration decay rate. Frequency domain analysis shows that this controller accurately suppresses vibrations in the sensitive 4-8Hz frequency band, achieving an optimal balance between ride comfort and driving safety. See [link / details omitted] Figures 9-13 As shown.

[0197] Under random road surface conditions at low speeds of 10 m / s and high speeds of 20 m / s, the IHOSMO-TD3 controller exhibits excellent continuous vibration suppression capabilities. At low speeds, its root mean square (RMS) vehicle acceleration is reduced by 16.7% compared to passive suspension and by 3.0% compared to TD3 control; the RMS values ​​of suspension dynamic deflection and tire dynamic travel are reduced by 22.3% and 24.2%, respectively. The advantages are even more pronounced at high speeds, with the RMS vehicle acceleration reduced by 20.5% compared to passive suspension and by 8.0% compared to TD3 control, and the RMS suspension dynamic deflection reduced by 31.8%. This controller offers a smoother response, significantly attenuates high-frequency vibration components, and achieves optimal vibration isolation performance across the entire frequency range. Even under complex random excitation at high speeds, it maintains both ride comfort and driving safety. Figures 14-18 As shown.

[0198] Step 5, which involves constructing a quarter-magnetorheological suspension experimental platform to verify the engineering effectiveness of the control method, is as follows:

[0199] See Figures 19-21As shown, the platform comprises three main modules:

[0200] 1) Excitation simulation module: The servo electric cylinder generates road excitation, which is connected to the tire stiffness spring to simulate the vertical elasticity of the tire;

[0201] 2) Suspension body module: The magnetorheological damper is connected in parallel with the suspension stiffness spring, supporting the sprung mass plate with precise weighting, matching the target vehicle's quarter sprung mass parameters;

[0202] 3) Measurement and control system module: The lower-level STM32 controller executes the control algorithm at a frequency of 1kHz, while the upper-level industrial PC uses LabVIEW to realize the full process control of the experiment and collect core data such as displacement, acceleration, and force.

[0203] Based on this platform, this invention conducted experimental verification of bumpy road surfaces and random road surfaces. The experimental principle is described below. Figure 20 As shown: Multiple sensors collect real-time data on vehicle vertical acceleration, tire displacement, and other suspension motion states. The data acquisition unit performs signal conditioning and analog-to-digital conversion to provide raw input for the control algorithm. The reinforcement learning algorithm module developed in the Keil environment calculates the optimal damping force control law, which is parsed and commands are issued in real time by the STM32 microcontroller to ensure high real-time control. Finally, the magnetorheological damper is driven to output active control force, while the servo electric cylinder reproduces the driving conditions according to the preset road surface spectrum, realizing a quantitative comparison of suspension performance under different control strategies.

[0204] In the 10 m / s bump road test, the IHOSMO-TD3 controller significantly outperformed the comparison scheme in vibration reduction. The passive suspension exhibited a peak positive acceleration close to 1.8 m / s² and the slowest oscillation decay, while the TD3 control still showed significant residual oscillation. In contrast, the IHOSMO-TD3 demonstrated the most outstanding suppression of both positive and negative peak values, resulting in the fastest recovery of the vehicle body after impact. Its vehicle acceleration RMS value was 0.376 m / s², a 25.2% reduction compared to the passive suspension and a further 12.8% improvement over the TD3 control, validating the effectiveness of this method in real-world physical systems. (See...) Figure 22 As shown;

[0205] In the 10 m / s random road surface test, see Figure 23As shown, the IHOSMO-TD3 controller exhibits stable and continuous vibration reduction capability. The passive suspension exhibits the largest fluctuation in vehicle acceleration, with positive and negative peak values ​​approaching 1.8 m / s² and -1.7 m / s², respectively, indicating that pure TD3 control still exhibits significant high-frequency fluctuations. In contrast, the vibration intensity is significantly reduced under IHOSMO-TD3 control, with a smoother response curve. Its vehicle acceleration RMS value is 0.451 m / s², a 12.3% reduction compared to the passive suspension and a further 5.3% optimization compared to pure TD3 control. This verifies the engineering feasibility and effectiveness of the method of this invention.

Claims

1. An adaptive control method for magnetorheological suspension based on IHOSMO-TD3, characterized in that, Includes the following steps: Step 1: Establish a dynamic model of the quarter-vehicle magnetorheological suspension and parametrically describe the nonlinear output characteristics of the magnetorheological damper. Step 2: Design an improved high-order sliding mode observer to reconstruct the unmeasurable critical state and equivalent disturbance of the magnetorheological suspension based on the output signal collected by the sensor. Step 3: Design a dual-delay deep deterministic policy gradient agent. Using the estimated state output by the improved high-order sliding mode observer as input, construct the state space, action space, and reward function adapted to the suspension control, and learn the optimal control strategy of the magnetorheological damper. Step 4: The improved high-order sliding mode observer and the dual-delay deep deterministic policy gradient agent are jointly integrated to design the IHOSMO-TD3 magnetorheological suspension control method, realize the adaptive cooperative control of the magnetorheological suspension, and perform stability analysis on the joint closed-loop system. Step 5: Build a quarter magnetorheological suspension experimental platform, compare the performance indicators of different control strategies, and verify the engineering effectiveness of the adaptive cooperative control method.

2. The adaptive control method for magnetorheological suspension based on IHOSMO-TD3 according to claim 1, characterized in that, In step 1, the dynamic equation of the quarter vehicle magnetorheological suspension model is represented by equation (1); （1）； In the formula, m s For the sprung mass, m u For unsprung mass, k s For the spring stiffness, k u For tire stiffness, F d The damping force output by the magnetorheological damper. and These represent the displacements of the vehicle body and the unsprung mass, respectively. Represents vehicle acceleration. Represents suspension dynamic deflection. This represents the tire's travel distance. Input for road surface displacement; The output force of the magnetorheological damper adopts the modified Bingham model, and the output force of the magnetorheological damper is expressed by equation (2): ； in, To control the current The relevant yield strength, For friction compensation, and The vehicle speed and unsprung mass speed are respectively, and sgn is the sign function. To describe the effect of current on yield force, a polynomial approximation (3) is used: （3）； Parameters in the formula These are the polynomial fitting coefficients, which can be obtained through experimentation; The system state variables are defined by equation (4): （4）； The state-space form of the system is shown in equation (5): （5）； In the formula, A is the system state matrix, and B is the system input matrix. For the system input vector, Here is the road surface displacement disturbance matrix. Input for road surface displacement disturbance; The system output is defined by equation (6): （6）。 3. The adaptive control method for magnetorheological suspension based on IHOSMO-TD3 according to claim 1, characterized in that, In step 2, the design of the improved high-order sliding mode observer includes: defining the observer state estimate and output error, the error of which is defined to satisfy equation (7): （7）； in To estimate the error, This is the estimated value of the system state vector. For system output estimation, This is the output matrix; The observer dynamics that integrate the saturated boundary layer and superhelical mechanism are constructed, and their expressions satisfy equations (8) and (9): （8）； Where L is the linear observation gain. For the auxiliary integration state, e is the system output estimation error vector, i.e., e as defined in equation (7). y , For higher-order sliding mode correction terms, f(·) is the nominal dynamic function of the system; （9）； In the formula These are the estimated values ​​for the 1st to 4th system states, corresponding to the suspension dynamic deflection in turn. Vertical speed of the vehicle body Tire travel Unsprung mass vertical velocity The estimation results; These are the estimated values ​​for the first to fourth system states, respectively. This is an estimate of the output damping force of the magnetorheological damper. This is an estimate of the vertical input velocity of the road surface. The four elements of the linear observation gain matrix L correspond to the linear feedback correction gains for the four states, and are determined globally through the GQPSO optimization algorithm in step 2. For the superhelical sliding mode switching function that integrates saturated boundary layers, This is the gain coefficient of the higher-order sliding mode correction term, used to adjust the strength of the sliding mode correction. Auxiliary Integral Status We obtain the first derivative; The update gain coefficients for the auxiliary integral state are used to adjust the convergence speed of the perturbation estimation; It is a saturation function used to replace the sign function of traditional sliding mode, suppressing high-frequency chattering from the root. Its complete expression is shown in equation (12). The saturated boundary layer thickness is used to balance the convergence speed and chattering suppression effect of the sliding mode observer. The thicker the boundary layer, the smaller the chattering, but the steady-state error increases slightly. in The superhelical term is adopted, and the expression of the superhelical term satisfies equations (10) and (11): （10）； （11）； in This is the proportional gain coefficient for the superspiral term, used to adjust the approach speed of the sliding surface; The integral gain coefficient of the superspiral term is used to adjust the intensity of the integral correction; The integral state variable for the superspinning term is used to realize higher-order sliding model properties; For integration state variables The first derivative; The error in estimating the vertical acceleration of the vehicle body; Estimation error of vehicle body vertical acceleration The absolute value; To reduce chattering, Replace it with a saturation function, the expression of which satisfies equation (12): （12）； The estimation error dynamics satisfy equation (13): （13）； in, ,in This represents the actual state of the suspension system. The state estimate output by the observer is defined in the same way as in equation (9); The estimation error of the vertical input velocity of the road surface is defined as the actual road surface velocity. Estimating road surface speed with the observer The difference, i.e. ; The estimation error of the output damping force of the magnetorheological damper is defined as the true damping force. The difference between the estimated damping force and the value of the damping force, if satisfying If the modeling error is bounded, then all unmodeled terms can be uniformly represented as disturbance terms. .

4. The adaptive control method for magnetorheological suspension based on IHOSMO-TD3 according to claim 1, characterized in that, Step 2 further includes: optimizing the gain parameters of the improved high-order sliding mode observer (IHOSMO) using the GQPSO optimization algorithm, wherein the optimization objective function satisfies equation (15): （15）； Where RMSE(·) is the root mean square error function, used to measure the overall steady-state accuracy of the state estimation; These are the weighting coefficients for each performance indicator, used to balance the importance of the vehicle body vertical velocity estimation accuracy, the unsprung mass vertical velocity estimation accuracy, and the peak acceleration error.

5. The adaptive control method for magnetorheological suspension based on IHOSMO-TD3 according to claim 1, characterized in that, In step 3, the design of the TD3 agent includes: state space design: constructing the input state of TD3 using the estimated state output by the improved high-order sliding mode observer (IHOSMO), wherein the input state vector satisfies equation (14): （16）； in This represents the equivalent disturbance or auxiliary state output by the observer; Action space design: The continuous output action of TD3 is mapped to the target damping force of the magnetorheological damper, and the mapping relationship satisfies equation (15): （17）； The raw, continuous action values ​​output by the TD3 agent network at time t; This represents the minimum damping force that the magnetorheological damper is allowed to output. This represents the maximum damping force that the magnetorheological damper can output. Reward function design: A quadratic negative reward function is adopted, the expression of which satisfies equation (19): （19）； in, As the weighting coefficient, the quadratic negative reward function encourages the system to reduce vehicle acceleration, suppress suspension deflection and tire dynamic deformation, while avoiding excessive control fluctuations; The target damping force of the magnetorheological damper at time t is used to limit the amplitude of the control input and prevent the damping force from exceeding the physical limit of the actuator. denoted as the rate of change of magnetorheological damping force at time t, this term is used to suppress drastic fluctuations in control input, extend the service life of the magnetorheological damper, and reduce actuator energy consumption.

6. The adaptive control method for magnetorheological suspension based on IHOSMO-TD3 according to claim 1, characterized in that, In step 3, the neural network structure of the TD3 agent includes two Critic value networks and one Actor policy network; the input of the Critic network is a concatenated vector of state and action; the output of the Actor network is mapped to the actual damping force after being activated by the tanh activation function; the TD3 agent adopts a mechanism of overestimation suppression of dual Critic networks, target policy smoothing, and soft update.

7. The adaptive control method for magnetorheological suspension based on IHOSMO-TD3 according to claim 1, characterized in that, In step 4, the stability analysis of the IHOSMO-TD3 controller specifically involves: decomposing the joint closed-loop system into an observation error subsystem and a control subsystem, and performing stability analysis based on the input-to-state stability (ISS) theory; under the condition that the observation error, strategy approximation error, and external disturbances are bounded, proving that the joint closed-loop system satisfies input-to-state stability, and that the system state is uniformly bounded eventually; under ideal conditions, the system can achieve asymptotic stability.

8. The adaptive control method for magnetorheological suspension based on IHOSMO-TD3 according to claim 1, characterized in that, In step 5, the experimental platform includes an excitation simulation module, a suspension body module, and a measurement and control system module; the experimental verification includes testing under bumpy road surface and random road surface conditions, and comparing the vehicle body acceleration response of passive suspension, TD3 control and the method of this method.

Images