An active suspension end-to-end control method for intelligent chassis of motor vehicles

Abstract

The application relates to a motor vehicle intelligent chassis-oriented active suspension end-to-end control method, which comprises the following steps: constructing a nonlinear quarter-vehicle vertical dynamics model; constructing a high-fidelity dynamics model of a chassis active actuator containing multi-physical coupling dynamic characteristics; constructing a state-related reward function adjustment mechanism based on a vehicle vertical dynamic state; constructing an observation space, an action space and a reward function according to the vehicle vertical dynamics model, the high-fidelity dynamics model of the chassis active actuator and the state-related reward function adjustment mechanism; training an end-to-end adaptive controller through reinforcement learning; inputting current vehicle state information into the end-to-end adaptive controller; and directly mapping output of the chassis active actuator driving signal. Compared with the prior art, the application can simultaneously consider the real dynamic characteristics of the actuator and the adaptability to complex roads, and improves the ride comfort, tire ground performance and control robustness of the active suspension system under complex road conditions.

Description

Technical Field

[0001] This invention relates to the field of active suspension control technology for vehicles, and in particular to an end-to-end active suspension control method for intelligent chassis of motor vehicles. Background Technology

[0002] With the rapid development of motor vehicles towards electrification and intelligence, drive-by-wire chassis have become a crucial foundation for improving overall vehicle dynamics, active safety, and intelligent driving capabilities. As an important component of drive-by-wire chassis, electronically controlled suspension systems can adjust suspension response based on vehicle operating conditions and changes in road surface excitation, thereby improving ride comfort, tire contact performance, and vehicle stability. Among these, active suspension systems can actively generate bidirectional control forces through externally powered actuators, offering stronger vibration damping and dynamic adjustment capabilities compared to semi-active suspensions.

[0003] In active suspension systems, the dynamic characteristics of the active actuator directly affect the control effect and the achievable control bandwidth. Hydraulic active actuators have attracted widespread attention due to their high force density and fast power transfer capability; however, they typically contain multiple physically coupled components such as motor-pump units, electro-hydraulic valves, hydraulic chambers, and piston assemblies, exhibiting significant nonlinearity, response hysteresis, output saturation, and energy consumption constraints. Most existing methods employ a hierarchical control structure, where the upper-level controller calculates the desired suspension force, and the lower-level force-tracking controller converts it into actuator drive signals such as valve current and motor torque. While this hierarchical control structure is convenient for design, its control performance depends on the accuracy of the actuator model, and the lower-level force-tracking component may introduce delays and phase lags, leading to unreliable actual control effects.

[0004] In recent years, reinforcement learning has attracted attention in active suspension control due to its ability to learn control strategies through interaction with the environment and its applicability to nonlinear systems and multi-objective optimization problems. However, most existing reinforcement learning suspension control methods simplify actuators to ideal force sources or simple constraint force sources, failing to fully consider the nonlinear dynamic characteristics of real chassis active actuators. This results in a gap between the learned control strategy and the actual executable drive signal. Furthermore, vehicles encounter various excitation forms during actual driving, such as random road surfaces, potholes, bulges, and random-impact composite road surfaces. Existing strategies are usually trained only under a single road surface condition or rely on road surface recognition and multi-strategy switching, resulting in high system complexity and adaptability affected by the accuracy of recognition. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art by providing an end-to-end active suspension control method for intelligent chassis of motor vehicles, which can simultaneously consider the real dynamic characteristics of actuators and adaptability to complex road surfaces, thereby improving the ride comfort, tire contact performance and control robustness of the active suspension system under complex road conditions.

[0006] The objective of this invention can be achieved through the following technical solution: an end-to-end active suspension control method for intelligent chassis of motor vehicles, comprising the following steps: S1. Construct a nonlinear quarter-vehicle vertical dynamics model to characterize the vehicle's vertical motion, suspension relative motion, and tire contact state under road surface excitation. S2. Construct a high-fidelity dynamic model of the chassis active actuator that includes multi-physical coupling dynamic characteristics to characterize the nonlinear mapping relationship between the actuator drive signal and the output force. S3. A state-related reward function adjustment mechanism is constructed based on the vertical dynamic state of the vehicle to adaptively weight the vehicle body acceleration, suspension dynamic deflection, unsprung mass response and tire contact state related indicators. S4. Based on the vehicle vertical dynamics model, the high-fidelity dynamics model of the chassis active actuator, and the state-related reward function adjustment mechanism, construct the observation space, action space, and reward function. Train the end-to-end adaptive controller through reinforcement learning, input the current vehicle state information into the end-to-end adaptive controller, and directly map and output the chassis active actuator drive signal to control the working state of the chassis active actuator.

[0007] Furthermore, S1 includes the following process: S11. Determine the suspension dynamic deflection based on the relative displacement between the sprung and unsprung masses of the vehicle, and construct the suspension equivalent stiffness based on the suspension dynamic deflection. The suspension equivalent stiffness includes the stiffness of the suspension elastic elements, the nonlinear stiffness of the upper stop block, and the nonlinear stiffness of the lower stop block. The suspension equivalent stiffness is expressed as: in, k s,eff This is the equivalent stiffness of the suspension. k s For the stiffness of the suspension elastic element, and These are the nonlinear stiffnesses of the upper stop block and the lower stop block, respectively. and These are the clearances corresponding to the upper and lower stop blocks, respectively. z s and z u These represent the vertical displacements of the sprung mass and the unsprung mass, respectively. S12. Determine the tire dynamic deflection based on the relative displacement between the unsprung mass and the road surface excitation, and determine the contact state between the tire and the road surface based on the tire dynamic deflection and the tire static compression, thereby calculating the tire force, which is expressed as: in, F t For the force exerted by the tire, k t For tire ground stiffness, z g For road surface displacement input, r 0 This refers to the static compression of the tire. m s For the sprung mass, m u For unsprung mass, It is the acceleration due to gravity; S13. Based on the equivalent stiffness of the suspension, suspension damping, tire force, and output force of the active actuator, establish the vertical dynamic equations for the sprung and unsprung masses: in, m s and m u They are spring-loaded mass and unspring-loaded mass, respectively. Let be the vertical acceleration of the sprung mass. The vertical acceleration of the unsprung mass. F a The output force of the active actuator is the total control force acting on the active suspension system.

[0008] Furthermore, S2 includes the following process: S21. Construct a dynamic model of a hydraulic active actuator. The dynamic model of the hydraulic active actuator includes one or more of a motor pump unit, an electro-hydraulic valve, a hydraulic chamber, and an actuator piston assembly. It is used to characterize the nonlinear dynamic relationship between the actuator drive signal and the actual active control force. S22. Use one or more of the compression valve current, the reset valve current and the motor torque command as the actuator drive signal, and obtain one or more of the hydraulic pressure difference, motor speed, motor torque, valve current and system power as the actuator working state. S23. Based on the actuator drive signal and actuator working state, calculate the actual active control force acting on the active suspension system. The actual active control force is input to the vehicle vertical dynamics model to form a closed-loop coupling between the vehicle dynamics model and the actuator dynamics model.

[0009] Furthermore, S3 includes the following process: S31. Obtain the unsprung mass acceleration signal, and construct a pavement excitation intensity characterization quantity based on the unsprung mass acceleration signal. The pavement excitation intensity characterization quantity is the unsprung mass acceleration envelope, which is expressed as: in, a u ( k ) is the first k The unsprung mass acceleration at each sampling time, a env ( k ) is the first k The envelope of unsprung mass acceleration at each sampling time α Let be the envelope attenuation coefficient, and 0 < 0. α <1; S32. Determine the excitation intensity threshold based on the unsprung mass response characteristics of low-amplitude random road excitation and high-amplitude transient impact road excitation. a th The excitation intensity threshold is used to distinguish between low-intensity continuous random excitation and high-intensity transient impact excitation; wherein, the low-amplitude random road surface excitation and the high-amplitude transient impact road surface excitation can be used individually as training road surface conditions or combined to form composite road surface conditions. S33. Obtain the reference weights of the reward function under low-amplitude random road surface excitation through pre-simulation or offline calibration. c i,ran Reference weights for the reward function under high-amplitude transient impact road surface excitation c i,imp ,in, c i,ran and c i,imp Used to characterize the first i The optimization priority of each performance index under low-amplitude random road conditions and high-amplitude transient impact conditions; S34. Based on the unsprung mass acceleration envelope a env ( k ), excitation intensity threshold a th Reference weights for the reward function under low-amplitude random road surface incentives ci,ran and the reference weight of the reward function under high-amplitude transient impact road surface excitation. c i,imp Construct state-dependent reward functions and weighted transition functions: in, c i ( k ) is the first k The sampling time of the first sampling moment i Adaptive weights for each reward item p It is a transition index, and p >0, when a env ( k )≤ a th At that time, the weight of the reward function transitions continuously with the intensity of the road surface incentive; when a env ( k )> a th At that time, the reward function weights are maintained at the reference weights corresponding to high-amplitude transient shock conditions to improve control stability under strong transient disturbances.

[0010] Furthermore, S4 includes the following process: S41. Based on the dynamic response of the vehicle vertical dynamics model and the chassis active actuator high-fidelity dynamics model, construct the observed state variables in the observation space; The actuator drive signal of the high-fidelity dynamic model of the chassis active actuator is used as the motion space; Based on the state-dependent reward function adjustment mechanism, a reward function is designed, including a performance reward term and a soft constraint penalty term; S42. Using a teacher strategy guidance mechanism, combined with the observation space, action space and reward function, the end-to-end reinforcement learning controller is trained so that the end-to-end reinforcement learning controller can directly map the vehicle's vertical dynamic state and actuator working state into the chassis active actuator drive signal. S43. Input the current vehicle status information into the trained end-to-end adaptive controller, and output the chassis active actuator drive signal to control the working state of the chassis active actuator.

[0011] Furthermore, the specific process of S41 is as follows: Constructing the observation space s t for: ],in, This refers to the vehicle's vertical dynamic state. For the actuator's working state, The vertical dynamic state of the vehicle is a quantity characterizing the road surface excitation intensity. Including vehicle acceleration Vehicle speed Unsprung mass acceleration Suspension dynamic deflection z s - z u Relative speed with suspension The actuator working state Including the hydraulic pressure difference Δ at both ends of the piston rod p Motor speed n motor Motor torque T motor and actuator power P sys The road surface excitation intensity characterization quantity Including vehicle body acceleration envelope Unsprung mass acceleration envelope and suspension dynamic deflection envelope ; Constructing action space a t for: ,in, I c For compression valve current, I r To restore valve current, T rq This is the motor torque command; Design reward function R ( s t , a t )for: in, r t ( s t , a t ) is a performance bonus item. P This is a soft constraint penalty item. c i=1，2、3，4 The positive weighting coefficients are... A i=1，2、3，4 The normalization coefficient is... S max For suspension dynamic deflection constraints, For the unsprung mass acceleration envelope, This represents the electrode power.

[0012] Furthermore, the process of training the end-to-end reinforcement learning controller in S42 is specifically as follows: The teacher policy is pre-trained under random road surface excitation conditions, and the network parameters of the teacher policy are frozen during the training of the student policy. The teacher policy and the student policy have the same observation space and action space. Student policies are trained under composite road conditions including random road surface excitation and transient impact road surface excitation. During the training process, the teacher's policy actions and student's policy actions are evaluated based on the value function. The guidance strength of the teacher's policy on the student's policy is determined based on the evaluation results, so as to guide the student's policy using the teacher's policy guidance mechanism. At the same time, a distillation loss term is constructed based on the difference between the teacher's policy actions and the student's policy actions, and the distillation loss term is used together with the reinforcement learning policy loss for updating the student policy network parameters.

[0013] Furthermore, the process of evaluating teacher and student strategic actions based on the value function includes: In the k Each sampling time, based on the current state s k Teacher strategy actions were obtained separately. and student candidate actions The teacher's strategic action and the student's candidate action are respectively represented as follows: in, To freeze teacher policies, For deterministic output of student strategies; Then, the teacher's strategic actions are calculated using a value function. and student candidate actions Corresponding value assessment values, and teacher strategies and actions Compared to student candidate actions Value difference: in, Teacher strategy actions The corresponding value assessment value, Candidate actions for students The corresponding value assessment value, Teacher strategy actions Compared to student candidate actions The value difference, when Δ Q ( k )> orq When Δ is used, the teacher's strategic action is adopted as the action to be executed in the current training step; when Δ Q ( k )≤ or q When this happens, the student's candidate action is used as the action to be executed in the current training step. or q It determines the margin of value advantage and is used to suppress frequent switching caused by fluctuations in the value function.

[0014] Furthermore, the teacher strategy guidance mechanism includes a teacher intervention gating function and a progressive activation coefficient. The teacher intervention gating function determines whether teacher strategy guidance is turned off or on, and the progressive activation coefficient adjusts the intensity of teacher strategy guidance. The teacher intervention gating function is as follows: in, This is the current training round number. N warm Number of warm-up rounds for teacher strategy intervention; when When = 0, teacher-guided strategies are disabled, and only student strategies are used for interactive training. During the update phase, standard reinforcement learning loss is used to update the policy network and value network; when... When =1, teacher strategy guidance action selection is enabled in the interaction phase, and the student strategy network is jointly updated in the update phase using teacher-guided target action construction and distillation loss. The progressive activation coefficient is: in, N ramp The number of rounds for progressive activation guided by teacher strategies. β ramp ( e This is used to gradually increase the guiding role of teacher strategies on student strategies after the warm-up phase. clip (·) is the amplitude limiting function.

[0015] Furthermore, the distillation loss specifically refers to: Among them, for the batch training samples obtained from the experience replay buffer, N b For the number of training samples in a batch, the first... i The state of each sample is s i , m ϕ ( si ) for student strategies in state s i The deterministic action of the output, m ϕT ( s i (To freeze teacher strategies in state) s i The deterministic action of the output, e To prevent positive numbers with a denominator of zero, For the first i The teacher advantage weights for each sample, Δ Q i For the first i The value difference Δ between teacher's strategic actions and student's strategic actions in each sample Q max This is the normalization coefficient for the value difference; When teacher-guided strategies are enabled, the update objective of the student's strategy network is jointly constituted by the distillation loss term and the reinforcement learning strategy loss: in, To enhance the learning strategy loss, l dis ( e ) represents the distillation loss coefficient. r t This represents the proportion of samples in the current batch where the teacher's strategy has an effective value advantage. l 0 is the basic distillation coefficient. EMA (·) represents the exponential moving average operator. l min and l max These are the lower and upper limits of the distillation loss coefficient, respectively.

[0016] Compared with the prior art, the present invention has the following advantages: This invention constructs a nonlinear quarter-vehicle vertical dynamics model to characterize the vehicle's vertical motion, suspension relative motion, and tire contact state under road surface excitation. It also constructs a high-fidelity dynamics model of the chassis active actuators, incorporating multi-physics coupled dynamic characteristics, to depict the nonlinear mapping relationship between actuator drive signals and output forces. Furthermore, based on the vehicle's vertical dynamic state, a state-related reward function adjustment mechanism is built to adaptively weight vehicle acceleration, suspension deflection, unsprung mass response, and tire contact state-related indicators. Finally, based on the vehicle vertical dynamics model, the high-fidelity dynamics model of the chassis active actuators, and the state-related reward function adjustment mechanism, an observation space, action space, and reward function are constructed. An end-to-end adaptive controller is trained through reinforcement learning. Inputting the current vehicle state information into the end-to-end adaptive controller allows for direct mapping of the output chassis active actuator drive signals. This achieves direct mapping from vehicle state information to the actuator's underlying drive signals while considering the actual dynamic characteristics of the chassis active actuators, thereby improving the ride comfort, tire contact performance, and control robustness of the active suspension system under complex road conditions.

[0017] This invention embeds a high-fidelity dynamic model of the chassis active actuator into the reinforcement learning training environment, and uses the dynamic model of the hydraulic active actuator as a component of the training environment of the reinforcement learning controller. This allows the reinforcement learning controller to directly perceive the actual dynamic characteristics of the actuator, such as nonlinearity, response lag, output saturation, and energy consumption, during the training process, and output control signals to the underlying drive end of the actuator, thereby improving the matching between the control strategy and the actual actuator.

[0018] This invention adopts an end-to-end control architecture, which directly maps vehicle status information into actuator low-level drive signals such as compression valve current, reset valve current and motor torque. It omits intermediate force commands and lower-level force tracking links in traditional hierarchical control, which helps to reduce system complexity and reduce delay and phase lag caused by the force tracking process.

[0019] The present invention designs a state-dependent reward function adjustment mechanism that can adaptively adjust the weights of indicators such as vehicle acceleration, suspension deflection, unsprung mass response, and tire contact state according to the road excitation intensity, so that the control strategy can take into account ride comfort, suspension travel constraints, and tire contact stability.

[0020] This invention introduces a teacher strategy guidance and soft distillation mechanism to guide the student strategy in the state region where the teacher strategy has an effective value advantage. This improves the learning stability and control robustness of the reinforcement learning controller under random road surface, transient impact road surface and composite road surface conditions, and avoids additional reliance on explicit road surface recognition module and multi-mode switching control. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method of the present invention;

[0022] Figure 2 This is a schematic diagram of a quarter-vehicle hydraulic active suspension system model in the embodiment; Figure 3 This is a diagram of the end-to-end reinforcement learning active suspension control framework in the embodiment; Figure 4 This is a schematic diagram of the road surface excitation input in the embodiment; Figure 5 This is a layered display diagram of the time-domain response of the vehicle's vertical acceleration under different control methods in the embodiment, after constant longitudinal offset processing; Figure 6 for Figure 5 Local comparison diagrams of the original vertical acceleration response of the vehicle body after canceling the constant longitudinal offset within the corresponding time periods of Region I, Region II and Region III; Figure 7 This is a time-domain response diagram of the chassis active actuator output action under the method proposed in this invention, as shown in the embodiment. Figure 8 This is a schematic diagram illustrating the action value judgment and switching results of the teacher's strategy and the student's strategy in the embodiment. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0024] Example like Figure 1 As shown, an end-to-end active suspension control method for intelligent chassis of motor vehicles includes the following steps: S1. Construct a nonlinear quarter-vehicle vertical dynamics model to characterize the vehicle's vertical motion, suspension relative motion, and tire contact state under road surface excitation. S2. Construct a high-fidelity dynamic model of the chassis active actuator that includes multi-physical coupling dynamic characteristics to characterize the nonlinear mapping relationship between the actuator drive signal and the output force. S3. A state-related reward function adjustment mechanism is constructed based on the vertical dynamic state of the vehicle to adaptively weight the vehicle body acceleration, suspension dynamic deflection, unsprung mass response and tire contact state related indicators. S4. Based on the vehicle vertical dynamics model, the high-fidelity dynamics model of the chassis active actuator, and the state-related reward function adjustment mechanism, construct the observation space, action space, and reward function. Train the end-to-end adaptive controller through reinforcement learning, input the current vehicle state information into the end-to-end adaptive controller, and directly map and output the chassis active actuator drive signal to control the working state of the chassis active actuator.

[0025] In this embodiment, when constructing a nonlinear quarter-vehicle vertical dynamics model, as follows: Figure 2 As shown in the left half, this nonlinear quarter-vehicle vertical dynamics model includes sprung mass, unsprung mass, suspension elastic elements, suspension damping elements, tire equivalent stiffness, road excitation, and active actuator output force. The suspension dynamic deflection is determined based on the relative displacement between the sprung and unsprung masses, and the suspension equivalent stiffness is constructed based on the suspension dynamic deflection. The suspension equivalent stiffness includes the stiffness of the suspension elastic elements, the nonlinear stiffness of the upper stop block, and the nonlinear stiffness of the lower stop block. The suspension equivalent stiffness is expressed as: in, k s,eff This is the equivalent stiffness of the suspension. k s For the stiffness of the suspension elastic element, and These are the nonlinear stiffnesses of the upper stop block and the lower stop block, respectively. and These represent the clearances corresponding to the upper and lower stop blocks, respectively. In this embodiment, they are taken as 0.03 m and 0.06 m, respectively. z s and z u These represent the vertical displacements of the sprung mass and the unsprung mass, respectively.

[0026] The tire dynamic deflection is determined based on the relative displacement between the unsprung mass and the road surface excitation. The contact state between the tire and the road surface is then determined based on the tire dynamic deflection and the tire static compression, and the tire force is calculated accordingly. The tire force is expressed as: in, F t For the force exerted by the tire, k t For tire ground stiffness, z g For road surface displacement input, r 0 This refers to the static compression of the tire. m s For the sprung mass, m u For unsprung mass, and: Based on the equivalent stiffness of the suspension, suspension damping, tire force, and output force of the active actuator, the vertical dynamic equations for the sprung and unsprung masses are established: in, ms and m u They are spring-loaded mass and unspring-loaded mass, respectively. Let be the vertical acceleration of the sprung mass. The vertical acceleration of the unsprung mass. F a The output force of the active actuator is the total control force acting on the active suspension system. Since this embodiment uses a hydraulic active actuator, the output force of the active actuator... F a It includes the active force and equivalent damping effect caused by hydraulic pressure difference, valve orifice throttling characteristics, flow dynamics and piston movement. Therefore, the additional damping force term connected in parallel with the hydraulic active actuator is no longer introduced in the vehicle vertical dynamics model to avoid redundant modeling of the actuator damping effect.

[0027] In this embodiment, when constructing a high-fidelity dynamic model of the chassis active actuator, as follows: Figure 2 As shown in the right half, a hydraulic active actuator is used as the chassis active actuator object, and a dynamic model is established based on this hydraulic active actuator. The dynamic model of the hydraulic active actuator is used to characterize the nonlinear dynamic relationship between the actuator drive signal and the actual active control force. It includes a motor pump unit, an electro-hydraulic valve, a hydraulic chamber, and an actuator piston assembly. Figure 2 middle p 1 and p 2 represents the pressure in the lower chamber of the piston assembly and the pressure in the upper chamber of the piston assembly, respectively. It should be noted that... Figure 2 The hydraulic active actuator configuration shown is only used to illustrate the actuator dynamics modeling object in this embodiment and does not constitute a limitation on the scope of protection of this invention.

[0028] Compression valve current, reset valve current, and motor torque commands are used as actuator drive signals, which are directly output by the end-to-end reinforcement learning controller. Hydraulic pressure difference, motor speed, motor torque, actual valve current feedback value, and system power are acquired as actuator operating states, which form part of the reinforcement learning controller's observation space. Based on the actuator drive signals and actuator operating states, the actual active control force acting on the active suspension system is calculated. This actual active control force is input to the vehicle's vertical dynamics model to form a closed-loop coupling between the vehicle dynamics model and the actuator dynamics model. Subsequently, the hydraulic active actuator dynamics model is used as part of the reinforcement learning controller's training environment, allowing the controller to directly perceive actuator nonlinearity, response hysteresis, output saturation, and energy consumption characteristics during training, and learn the end-to-end mapping relationship from the vehicle's vertical dynamic state and actuator operating states to the actuator's underlying drive signals.

[0029] In this embodiment, when constructing the state-related reward function adjustment mechanism, the unsprung mass acceleration signal is first obtained, and a road excitation intensity characterization quantity is constructed based on the unsprung mass acceleration signal. The road excitation intensity characterization quantity is the unsprung mass acceleration envelope, which is expressed as: in, a u ( k ) is the first k The unsprung mass acceleration at each sampling time, a env ( k ) is the first k The envelope of unsprung mass acceleration at each sampling time α Let be the envelope attenuation coefficient, and 0 < 0. α <1; Based on the unsprung mass response characteristics of low-amplitude random road excitation and high-amplitude transient impact road excitation, the excitation intensity threshold is determined. a th The excitation intensity threshold is used to distinguish between low-intensity continuous random excitation and high-intensity transient impact excitation; among them, low-amplitude random road surface excitation and high-amplitude transient impact road surface excitation can be used as training road surface conditions alone, or combined to form composite road surface conditions. Subsequently, reference weights for the reward function under low-amplitude random road surface excitation were obtained through pre-simulation or offline calibration. c i,ran Reference weights for the reward function under high-amplitude transient impact road surface excitation c i,imp ,in, c i,ran and c i,imp Used to characterize the first i The optimization priority of each performance index under low-amplitude random road conditions and high-amplitude transient impact conditions; Then, based on the unsprung mass acceleration envelope a env ( k ), excitation intensity threshold a th Reference weights for the reward function under low-amplitude random road surface incentives c i,ran and the reference weight of the reward function under high-amplitude transient impact road surface excitation. c i,imp Construct state-dependent reward functions and weighted transition functions: in, c i (k ) is the first k The sampling time of the first sampling moment i Adaptive weights for each reward item p It is a transition index, and p >0, when a env ( k )≤ a th At that time, the weight of the reward function transitions continuously with the intensity of the road surface incentive; when a env ( k )> a th At that time, the reward function weights are maintained at the reference weights corresponding to high-amplitude transient shock conditions to improve control stability under strong transient disturbances.

[0030] In this embodiment, when constructing the observation space, action space, and reward function, the observation space s t Including the vehicle's vertical dynamic state Actuator working status and pavement excitation strength characterization quantity Among them, the vehicle's vertical dynamic state includes vehicle body acceleration. Vehicle speed Unsprung mass acceleration Suspension dynamic deflection z s - z u Relative speed with suspension The actuator's operating state includes the hydraulic pressure difference Δ between the two ends of the piston rod. p Motor speed n motor Motor torque T motor and actuator power P sys The road surface excitation intensity characterization quantity includes the vehicle body acceleration envelope. Unsprung mass acceleration envelope and suspension dynamic deflection envelope Thus, the observation space is constructed as follows: ].

[0031] Action space a t This includes the underlying drive signals of the chassis active actuator, which include the compression valve current. I c , Restoration valve current I r and motor torque command T rq The motion space is: .

[0032] reward function R ( s t , a t This includes performance bonus items. r t ( s t , a t ) and soft constraint penalty items P The expressions are as follows:

[0033]

[0034] In the formula, c i The positive weighting coefficient is determined by the weighting transition function of the reward function. A i The normalization coefficient is... S max This is for suspension dynamic deflection constraints. In this embodiment, the normalization coefficients are taken as follows: A 1 = 15 A 2 = 150 A 3 = 0.06 A 4=4500; The reference weight for the reward function under low-amplitude random road surface incentives is: [ c 1,ran , c 2,ran , c 3,ran , c 4,ran ]=[10, 1.5, 0.5, 0.4]; The reference weights for the reward function under high-amplitude transient impact road surface excitation are: [ c 1,imp , c 2,imp , c 3,imp , c 4,imp = [10, 0.2, 0.2, 0.1].

[0035] Then, based on the constructed observation space, action space, and reward function, an end-to-end reinforcement learning controller is trained, enabling the end-to-end reinforcement learning controller to directly map the vehicle's vertical dynamic state and actuator operating state to the chassis's active actuator drive signals. The control framework diagram is as follows: Figure 3 As shown.

[0036] This embodiment employs a teacher policy guidance mechanism during the training of the end-to-end reinforcement learning controller. The teacher policy is pre-trained under random road surface excitation conditions, and the network parameters of the teacher policy are frozen during the training of the student policy. The teacher policy and the student policy have the same observation space and action space. The student policy is trained under a composite road surface condition that includes random road surface excitation and transient impact road surface excitation. During the training of the student policy, the teacher policy actions and student policy actions are evaluated based on the value function. The guidance strength of the teacher policy on the student policy is determined according to the evaluation results. At the same time, a distillation loss term is constructed based on the difference between the teacher policy actions and the student policy actions, and the distillation loss term is used together with the reinforcement learning policy loss to update the network parameters of the student policy.

[0037] Among them, the evaluation of teacher and student strategic actions based on the value function includes: in the first... k Each sampling time, based on the current state s k Teacher strategy actions were obtained separately. and student candidate actions The teacher's strategic actions and the student's candidate actions are represented as follows:

[0038] in, To freeze teacher policies, This represents the deterministic output of the student's strategy. The corresponding value evaluation values ​​for both strategies and the value difference between the teacher's strategy action and the student's candidate action are calculated using a value function, expressed as follows:

[0039]

[0040] When Δ Q ( k )> or q When Δ is used, the teacher's strategic action is adopted as the action to be executed in the current training step; when Δ Q ( k )≤ or q At that time, the student's candidate action is used as the action to be executed in the current training step. or q A margin is used to determine the value advantage, which is used to suppress frequent switching caused by fluctuations in the value function. In this embodiment, or q Take 0.02.

[0041] The teacher strategy guidance mechanism includes a teacher intervention gating function, which is expressed as follows:

[0042] in, This is the current training round number. N warm In this embodiment, the number of warm-up rounds for teacher strategy intervention is set to 217; when When = 0, teacher-guided strategies are disabled, and only student strategies are used for interactive training. During the update phase, standard reinforcement learning loss is used to update the policy network and value network; when... When =1, teacher-guided action selection is enabled during the interaction phase, and the student policy network is jointly updated during the update phase using teacher-guided target action construction and distillation loss.

[0043] The intensity of teacher-led strategies is adjusted using a progressive activation coefficient, which is expressed as:

[0044] in, N ramp In this embodiment, the number of rounds for progressive activation guided by teacher strategies is set to 20. β ramp ( e This is used to gradually enhance the guiding role of teacher strategies on student strategies after the warm-up phase.

[0045] The distillation loss term is constructed based on the deviation between the teacher's and student's policy actions; for a batch of training samples sampled from the experience replay buffer, the term is denoted as the first... i The state of each sample is s i Based on the value difference Δ between teacher's strategic actions and student's strategic actions Q i Constructing teacher advantage weights:

[0046] in, w i For the first i The teacher advantage weights for each sample, Δ Q i Let Δ be the value difference between the teacher's strategy action and the student's strategy action in the i-th sample. Q max The value difference normalization coefficient is set to 0.1 in this embodiment, and clip(·) is the clipping function; the teacher activation ratio in the current batch of samples is expressed as:

[0047] in, N bIn this embodiment, the number of training samples is set to 256. r t This indicates the proportion of samples in the current batch where the teacher's strategy has an effective value advantage; The distillation loss term is expressed as:

[0048] in, m ϕ ( s i ) for student strategies in state s i The deterministic action of the output, m ϕT ( s i (To freeze teacher strategies in state) s i The deterministic action of the output, e To prevent positive numbers with a denominator of zero.

[0049] After the teacher-intervention gating function is enabled, the update objective of the student policy network is composed of the reinforcement learning policy loss and the teacher-guided distillation loss, expressed as:

[0050] in, To enhance the learning strategy loss, L dis To guide teachers on distillation losses, l dis ( e ) represents the distillation loss coefficient.

[0051] Distillation loss coefficient l dis ( e The loss is adaptively adjusted based on the moving average magnitude of the reinforcement learning policy loss and the distillation loss, as expressed as:

[0052] in, l 0 is the basic distillation coefficient; in this embodiment, it is set to 1. EMA (·) represents the exponential moving average operator. l min and l max These are the lower and upper limits of the distillation loss coefficient, respectively. In this embodiment, they are set to 1 and 2000, respectively. e To prevent positive numbers with a denominator of zero.

[0053] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0054] In this embodiment, the vehicle parameters are set as follows: sprung mass m s = 604 kg, unsprung mass m u =68 kg, suspension stiffness k s = 55500 N / m, tire stiffness k t =300000 N / m. During simulation and control implementation, the control force output update frequency of the active suspension controller was set to 100 Hz. The numerical simulation frequency of the vehicle dynamics model was set to 1000 Hz.

[0055] In this embodiment, the simulation conditions used to verify the effectiveness of the control strategy are set as follows: the vehicle speed is 10 m / s, and the road excitation includes two stages. The first stage is a Class C random road excitation, used to characterize continuous random road disturbances; the second stage is a composite road excitation formed by superimposing the Class C random road surface with a half-sine pulse input, used to characterize transient impact disturbances under the background of random road surface. The amplitude of the half-sine pulse input is set to 0.05 m, and includes two types of impact forms: bumps and dents, such as... Figure 4 As shown, the road excitation input includes C-level random road excitation and composite road excitation formed by superimposing C-level random road and half-sine pulse input.

[0056] To verify the control effect of the method of the present invention, the active suspension end-to-end control method proposed in this invention is compared with a quarter-vehicle passive suspension, an ideal force source model predictive control method, and an upper-level model predictive control and lower-level PID force tracking control method, denoted as SDA-SAC, Passive, MPC, and MPC-PID, respectively. Among them, the ideal force source model predictive control method MPC does not consider the dynamic model of the lower-level controller and hydraulic actuator, and directly applies the ideal active control force output by the model predictive controller to the vehicle's vertical dynamic model as a reference for the theoretical performance upper limit.

[0057] To clearly illustrate the full-time variation of the vehicle's vertical acceleration under different control methods in the same figure, Figure 5 Different constant longitudinal offsets were applied to each response curve. The offset amount was determined based on the maximum absolute amplitude of the corresponding curve, and an additional 0.5 m / s was set between adjacent curves. 2 The display interval is adjusted to avoid overlap between different response curves. This vertical offset is only used for layered display of curves and does not change the waveform shape, peak occurrence time, or variation pattern of the original response. Figure 6 Then they correspond to Figure 5The response data for regions (I) to (III) are used to compare the acceleration amplitudes of different control methods in the same real coordinate system. Figure 6 The partial view shown has removed the aforementioned longitudinal offset and redrawn each curve according to its original vehicle body vertical acceleration values. For example... Figure 5 and Figure 6 As shown, in the random road surface stage, the SDA-SAC method of this invention can effectively reduce vehicle body acceleration fluctuations, demonstrating good vibration suppression capabilities. In the composite road surface stage, the combined effects of random excitation and transient impact cause the passive suspension to generate significant vibration peaks. The MPC-PID method is affected by the nonlinearity of the hydraulic actuator and force tracking lag, limiting its impact response suppression effect. In contrast, the method of this invention, through state-related reward function adjustment and teacher-guided training, can still maintain good vehicle body vibration suppression performance without relying on explicit road surface recognition and multi-mode switching.

[0058] like Figure 7 As shown, the actuator output action results indicate that the method of the present invention directly outputs the actuator's underlying drive signals such as the compression valve current, the restoration valve current, and the motor torque, enabling the controller to perceive the actuator's response lag, output saturation, and energy consumption characteristics during the training process. Therefore, its actuator action changes more smoothly, can maintain a low control intensity in the random road surface stage, and appropriately increase the control action when impact disturbances occur, so as to achieve an adaptive response to the road excitation intensity.

[0059] like Figure 8 As shown ( Figure 8 The following are given: switching signal, value difference between teacher's policy action and student's candidate action, action value of teacher's policy and student's policy (the action evaluation results of teacher's policy and student's policy are given respectively). The action evaluation results of teacher's policy and student's policy show that, within the time interval where random disturbance and transient impact are combined, Δ is satisfied most of the time. Q (k)≤0, meaning the student's strategy action has a value evaluation value no lower than the teacher's strategy action, and the controller mainly uses the student's strategy action as the execution action. When some transient disturbances or state changes occur, the value evaluation value of the teacher's strategy action is higher than that of the student's strategy action, and the teacher's strategy intervenes intermittently to provide action guidance. This result indicates that the teacher's strategy guidance mechanism does not continuously replace the student's strategy, but rather provides auxiliary correction in local state regions where the teacher's strategy has an effective value advantage. This improves the stability and adaptability of the controller under random-impact composite road conditions while maintaining the autonomous control capability of the student's strategy.

[0060] In summary, the active suspension end-to-end control method for intelligent chassis of motor vehicles proposed in this invention can simultaneously consider the real dynamic characteristics of actuators and the adaptability to complex road surfaces, realize the direct mapping of vehicle state information to the underlying drive signals of actuators, eliminate the intermediate force command tracking link in traditional hierarchical control, and achieve adaptive control of random road surfaces, transient impact road surfaces and composite road surface conditions through state-related reward functions, thereby improving vehicle ride comfort, tire contact performance and the robustness and real-time performance of the intelligent chassis control system.

Claims

1. An end-to-end control method for active suspension of a smart chassis for motor vehicles, characterized in that, Includes the following steps: S1. Construct a nonlinear quarter-vehicle vertical dynamics model to characterize the vehicle's vertical motion, suspension relative motion, and tire contact state under road surface excitation. S2. Construct a high-fidelity dynamic model of the chassis active actuator that includes multi-physical coupling dynamic characteristics to characterize the nonlinear mapping relationship between the actuator drive signal and the output force. S3. A state-related reward function adjustment mechanism is constructed based on the vertical dynamic state of the vehicle to adaptively weight the vehicle body acceleration, suspension dynamic deflection, unsprung mass response and tire contact state related indicators. S4. Based on the vehicle vertical dynamics model, the high-fidelity dynamics model of the chassis active actuator, and the state-related reward function adjustment mechanism, construct the observation space, action space, and reward function. Train the end-to-end adaptive controller through reinforcement learning, input the current vehicle state information into the end-to-end adaptive controller, and directly map and output the chassis active actuator drive signal to control the working state of the chassis active actuator.

2. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 1, characterized in that, S1 includes the following process: S11. Determine the suspension dynamic deflection based on the relative displacement between the sprung and unsprung masses of the vehicle, and construct the suspension equivalent stiffness based on the suspension dynamic deflection. The suspension equivalent stiffness includes the stiffness of the suspension elastic elements, the nonlinear stiffness of the upper stop block, and the nonlinear stiffness of the lower stop block. The suspension equivalent stiffness is expressed as: in, k s,eff This is the equivalent stiffness of the suspension. k s For the stiffness of the suspension elastic element, and These are the nonlinear stiffnesses of the upper stop block and the lower stop block, respectively. and These are the clearances corresponding to the upper and lower stop blocks, respectively. z s and z u These represent the vertical displacements of the sprung mass and the unsprung mass, respectively. S12. Determine the tire dynamic deflection based on the relative displacement between the unsprung mass and the road surface excitation, and determine the contact state between the tire and the road surface based on the tire dynamic deflection and the tire static compression, thereby calculating the tire force, which is expressed as: in, F t For the force exerted by the tire, k t For tire ground stiffness, z g For road surface displacement input, ρ 0 This refers to the static compression of the tire. m s For the sprung mass, m u For unsprung mass, It is the acceleration due to gravity; S13. Based on the equivalent stiffness of the suspension, suspension damping, tire force, and output force of the active actuator, establish the vertical dynamic equations for the sprung and unsprung masses: in, m s and m u They are spring-loaded mass and unspring-loaded mass, respectively. Let be the vertical acceleration of the sprung mass. The vertical acceleration of the unsprung mass. F a The output force of the active actuator is the total control force acting on the active suspension system.

3. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 1, characterized in that, S2 includes the following process: S21. Construct a dynamic model of a hydraulic active actuator. The dynamic model of the hydraulic active actuator includes one or more of a motor pump unit, an electro-hydraulic valve, a hydraulic chamber, and an actuator piston assembly. It is used to characterize the nonlinear dynamic relationship between the actuator drive signal and the actual active control force. S22. Use one or more of the compression valve current, the reset valve current and the motor torque command as the actuator drive signal, and obtain one or more of the hydraulic pressure difference, motor speed, motor torque, valve current and system power as the actuator working state. S23. Based on the actuator drive signal and actuator working state, calculate the actual active control force acting on the active suspension system. The actual active control force is input to the vehicle vertical dynamics model to form a closed-loop coupling between the vehicle dynamics model and the actuator dynamics model.

4. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 1, characterized in that, S3 includes the following process: S31. Obtain the unsprung mass acceleration signal, and construct a pavement excitation intensity characterization quantity based on the unsprung mass acceleration signal. The pavement excitation intensity characterization quantity is the unsprung mass acceleration envelope, which is expressed as: in, a u ( k ) is the first k The unsprung mass acceleration at each sampling time, a env ( k ) is the first k The envelope of unsprung mass acceleration at each sampling time α Let be the envelope attenuation coefficient, and 0 < α <1; S32. Determine the excitation intensity threshold based on the unsprung mass response characteristics of low-amplitude random road excitation and high-amplitude transient impact road excitation. a th The excitation intensity threshold is used to distinguish between low-intensity continuous random excitation and high-intensity transient impact excitation; wherein, the low-amplitude random road surface excitation and the high-amplitude transient impact road surface excitation can be used individually as training road surface conditions or combined to form composite road surface conditions. S33. Obtain the reference weights of the reward function under low-amplitude random road surface excitation through pre-simulation or offline calibration. γ i,ran Reference weights for the reward function under high-amplitude transient impact road surface excitation γ i,imp ,in, γ i,ran and γ i,imp Used to characterize the first i The optimization priority of each performance index under low-amplitude random road conditions and high-amplitude transient impact conditions; S34. Based on the unsprung mass acceleration envelope a env ( k ), excitation intensity threshold a th Reference weights for the reward function under low-amplitude random road surface incentives γ i,ran and the reference weight of the reward function under high-amplitude transient impact road surface excitation. γ i,imp Construct state-dependent reward functions and weighted transition functions: in, γ i ( k ) is the first k The sampling time of the first sampling moment i Adaptive weights for each reward item p It is a transition index, and p >0, when a env ( k )≤ a th At that time, the weight of the reward function transitions continuously with the intensity of the road surface incentive; when a env ( k )> a th At that time, the reward function weights are maintained at the reference weights corresponding to high-amplitude transient shock conditions to improve control stability under strong transient disturbances.

5. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 1, characterized in that, S4 includes the following process: S41. Based on the dynamic response of the vehicle vertical dynamics model and the chassis active actuator high-fidelity dynamics model, construct the observed state variables in the observation space; The actuator drive signal of the high-fidelity dynamic model of the chassis active actuator is used as the motion space; Based on the state-dependent reward function adjustment mechanism, a reward function is designed, including a performance reward term and a soft constraint penalty term; S42. Using a teacher strategy guidance mechanism, combined with the observation space, action space and reward function, the end-to-end reinforcement learning controller is trained so that the end-to-end reinforcement learning controller can directly map the vehicle's vertical dynamic state and actuator working state into the chassis active actuator drive signal. S43. Input the current vehicle status information into the trained end-to-end adaptive controller, and output the chassis active actuator drive signal to control the working state of the chassis active actuator.

6. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 5, characterized in that, The specific process of S41 is as follows: Constructing the observation space s t for: ],in, This refers to the vehicle's vertical dynamic state. For the actuator's working state, The vertical dynamic state of the vehicle is a quantity characterizing the road surface excitation intensity. Including vehicle acceleration Vehicle speed Unsprung mass acceleration Suspension dynamic deflection z s - z u Relative speed with suspension The actuator working state Including the hydraulic pressure difference Δ at both ends of the piston rod p Motor speed n motor Motor torque T motor and actuator power P sys The road surface excitation intensity characterization quantity Including vehicle acceleration envelope Unsprung mass acceleration envelope and suspension dynamic deflection envelope ; Construct action space a t for: ,in, I c For compression valve current, I r To restore valve current, T rq This is the motor torque command; Design reward function R ( s t , a t )for: in, r t ( s t , a t ) is a performance bonus item. P This is a soft constraint penalty item. γ i=1，2、3，4 The positive weighting coefficients are... A i=1，2、3，4 The normalization coefficient is... S max For suspension dynamic deflection constraints, For the unsprung mass acceleration envelope, This refers to the electrode power.

7. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 6, characterized in that, The process of training the end-to-end reinforcement learning controller in S42 is as follows: The teacher policy is pre-trained under random road surface excitation conditions, and the network parameters of the teacher policy are frozen during the training of the student policy. The teacher policy and the student policy have the same observation space and action space. Student policies are trained under composite road conditions including random road surface excitation and transient impact road surface excitation. During the training process, the teacher's policy actions and student's policy actions are evaluated based on the value function. The guidance strength of the teacher's policy on the student's policy is determined based on the evaluation results, so as to guide the student's policy using the teacher's policy guidance mechanism. At the same time, a distillation loss term is constructed based on the difference between the teacher's policy actions and the student's policy actions, and the distillation loss term is used together with the reinforcement learning policy loss for updating the student policy network parameters.

8. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 7, characterized in that, The process of evaluating teacher and student strategic actions based on value functions includes: In the k Each sampling time, based on the current state s k Teacher strategy actions were obtained separately. and student candidate actions The teacher's strategic action and the student's candidate action are respectively represented as follows: in, To freeze teacher policies, For deterministic output of student strategies; Then, the teacher's strategic actions are calculated using a value function. and student candidate actions Corresponding value assessment values, and teacher strategies and actions Compared to student candidate actions Value difference: in, Teacher strategy actions The corresponding value assessment value, Candidate actions for students The corresponding value assessment value, Teacher strategy actions Compared to student candidate actions The value difference, when Δ Q ( k )> η q When Δ is used, the teacher's strategic action is adopted as the action to be executed in the current training step; when Δ Q ( k )≤ η q When this happens, the student's candidate action is used as the action to be executed in the current training step. η q It determines the margin of value advantage and is used to suppress frequent switching caused by fluctuations in the value function.

9. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 8, characterized in that, The teacher strategy guidance mechanism includes a teacher intervention gating function and a progressive activation coefficient. The teacher intervention gating function determines whether teacher strategy guidance is turned on or off, and the progressive activation coefficient adjusts the intensity of teacher strategy guidance. The teacher intervention gating function is as follows: in, This is the current training round number. N warm Number of warm-up rounds for teacher strategy intervention; when When = 0, teacher-guided strategies are disabled, and only student strategies are used for interactive training. During the update phase, standard reinforcement learning loss is used to update the policy network and value network; when... When =1, teacher strategy guidance action selection is enabled in the interaction phase, and the student strategy network is jointly updated in the update phase using teacher-guided target action construction and distillation loss. The progressive activation coefficient is: in, N ramp The number of rounds for progressive activation guided by teacher strategies. β ramp ( e This is used to gradually increase the guiding role of teacher strategies on student strategies after the warm-up phase. clip (·) is the amplitude limiting function.

10. The active suspension end-to-end control method for intelligent chassis of motor vehicles according to claim 9, characterized in that, The distillation loss specifically refers to: Among them, for the batch training samples obtained from the experience replay buffer, N b For the number of training samples in a batch, the first... i The state of each sample is s i , μ ϕ ( s i ) for student strategies in state s i The deterministic action of the output, μ ϕT ( s i (To freeze teacher strategies in state) s i The deterministic action of the output, ε To prevent positive numbers with a denominator of zero, For the first i The teacher advantage weights for each sample, Δ Q i For the first i The value difference Δ between teacher's strategic actions and student's strategic actions in each sample Q max This is the normalization coefficient for the value difference; When teacher-guided strategies are enabled, the update objective of the student's strategy network is jointly constituted by the distillation loss term and the reinforcement learning strategy loss: in, To enhance the learning strategy loss, λ dis ( e ) represents the distillation loss coefficient. ρ t This represents the proportion of the current batch of samples where the teacher's strategy has an effective value advantage. λ 0 is the basic distillation coefficient. EMA (·) represents the exponential moving average operator. λ min and λ max These are the lower and upper limits of the distillation loss coefficient, respectively.

Images