Nonlinear active suspension vertical preview control method based on TD3 intelligent agent

Abstract

The invention discloses a nonlinear active suspension vertical preview control method based on a TD3 intelligent agent. The nonlinear active suspension vertical preview control method comprises the steps that 1, a two-degree-of-freedom active suspension dynamic model considering suspension spring and damping nonlinear factors is established; 2, acquiring a height sequence of a front road surface through a vehicle-mounted sensor, and calculating a preview elevation difference; and 3, controlling the vertical vibration of the active suspension through a deep reinforcement learning TD3 algorithm, and designing a state space, an action space, a reward function and a network structure of a TD3 intelligent agent, thereby outputting reasonable actions through the active suspension to achieve a control effect. According to the method, the control stability and smoothness of the suspension under complex road excitation can be remarkably improved, and the method has excellent working condition adaptability and is suitable for fine control over an intelligent vehicle active suspension system.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle suspension control, and particularly relates to a vertical control method for a non-linear active suspension based on deep reinforcement learning, which combines preview information perception and TD3 algorithm for real-time optimal control of vehicle vertical vibration. Background Art

[0002] As the core control component of vehicle vertical vibration, the performance of the suspension system directly determines the ride comfort and handling stability. The active suspension can effectively suppress the road excitation disturbance by adjusting the actuating force in real time, but its control efficiency is limited by two major bottlenecks in strategy design: First, the modeling dependence defect of traditional control methods - classical algorithms based on linear assumptions (such as PID, LQR, MPC) are difficult to accurately characterize the dynamic characteristics such as the non-linearity of the suspension spring stiffness and the damping hysteresis effect, resulting in a sharp degradation of control accuracy under complex road conditions; Second, the decision-making limitation without a preview mechanism - existing control strategies do not integrate the time-series information of the road surface ahead and cannot perceive the sudden change characteristics of the road surface elevation in advance, resulting in the action output can only passively compensate for the current disturbance, and the control time delay effect is aggravated when controlling in unstructured terrain (such as continuously undulating road surface), and the multi-objective optimization ability is limited and the control fineness decreases. Summary of the Invention

[0003] In order to overcome the deficiencies of the prior art, the present invention provides a vertical preview control method for a non-linear active suspension based on a TD3 agent, aiming to improve the handling stability and ride comfort of the suspension under complex road surface excitations and non-linear factors.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A vertical preview control method for a non-linear active suspension based on a TD3 agent of the present invention is characterized in that it is applied to an active suspension vehicle and is carried out according to the following steps:

[0006] Step 1: Establish a two-degree-of-freedom active suspension dynamic model considering the non-linear factors of the suspension spring and damping;

[0007] Step 2: Obtain the preview height sequence of the road surface ahead through on-vehicle sensors and use it to calculate the preview elevation difference;

[0008] Step 3: Construct the state space, action space, reward function and Actor-Critic network of the TD3 agent;

[0009] Step 4: Train the Actor-Critic network of the TD3 agent on a random road surface until the reward converges to a high reward level, so as to obtain the optimal agent for off-line control;

[0010] Step 5: Input the real-time vehicle state and preview elevation difference into the trained optimal agent, output the optimal active control force, and drive the suspension actuator to perform vertical vibration suppression.

[0011] The characteristics of a vertical preview control method for a nonlinear active suspension based on a TD3 agent according to the present invention also lie in that in the step 1, a two-degree-of-freedom nonlinear active suspension dynamic model is established according to Equation (1):

[0012] (1)

[0013] In Equation (1), and are the sprung mass and unsprung mass respectively, and are the accelerations of the sprung mass and unsprung mass respectively, and and and are the suspension spring force, suspension damping force, tire elastic force, and active force respectively, and there are:

[0014] (2)

[0015] In Equation (2), is the spring stiffness, is the damping coefficient, is the tire stiffness, is the road surface excitation input, and are the displacements of the sprung mass and unsprung mass respectively, and are the velocities of the sprung mass and unsprung mass respectively, and and are 3 nonlinear fitting coefficients.

[0016] Furthermore, in the step 2, the on-vehicle sensor is used to obtain the high-order sequence of pre-preview points on the front road surface at moments, let the distance between adjacent pre-preview points be , , is the vehicle longitudinal speed, is the control step size, and the filtered elevation difference is calculated using Equation (3):

[0017] (3)

[0018] In Equation (3), represents the The elevation of a preview point denotes the elevation of the th preview point on the road surface ahead at time is the weight coefficient of the elevation of the th preview point, and satisfies and .

[0019] Furthermore, step 3 includes:

[0020] Step 3.1: Select the state space of the TD3 agent , where T represents matrix transpose;

[0021] Step 3.2: Select the action space of the TD3 agent ;

[0022] Step 3.3: Calculate and construct the total reward function using Equation (4) :

[0023] (4)

[0024] In Equation (4), R1 is the penalty term for the dynamic stroke safety constraint, R2 is the penalty term for the dynamic tire load, R3 is the penalty term for the acceleration above the spring; R4 is the penalty term for the body displacement, R5 is the optimization term for the active force energy consumption, and there are:

[0025] (5)

[0026] In Equation (5), is the limit value of the suspension dynamic stroke, and are two dynamic stroke proportionality coefficients, and satisfy , and are two dynamic stroke reward and penalty coefficients, and satisfy , represents the training termination;

[0027] (6)

[0028] In Equation (6), is the static-dynamic load ratio, and , is the gravitational acceleration, and are two dynamic load reward and penalty coefficients, and satisfy , is the dynamic load proportionality coefficient, and satisfies ;

[0029] (7)

[0030] In Equation (7), is the reward and punishment coefficient of the acceleration above the spring, and ;

[0031] (8)

[0032] In Equation (8), is the reward and punishment coefficient of the displacement above the spring, and ;

[0033] (9)

[0034] In Equation (9), and are two reward and punishment coefficients of the active force power consumption, and and are both positive numbers, is the energy consumption optimization force threshold, and , is the maximum active force limit, is the active force proportionality coefficient, and ;

[0035] Step 3.4: Construct an Actor-Critic network, including: Critic1 network, Critic2 network with the same structure, target Critic1 network, target Critic2 network, Actor network with the same structure and target Actor network;

[0036] The Critic1 network, Critic2 network, target Critic1 network, and target Critic2 network all include a first fully connected layer, a feature splicing layer, a second fully connected layer, a third fully connected layer, and an output layer connected in sequence; wherein, the input of the feature splicing layer is the output of the first fully connected layer and the action, and the output layer generates the value of the state-action pair;

[0037] The Actor network and the target Actor both include a first fully connected layer, a second fully connected layer, and an output layer connected in sequence, and the output layer generates the active suspension acting force.

[0038] Furthermore, the said Step 4 includes:

[0039] Step 4.1: Randomly initialize the parameters of the Critic1 network , the parameters of the Critic2 network and the parameters of the Actor network ; and assign the parameters of the Critic1 network to the parameters ​, assign the parameters of the Critic2 network to the parameters of the target Critic2 network , assign the parameters of the Actor network to the parameters of the target Actor network ;

[0040] Step 4.2: Input the state at time into the Actor network, output the action at time and apply it to the two-degree-of-freedom nonlinear active suspension dynamics model to obtain the state at time , and calculate the total reward at by , to obtain a sample data at time , and store it in the experience pool;

[0041] Step 4.3: Randomly draw a sample data from the experience pool, and input , into the Critic1 network and the Critic2 network to obtain the first value of the state-action pair at time, the second value ;

[0042] Input into the target Actor network for processing, and output the action at time ;

[0043] Input , into the target Critic1 network and the target Critic2 network to obtain the first target value of the state-action pair at time, the second target value ; Then calculate the target value at time according to Equation (10):

[0044] (10)(10)

[0045] In Equation (10), is the discount factor;

[0046] Step 4.4: Compare with and Calculate the mean square error to construct the Loss function, and then use the gradient descent method to optimize the parameters of the Critic1 network and the parameters of the Critic2 network for updating, obtain the updated parameters of the Critic1 network and assign them to and the updated parameters of the Critic2 network and assign them to ;

[0047] Step 4.5: According to , use the policy gradient ascent method to update the parameters of the Actor network for updating, obtain the updated parameters of the Actor network and assign them to ;

[0048] Step 4.6: Use the soft update method shown in Equation (11) to update the parameters of the target Critic1 network, target Critic2 network, and target Actor network:

[0049] (11)

[0050] In Equation (11), is the smoothing factor, is the assignment;

[0051] Step 4.7: Iteratively train the Actor-Critic network of the TD3 agent according to the process of Step 4.2 - Step 4.6 until the reward reaches a stable high reward, so as to obtain the optimal agent for offline deployment.

[0052] An electronic device according to the present invention, comprising a memory and a processor, is characterized in that the memory stores a program for implementing the non-linear active suspension vertical preview control method, and the processor implements the active suspension vertical preview control when executing the program.

[0053] A computer-readable storage medium according to the present invention, storing a computer program, is characterized in that the computer program implements the steps of the non-linear active suspension vertical preview control method when executed by a processor.

[0054] Compared with the prior art, the beneficial effects of the present invention are reflected in:

[0055] 1. The present invention adopts the TD3 deep reinforcement learning framework, and the agent can autonomously perceive and learn the complex dynamic characteristics of the non-linear suspension system, overcoming the performance limitations of traditional linear control methods under modeling deviations and non-linear factors. Combining with the adaptive mapping ability of the deep neural network, it significantly improves the suppression accuracy of the vertical vibration of the suspension and realizes the multi-objective collaborative optimization of vehicle body acceleration, dynamic stroke, etc.

[0056] 2. The present invention innovatively integrates the pre-viewed elevation information of the front road surface into the state observation space of the TD3 agent, extracts the changing trend of the road surface elevation through weighted filtering, provides a feedforward control signal for the suspension system, and effectively weakens the interference of time delay on the control effect of the agent.

[0057] 3. By integrating the dynamic stroke safety constraint, the dynamic stability of the tire vertical load, and the active force hierarchical optimization mechanism, the present invention effectively reduces the vibration acceleration of the sprung mass to improve the ride comfort on the premise of ensuring the mechanical safety of the suspension, simultaneously suppresses the impact wear of the buffer block caused by the over-limit of the suspension dynamic stroke, enhances the tire ground contact stability under low adhesion road surfaces, and balances the active force output intensity and the actuator durability through an asymmetric regulation strategy, significantly improving the comprehensive control efficiency and service life of the suspension system under complex working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is the overall control strategy diagram of the present invention;

[0059] Figure 2 is the structural diagram of the Critic network of the TD3 agent of the present invention;

[0060] Figure 3 is the structural diagram of the Actor network of the TD3 agent of the present invention;

[0061] Figure 4 is the execution flow chart of the TD3 algorithm of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] In this embodiment, a non-linear active suspension vertical pre-view control method based on a TD3 agent is applied to an active suspension vehicle, and the overall control block diagram is as Figure 1 shown, and the following steps are carried out:

[0063] Step 1: Establish a two-degree-of-freedom non-linear active suspension dynamics model;

[0064] Step 1.1: Build a two-degree-of-freedom non-linear suspension model according to Equation (1):

[0065] (1)

[0066] In Equation (1), , are the sprung mass and the unsprung mass respectively, , are the accelerations of the sprung mass and the unsprung mass respectively, , , , are the suspension spring force, the suspension damping force, the tire elastic force, and the active force respectively.

[0067] Step 1.2: Model the non-linear force characteristics of the spring and shock absorber using a polynomial function, as shown in Equation (2):

[0068] (2)

[0069] In Equation (2), is the spring stiffness, is the damping coefficient, is the tire stiffness, is the road excitation input, and are the displacements of the sprung and unsprung masses respectively, and are the velocities of the sprung and unsprung masses respectively, and and are three non-linear fitting coefficients.

[0070] Step 2: Obtain the road preview elevation difference;

[0071] Step 2.1: Use on-vehicle sensors to obtain the high-order sequence of the road surface at moments for road preview points ahead, denoted as . Let the distance between adjacent preview points be , is the vehicle longitudinal speed, is the control step size;

[0072] Step 2.2: Calculate the preview elevation difference using weighted filtering as shown in Equation (3) :

[0073] (3)

[0074] In Equation (3), represents the elevation of the th preview point of the road surface ahead at moment, is the weight coefficient of the elevation of the th preview point, and satisfies and , which is used to strengthen the prediction weight of the proximal road surface change.

[0075] Step 3: Construct the state space, action space, reward function, and Actor-Critic network structure of the TD3 agent;

[0076] Step 3.1: Select the state space ;

[0077] Step 3.2: Select the action space ;

[0078] Step 3.3: Construct the reward function. The total reward function is calculated according to Equation (4):

[0079] (4)

[0080] In Equation (4): R1 is the penalty term for the dynamic stroke safety constraint, R2 is the penalty term for the dynamic tire load, R3 is the penalty term for the sprung mass acceleration; R4 is the penalty term for the body displacement, R5 is the optimization term for the active force energy consumption, and there are:

[0081] (5)

[0082] In Equation (5), is the limit value of the suspension dynamic stroke, and are two dynamic stroke proportionality coefficients, and satisfy , and are two dynamic stroke reward and penalty coefficients, and satisfy , represents the termination of this training. The dynamic stroke is used to characterize the relative displacement between the sprung mass and the unsprung mass, and it is hard constrained by the bump stop, that is . When the dynamic stroke exceeds the limit, the whole vehicle becomes a rigid body. At this time, the energy generated by the road surface excitation directly acts on the body structure and damages the vehicle. In practice, the safety threshold is often set to to reserve a safety margin. Therefore, in this example, is taken. At the same time, the active suspension needs to perform a certain range of height adjustment under certain working conditions to ensure that the body floats, in order to maximize the optimization goal. In this example, is taken. Therefore, a segmented reward mechanism is adopted to achieve the dynamic balance of multi-objective optimization while ensuring mechanical safety. In this example, is taken.

[0083] (6)

[0084] In Equation (6), is the ratio of the dynamic load to the static load, and , is the acceleration due to gravity, and are the dynamic load reward and penalty coefficients, and satisfy , is the dynamic load proportionality coefficient, and satisfies . characterizes the grounding state of the tire. The larger it is, the greater or smaller the vertical load between the tire and the ground is, which will have an adverse impact on the stable driving of the vehicle and the tire life. Therefore, it is necessary to limit this coefficient from being too large. In this example, , .

[0085] The acceleration and displacement above the spring being close to zero are the core objectives of suspension system optimization. Calculate the continuous reward according to Equation (7):

[0086] (7)

[0087] In Equation (7), is the acceleration reward and punishment coefficient above the spring, and ; is the displacement reward and punishment coefficient above the spring, and .

[0088] For an active suspension, the greater the output force, the higher the energy consumption of the actuator. Therefore, it is necessary to limit and optimize it. To encourage the agent to make full use of the active force for control optimization, an asymmetric reward function as shown in Equation (8) is adopted to balance the optimization of energy consumption and control performance.

[0089] (8)

[0090] In Equation (8), and are two active force power consumption reward and punishment coefficients, and and are both positive numbers, is the energy consumption optimization force threshold, and , is the maximum active force limit, is the active force proportionality coefficient, and . In this example, take , when , give a linear reward to encourage the TD3 agent to use the active force for control; when , punish the excessive use of the active force.

[0091] Step 3.4: Construct the Actor-Critic network structure of the TD3 agent to achieve efficient perception of the nonlinear active suspension state and policy optimization of the agent:

[0092] The structures of Critic1 network, Critic2 network, target Critic1 network, and target Critic2 network are the same, and the structures of the Actor network and the target Actor network are the same, for a total of 6 networks;

[0093] Any Critic network includes a first fully connected layer, a feature concatenation layer, a second fully connected layer, a third fully connected layer, and an output layer connected in sequence; among them, the first fully connected layer linearly maps the input state quantity into a multi-dimensional feature vector, the feature concatenation layer concatenates the multi-dimensional feature vector with the action input, the second and third fully connected layers perform non-linear transformation on the concatenated fusion features, and the output layer generates the value of the state-action pair. In this example, the structure of the Critic network is as Figure 2 shown;

[0094] The Actor network includes a first fully connected layer, a second fully connected layer, and an output layer connected in sequence. The output layer generates the active suspension actuating force. In this example, the structure of the Actor network is as Figure 3 shown.

[0095] Step 4: Converge the reward training of the TD3 agent to a high reward level on a random road surface to obtain the optimal agent for offline control. The execution process is as Figure 4 shown;

[0096] Step 4.1: Initialize the Critic1 and Critic2 networks with random network parameters , ; Initialize the target Critic1 and target Critic2 networks with = , = ; Initialize the Actor network with random network parameters ; Initialize the target Actor network with the parameter ;

[0097] Step 4.2: Input the state at time into the Actor network, output the action and act on the two-degree-of-freedom non-linear active suspension dynamic model to obtain the state at time , and calculate the total reward , at time to obtain a sample data and store it in the experience pool; ;

[0098] Step 4.3: Randomly extract a sample data from the experience pool, and input , into the Critic1 network and the Critic2 network to obtain the first value and the second value of the state-action pair;

[0099] Input it into the target Actor network for processing and output the action at time; ;

[0100] Input , into the target Critic1 network and the target Critic2 network to obtain the first target value , the second target value ; Then calculate the target value at time according to Equation (9):

[0101] (9)

[0102] In Equation (9), is the discount factor.

[0103] Step 4.4: Calculate the mean squared error between and and respectively to construct the Loss function, and then use the gradient descent method to update the parameters of the Critic1 network, the parameters of the Critic2 network, obtain the updated parameters of the Critic1 network and assign them to ;

[0104] Step 4.5: According to , use the policy gradient ascent method to update the parameters of the Actor network, obtain the updated parameters of the Actor network and assign them to

[0105] Step 4.6: Use the soft update method shown in Equation (10) to update the parameters of the target Critic1 network, the target Critic2 network, and the target Actor network:

[0106] (10)

[0107] In Equation (10), is the smoothing factor, is the assignment symbol;

[0108] Step 4.7: Iteratively train the Actor network and Critic network of the TD3 agent according to the process of Step 4.2 - Step 4.6 until the reward converges to a stable high - reward level, thereby obtaining the optimal agent for offline deployment.

[0109] Step 5: Input the real - time vehicle state and preview elevation difference into the trained optimal agent, so as to output the optimal active control force, drive the suspension actuator to perform vertical vibration control, and improve the suspension operability and ride comfort.

[0110] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above - mentioned method, and the processor is configured to execute the program stored in the memory.

[0111] In this embodiment, a computer - readable storage medium stores a computer program, and when the computer program is run by a processor, it executes the steps of the above - mentioned method.

Claims

1. A vertical preview control method for a non-linear active suspension based on a TD3 agent, characterized in that It is applied to an active suspension vehicle and proceeds as follows: Step 1: Establish a two-degree-of-freedom active suspension dynamics model considering the nonlinear factors of the suspension spring and damping; Step 2: Obtain the preview elevation sequence of the road surface ahead through on-vehicle sensors and use it to calculate the preview elevation difference; Step 3: Construct the state space, action space, reward function, and Actor-Critic network of the TD3 agent; Step 4: Train the Actor-Critic network of the TD3 agent on a random road surface until the reward converges to a high reward level, thereby obtaining an optimal agent for offline control; Step 5: Input the real-time vehicle state and preview elevation difference into the trained optimal agent, output the optimal active control force, and drive the suspension actuator to perform vertical vibration suppression.

2. The non-linear active suspension vertical preview control method based on TD3 agent according to claim 1, wherein, In Step 1, a two-degree-of-freedom nonlinear active suspension dynamics model is built according to Equation (1); (1) In Equation (1), and are the unsprung and sprung masses respectively, and are the accelerations of the unsprung and sprung masses respectively, and and and are the suspension spring force, suspension damping force, tire elastic force, and driving force respectively, and there is: (2) In Equation (2), is the spring stiffness, is the damping coefficient, is the tire stiffness, is the road excitation input, , are the displacements of the sprung and unsprung masses respectively, , are the velocities of the sprung and unsprung masses respectively, , , are three non - linear fitting coefficients.

3. A vertical preview control method for a non-linear active suspension based on a TD3 agent according to claim 1, characterized in that In the said step 2, the on-vehicle sensor is used to obtain the high-order sequence of a plurality of preview points on the road surface ahead at a moment, and the distance between adjacent preview points is set as , where \(v\) is the longitudinal vehicle speed, \(\Delta t\) is the control step, and the filtered elevation difference is calculated using Equation (3) : (3) In formula (3), represents the elevation of the th preview point on the road surface ahead at time represents the elevation of the th preview point on the road surface ahead at time is the weight coefficient of the elevation of the th preview point, and satisfies and .

4. A vertical preview control method for a non-linear active suspension based on a TD3 agent according to claim 1, characterized in that, Step 3 includes: Step 3.1: Select the state space of the TD3 agent , where T represents matrix transpose; Step 3.2: Select the action space of the TD3 agent ; Step 3.3: Calculate and construct the total reward function using Equation (4) :[[]]END]] (4) In Equation (4), R1 is the penalty term for the dynamic stroke safety constraint, R2 is the penalty term for the dynamic tire load, R3 is the penalty term for the acceleration above the spring; R4 is the penalty term for the body displacement, R5 is the optimization term for the active force energy consumption, and there is: (5) In formula (5), is the limit value of the suspension dynamic stroke, and are two dynamic stroke proportionality coefficients, and satisfy , and are two dynamic stroke reward and punishment coefficients, and satisfy , represents the termination of training; (6) In formula (6), is the static-dynamic load ratio, and , is the acceleration due to gravity, and are two dynamic load reward and punishment coefficients, and satisfy , is the dynamic load proportionality coefficient, and satisfies ; (7) In formula (7), is the penalty and reward coefficient of the acceleration above the spring, and ; (8) In formula (8), is the reward and punishment coefficient of the displacement above the spring, and ; (9) In formula (9), and are two active power consumption reward and punishment coefficients, and and are both positive numbers, is the energy consumption optimization force threshold, and , is the maximum active force limit, is the active force proportionality coefficient, and ; Step 3.4: Construct the Actor-Critic network, including: Critic1 network, Critic2 network with the same structure, target Critic1 network, target Critic2 network with the same structure, Actor network with the same structure, and target Actor network; The Critic1 network, Critic2 network, target Critic1 network, and target Critic2 network all include a first fully connected layer, a feature splicing layer, a second fully connected layer, a third fully connected layer, and an output layer connected in sequence; among them, the input of the feature splicing layer is the output of the first fully connected layer and the action, and the output layer generates the value of the state-action pair; The Actor network and the target Actor both include a first fully connected layer, a second fully connected layer, and an output layer connected in sequence, and the output layer generates the active suspension acting force.

5. A vertical preview control method for a non-linear active suspension based on a TD3 agent according to claim 1, characterized in that, Step 4 includes: Step 4.1: Randomly initialize the parameters of the Critic1 network , the parameters of the Critic2 network and the parameters of the Actor network ; and assign the parameters of the Critic1 network to the parameters of the target Critic1 network , assign the parameters of the Critic2 network to the parameters of the target Critic2 network , assign the parameters of the Actor network to the parameters of the target Actor network ; Step 4.2: Input the state at time into the Actor network, and output the action at time which acts on the two-degree-of-freedom non-linear active suspension dynamics model to obtain the state at time , and calculate the total reward at , to obtain the total reward at time , thereby obtaining a sample data and storing it in the experience pool; Step 4.3: Randomly draw a sample data from the experience pool , and input , into Critic1 network and Critic2 network to obtain the first value of the state-action pair at , the second value ; Input it into the target Actor network for processing and output the action at time ; Input , into the target Critic1 network and the target Critic2 network to obtain the first target value of the state-action pair at time , the second target value ; and thus calculate according to Equation (10) the target value at time : (10) In Equation (10), is the discount factor; Step 4.4: Take the respectively and and to calculate the mean square error to construct a Loss function, and then use the gradient descent method to update the parameters of the Critic1 network and the parameters of the Critic2 network, obtaining the updated parameters of the Critic1 network and assigning them to and the updated parameters of the Critic2 network and assigning them to ; Step 4.5: According to , use the policy gradient ascent method to update the parameters of the Actor network, obtain the updated parameters of the Actor network and assign them to ; Step 4.6: Update the parameters of the target Critic1 network, target Critic2 network, and target Actor network using the soft update method shown in Equation (11); (11) In formula (11), is a smoothing factor, is an assignment; Step 4.7: Iteratively train the Actor-Critic network of the TD3 agent according to the process of Step 4.2 - Step 4.6 until the reward reaches a stable high reward, thereby obtaining an optimal agent for offline deployment.

6. An electronic device, comprising a memory and a processor, characterized in that, The memory stores a program for implementing the nonlinear active suspension vertical preview control method described in any one of claims 1 - 5, and when the processor executes the program, it realizes the vertical preview control of the active suspension.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it realizes the steps of the nonlinear active suspension vertical preview control method described in any one of claims 1 - 5.

Images