A damping force closed-loop control method, device, medium and product

Abstract

The application discloses a damping force closed-loop control method, device, medium and product, relates to the technical field of suspension control, and the method comprises the steps of constructing a data-driven model; obtaining a simulation estimated damping force based on characteristic data by using the data-driven model; obtaining a simulation expected damping force based on the characteristic data and a simulation expected force by using boundary constraints; obtaining a simulation control current based on the simulation expected damping force, the simulation estimated damping force and the characteristic data by using a deep deterministic policy gradient network; determining whether the characteristic data fed back by a simulation semi-active suspension system meets a set requirement, and then obtaining a force tracking controller; integrating the data-driven model, the boundary constraints and the force tracking controller into a force tracking module; obtaining a control current based on original characteristic data and an expected force by using the force tracking module; and the control current is used for controlling the semi-active suspension system. The application can improve the force tracking precision of the semi-active suspension system.

Description

Technical Field

[0001] This application relates to the field of suspension control technology, and in particular to a damping force closed-loop control method, device, medium and product. Background Technology

[0002] Force tracking control is one of the core conditions for fully realizing the performance of a semi-active suspension system. The ideal damping force calculated by the suspension controller must be accurately realized through the underlying actuator. Existing force tracking controllers face the dual challenges of force tracking accuracy and dynamic boundary characteristic constraints of damping force under complex operating conditions. Traditional force feedback modules use simplified or low-order models, and since the force feedback module is directly embedded in the force tracking controller, model errors are introduced into the controller. Furthermore, the damping force of a semi-active suspension system is limited by the suspension state, which is constrained by unknown road surface excitations, thus limiting the effectiveness of traditional force tracking strategies.

[0003] Existing semi-active suspension systems mostly rely on PID (Proportional-Integral-Derivative) or sliding mode control (SMC) to dynamically adjust the control output. For example, they use accelerometers and displacement sensors to obtain suspension motion data, back-calculate the actual damping force using a low-order damper model, and then use PID or SMC to adjust the control output, reducing the tracking error of the damping force and thus improving suspension performance. Alternatively, as mentioned in one study, the dynamic characteristics of the damping force are characterized by an RC (Resistor-Capacitor) operator, using the force error as the optimization equation, and then outputting the control current through a quadratic optimization method. However, this method does not adequately characterize the dynamic characteristics of the damper, resulting in limited tracking performance. Furthermore, some researchers have proposed embedding the dynamic characteristics of damping force directly into the controller using a Takagi-Sugeno (TS) fuzzy Smith predictive time-delay compensation control strategy, evaluating the tracking effect based on the improvement in sprung acceleration. This strategy directly introduces the uncertainty of the actuator model into the suspension control, causing model errors that affect the control effect. Semi-active suspension systems exhibit strong nonlinear time-varying characteristics, which traditional mechanistic models cannot accurately characterize. Therefore, force tracking strategies using mechanistic models as feedback have limited accuracy and may even worsen the force tracking effect. In addition, as mentioned earlier, damping force and suspension state are coupled, and suspension state is constrained by unknown road surface excitation. This means that damping force is limited by unknown road surface excitation. Traditional force tracking strategies cannot meet the force tracking accuracy requirements under complex instantaneous impact and other dynamic environments, resulting in limited improvement in suspension performance in complex dynamic environments. Summary of the Invention

[0004] The purpose of this application is to provide a damping force closed-loop control method, device, medium, and product that can improve the force tracking accuracy of a semi-active suspension system and enhance the performance of the semi-active suspension system under complex dynamic environments.

[0005] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a closed-loop control method for damping force, including: A data-driven model is built based on convolutional neural networks and long short-term memory neural networks; Obtain characteristic data of the simulated semi-active suspension system; obtain the desired force in the simulation. A data-driven model is used to obtain the simulated estimated damping force based on feature data; boundary constraints are used to obtain the simulated expected damping force based on feature data and simulated expected force. A deep deterministic strategy gradient network is used to obtain the simulation control current based on the simulated expected damping force, the simulated estimated damping force, and characteristic data; the simulation control current is used to control the simulated semi-active suspension system. Acquire feature data from the simulated semi-active suspension system; determine whether the feature data from the simulated semi-active suspension system meets the set requirements; if it does not meet the set requirements, update the parameters of the deep deterministic policy gradient network based on the feedback feature data, and return to execute the step of using a data-driven model to obtain the simulated estimated damping force based on the feature data, until the set requirements are met, and use the deep deterministic policy gradient network with updated parameters as the force tracking controller; if the set requirements are met, use the deep deterministic policy gradient network as the force tracking controller. Integrate the data-driven model, boundary constraints, and force tracking controller into a force tracking module; Real-time acquisition of raw characteristic data of the semi-active suspension system; acquisition of desired force; A force tracking module is used to obtain the control current based on the original feature data and the desired force; the control current is used to control the semi-active suspension system.

[0006] In one embodiment, a data-driven model is constructed based on convolutional neural networks and long short-term memory neural networks, including: Acquire historical characteristic data of the semi-active suspension system and the corresponding actual damping force; historical characteristic data includes historical actual current, historical piston displacement, and historical piston speed. An initial data-driven model is constructed based on a convolutional neural network and a long short-term memory neural network; the initial data-driven model is trained using a loss function based on the historical feature data and the actual damping force corresponding to the historical feature data, thus obtaining the data-driven model.

[0007] In one embodiment, the deep deterministic policy gradient network includes a policy network and a value network; A deep deterministic strategy gradient network is used to obtain the simulated control current based on the simulated expected damping force, the simulated estimated damping force, and characteristic data, including: The simulation state variables are constructed based on the simulated expected damping force, the simulated estimated damping force, and the characteristic data; The simulated control current is obtained based on the simulated state variables using a deterministic strategy in the strategy network.

[0008] In one implementation, updating the parameters of the deep deterministic policy gradient network based on feedback feature data includes: The simulation state variables are constructed based on the simulated expected damping force, the simulated estimated damping force, and the characteristic data; The parameters of the policy network are updated based on the feedback feature data and the simulation state variables, thereby updating the parameters of the deep deterministic policy gradient network.

[0009] In one embodiment, the process of determining whether the characteristic data fed back by the simulated semi-active suspension system meets the set requirements includes: The reward value is determined based on the feedback feature data and the reward function in the value network. The cumulative reward value is determined based on the aforementioned reward value; Determine whether the cumulative reward value meets the set requirements.

[0010] In one implementation, the setting requirement is that the cumulative reward value no longer increases after a set number of consecutive iterations.

[0011] In one embodiment, the data-driven model includes a convolutional neural network layer, a long short-term memory neural network layer, and a fully connected layer; A data-driven model is used to obtain the simulated estimated damping force based on feature data, including: The convolutional neural network layer is used to extract features based on the feature data; The state features are obtained by using the long short-term memory neural network and the extracted features. Using the fully connected layer, the simulated estimated damping force is obtained based on the state characteristics.

[0012] In a second aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the damping force closed-loop control method described in any one of the above.

[0013] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the damping force closed-loop control method described in any one of the above.

[0014] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the damping force closed-loop control method described above.

[0015] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a damping force closed-loop control method, device, medium, and product. Through a constructed data-driven model, it can accurately capture the non-stationary response delay and hysteresis characteristics (i.e., raw characteristic data) of a semi-active suspension system and use them for real-time estimation of the damping force. Boundary constraints can constrain the desired force, ensuring the final control current conforms to the physical boundaries. The force tracking controller generates the control current based on the desired and estimated damping forces, completing the mapping from the desired force to the control current in the semi-active suspension system, thus improving the force tracking accuracy of the semi-active suspension system under complex instantaneous impact and other dynamic environments. Finally, the force tracking module consists of the data-driven model, boundary constraints, and the force tracking controller obtained based on a deep deterministic policy gradient network. This force tracking module is applicable to the control of any semi-active suspension system, improving the performance of semi-active suspension systems under complex dynamic environments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a damping force closed-loop control method in one embodiment of this application; Figure 2 A schematic diagram of boundary constraints provided for an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] In one exemplary embodiment, such as Figure 1 As shown, a damping force closed-loop control method is provided, including: Step S1: Construct a data-driven model based on convolutional neural networks and long short-term memory neural networks.

[0021] Step S2: Obtain the characteristic data of the simulated semi-active suspension system. Obtain the desired simulation force.

[0022] Step S3: Using a data-driven model, the simulated estimated damping force is obtained based on the feature data. Boundary constraints are used to obtain the simulated expected damping force based on the feature data and the simulated expected force.

[0023] Step S4 involves employing a deep deterministic strategy gradient network to obtain the simulation control current based on the simulated expected damping force, the simulated estimated damping force, and characteristic data. This simulation control current is used to control the simulated semi-active suspension system.

[0024] Step S5: Obtain the feature data fed back from the simulated semi-active suspension system. Determine whether the feature data fed back from the simulated semi-active suspension system meets the set requirements. If it does not meet the set requirements, update the parameters of the deep deterministic policy gradient network based on the feedback feature data, and return to execute the step of using a data-driven model to obtain the simulated estimated damping force based on the feature data, until the set requirements are met. Then, use the deep deterministic policy gradient network with updated parameters as the force tracking controller. If the set requirements are met, use the deep deterministic policy gradient network as the force tracking controller.

[0025] Step S6: Integrate the data-driven model, boundary constraints, and force tracking controller into a force tracking module.

[0026] Step S7: Acquire the raw characteristic data of the semi-active suspension system in real time. Obtain the desired force.

[0027] Step S8: Using a force tracking module, the control current is obtained based on the original feature data and the desired force. The control current is used to control the semi-active suspension system.

[0028] In one embodiment, step S1 includes: acquiring historical feature data of the semi-active suspension system and the corresponding actual damping force. The historical feature data includes historical actual current, historical piston displacement, and historical piston velocity. An initial data-driven model is constructed based on a convolutional neural network and a long short-term memory neural network. A loss function is used to train the initial data-driven model based on the historical feature data and the corresponding actual damping force, resulting in the data-driven model.

[0029] The data-driven model includes convolutional neural network layers, long short-term memory neural network layers, and fully connected layers. Based on this, step S3 uses the data-driven model to obtain the simulated estimated damping force based on feature data, including: using convolutional neural network layers to extract features from the feature data; using long short-term memory neural networks to obtain state features based on the extracted features; and using fully connected layers to obtain the simulated estimated damping force based on the state features.

[0030] For example, historical characteristic data (in sequence form) of the magnetorheological damper under various operating conditions and the corresponding actual damping force are synchronously collected using sensors (e.g., force sensors and displacement sensors; in this embodiment, the sensor acquisition frequency is 1000Hz). Sequence data is generated based on the historical characteristic data. The sequence data dimension is three-dimensional (number of samples, sequence period, number of features). For example, historical actual current, historical piston displacement, and historical piston velocity are collected at 1000 time points, and the actual damping force at these 1000 time points is obtained. A sequence period is predefined (e.g., 30 time points per sequence period), then the sequence data dimension is (971, 30, 4). The actual damping force corresponding to that sequence period is determined based on the actual damping force corresponding to all time points within the sequence period (e.g., taking the average of all actual damping forces within the sequence period). After denoising and normalizing the sequence data, a sliding window technique is used to construct sample data with temporal correlation characteristics. Sample pairs are constructed using the sample data as input and the corresponding actual damping force as output, further building a training sample set.

[0031] An initial data-driven model is constructed using Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM). The CNN network extracts spatial features from multi-dimensional input features (e.g., sample data, feature data, and raw feature data). The extracted feature vectors are then input into the LSTM network, which uses its internal gating mechanism to memorize historical state information, thus characterizing the unique hysteresis effect and response delay of the magnetorheological fluid. A fully connected layer is then added after the LSTM network, thereby establishing a high-precision data-driven damping force estimation model (i.e., a data-driven model) capable of replacing traditional physical models and providing real-time perception of the dynamic characteristics of the vibration damper. This provides an environmental observation foundation for subsequent reinforcement learning.

[0032] In this embodiment, the convolutional neural network layer uses a dual-branch convolutional neural network to extract features from the feature data at the current time and the feature data at historical time. The convolution operation slides along the time dimension to generate local features, as shown below: In the formula, For branches Output characteristics It is an activation function. It is a branch convolution kernel, yes Feature data at any given time It is a branch The bias term, This represents the branch index. The combined feature representation of the outputs from branch 1 and branch 2 is as follows: In the formula, This indicates a merge operation. for Merging features at each time step (i.e., extracting features). for The characteristics of the outputs of branch 1 and branch 2 at time points.

[0033] The input feature data, after being processed by the convolutional neural network layer, is encoded into a higher-level, more compact temporal feature sequence (i.e., merged features), and then input into the first LSTM unit of the Long Short-Term Memory (LSTM) neural network layer. The network's internal state update is as follows: .

[0034] .

[0035] .

[0036] .

[0037] In the formula, , and These represent the forget gate, input gate, and output gate at time t, respectively. Represents the hidden unit at time t; , and These represent the weights of the forget gate, input gate, and output gate, respectively. , and This represents the bias terms for the forget gate, input gate, and output gate. This represents the cell state at time t. This represents the candidate value at time t. This indicates element-wise multiplication.

[0038] The feature sequence output from the first LSTM unit is input into the second LSTM unit. Then, the feature sequence (i.e., state features) from the last time step of the second LSTM unit is input into the fully connected layer for linearization to obtain the simulated estimated damping force. , is represented as: In the formula, for State characteristics at any given moment It is the weight matrix of the fully connected layer. This is the bias for the fully connected layer.

[0039] Furthermore, because the output force of a semi-active suspension system is strictly limited by its real-time motion state (such as piston speed), it cannot generate control forces exceeding boundary limits. Therefore, a data-driven model is used to further construct the boundary characteristic curves of maximum and minimum damping forces, forming boundary constraints. The input data includes a maximum current of 3A and a minimum current of 0A, and piston speed. and simulation expectation force (Pre-calculated or pre-set), the boundary values ​​of the maximum damping force are obtained respectively. and minimum damping force boundary The output force is subjected to boundary constraints (i.e., constraints on the maximum and minimum damping force boundaries) to obtain the simulated desired damping force. ,like Figure 2 As shown, this characteristic curve will serve as a motion limiter for the subsequent desired force. When the target control force (i.e., the desired force) exceeds this physical boundary, the system will automatically map the output force (i.e., the desired force) to the achievable limit range.

[0040] In one embodiment, the deep deterministic policy gradient network includes a policy network and a value network. Based on this, step S4 includes: constructing simulation state variables based on the simulated expected damping force, the simulated estimated damping force, and feature data; and using a deterministic policy from the policy network to obtain the simulated control current based on the simulation state variables.

[0041] Step S5, determining whether the feature data fed back from the simulated semi-active suspension system meets the set requirements, includes: using the reward function in the value network to determine the reward value based on the feedback feature data; determining the cumulative reward value based on the reward value; and determining whether the cumulative reward value meets the set requirements. The set requirements are: the cumulative reward value will no longer increase after a set number of consecutive iterations.

[0042] Step S5 involves updating the parameters of the deep deterministic policy gradient network based on the feedback feature data, including: constructing simulation state variables based on the simulated expected damping force, the simulated estimated damping force, and the feature data. The parameters of the policy network are then updated based on the feedback feature data and the simulation state variables to update the parameters of the deep deterministic policy gradient network.

[0043] For example, a Deep Deterministic Policy Gradient (DDPG) network can be used as the force tracking controller for a semi-active suspension system. During the training of the DDPG network, the input parameters include the simulated desired damping force, the simulated estimated damping force, and feature data (including dynamic velocity). (i.e., piston speed) and feedback current The output parameter is the simulated control current. The deterministic policy output and Actor-Critic (policy network-value function network) architecture of the deep deterministic policy gradient network provide a stable and direct current mapping for the semi-active suspension system. Furthermore, the experience replay mechanism allows the deep deterministic policy gradient network to repeatedly learn from stored historical states, actions, and reward data, effectively capturing dynamic characteristics dependent on historical states. The deep deterministic policy gradient network can indirectly learn and compensate for the time delay caused by the characteristics of the semi-active suspension based on the input parameters, ultimately generating more predictable and appropriate control current commands to compensate for the time delay, thereby improving the performance of the semi-active suspension system. For ease of description, the simulated control current... Also named output action .

[0044] In DDPG networks, there are two types of adjustable networks: policy networks (Actors) and value function networks (Critics). The Actor-Critic architecture is an important method in reinforcement learning, combining the advantages of value function estimation and policy optimization. The policy network (in this embodiment, a multilayer perceptron) determines the control current to be taken based on the state response of the semi-active suspension system, while the value function network estimates the value of the action (i.e., the control current) taken in the current state, which guides the improvement of the policy network. The target policy network and the target value function network are constructed as a delayed and smoothly updated policy-value function pair, ensuring convergence stability under nonlinear function approximation and suppressing overestimation of the value function and policy oscillations caused by high-frequency changes in online network parameters.

[0045] Because semi-active suspensions have time-varying delay characteristics, we consider constructing simulation state variables using deep reinforcement learning based on historical and current information of the semi-active suspension system (i.e., current input parameters and historical input parameters) to enhance the ability to represent dynamic processes and perceive the hysteresis characteristics in the dynamic response of the simulated semi-active suspension system. .

[0046] In the formula, express Simulation state variables at time t, express Simulation estimation of damping force at time t, express The simulated expected damping force at time [time] express The current is constantly being fed back. express Simulate the dynamic velocity of the semi-active suspension system at all times. Indicates a historical window.

[0047] Policy networks employ deterministic policies. Directly output actions , . For the parameters of the policy network, express The output action at any given moment.

[0048] The reward is a feedback signal from the simulated semi-active suspension system, used to evaluate the performance of the DDPG network in taking specific actions under specific conditions. To fully consider the dynamic characteristics of the semi-active suspension system, a reward function based on force tracking characteristic evaluation variables is established in the value function network. This reward function is expressed as: .

[0049] In the formula, The weights representing force error and rate of change of current. express The reward value at any given moment.

[0050] The goal of the DDPG network iteration is to maximize the cumulative reward (i.e., the cumulative reward value), represented as: .

[0051] In the formula, This represents the cumulative reward value; It is the expectation, indicating that the simulation state variables follow the policy distribution. ; These are the parameters of the policy network. yes The reward value at any moment, It is a discount factor.

[0052] Specifically, the parameter update process for the policy network and value function network is as follows: In the DDPG network, the update objective of the policy network is to maximize the Q-value output of the value function network, thereby learning an optimal deterministic policy. First, a batch of states is sampled from the experience pool (i.e., feature data of the simulated semi-active suspension system is obtained). The current feature data is input into the current policy network to obtain the current action. Then, this state-action pair (i.e., feature data-action pair) is input into the current value function network to obtain the Q-value (the Q-value is calculated based on the forward propagation of the current state-action pair by the value function network). The gradient of the Q-value with respect to the policy network parameters is calculated using the chain rule. , is represented as: .

[0053] In the formula, This represents the number of samples randomly sampled from the experience replay pool. The value function network output Q is related to the action. gradient, Represents the policy network, The policy network represents the parameters. The gradient is calculated. Then, the parameters of the policy network are adjusted along the direction of gradient ascent so that the output action can obtain a higher reward.

[0054] In the above Q-value calculation and parameter update, the reward value It does not appear directly in the gradient calculation of the policy network, but is used to construct the network value of the objective value function. If there is no reward value ,Target The loss of feedback prevents value function networks from learning the correct values.

[0055] In the DDPG network, the update process of the value function network essentially employs temporal difference learning for parameter approximation. First, a mini-batch of data is randomly sampled from the experience replay pool (consisting of feature data from a simulated semi-active suspension system, corresponding actions, and reward values). This mini-batch includes the current state (i.e., feature data), action, reward, and the state at the next time step (i.e., feedback feature data). The state at the next time step is then input into the target policy network. Get the next moment's action Then, the objective value function network and the value function network are used to calculate respectively. and The loss function of a value function network is defined as the error between the two, expressed as: .

[0056] In the formula, This represents the error between the current value function network prediction value Q and the target value y. The network parameters of the value function are then updated using gradient descent. .

[0057] Finally, the target policy network and the target value function network update their network parameters using a soft update method, as follows: .

[0058] In the formula, This represents the parameters of the objective value function network and the objective policy network. The parameters of the value function network and the policy network are represented.

[0059] To ensure the force tracking module is robust to complex road conditions, a co-simulation architecture using Adams and Simulink was employed for offline policy training of the DDPG network. The simulated semi-active suspension system was deployed in Adams, while the constructed data-driven model, boundary constraints, and the DDPG network were deployed as the control strategy in Simulink. During training, the DDPG network continuously observed real-time feedback feature data from Adams, outputting control current within preset physical boundary constraints (determined based on the compressive limit stroke of the magnetorheological damper). It then continuously optimized the deterministic strategy of the policy network based on the reward value until the DDPG network achieved high-precision force tracking under various extreme conditions, thus serving as the force tracking controller.

[0060] The data-driven model, boundary constraints, and force tracking controller are integrated into a force tracking module. Finally, the trained and converged force tracking module is deployed on a quarter-mount test bench, thereby achieving accurate dynamic tracking of the target damping force in a real physical environment.

[0061] The force tracking module of the semi-active suspension system consists of a data-driven model, boundary constraints, and a force tracking controller. The data-driven model estimates the corresponding damping force in real time based on the suspension's response state (i.e., the output raw characteristic data). Boundary constraints ensure that the desired output damping force is within the physical constraints of the semi-active suspension system. The force tracking controller module calculates the control current based on the real-time estimated damping force, raw characteristic data, and desired damping force to control the semi-active suspension system and improve its performance.

[0062] Based on the above embodiments, the damping force closed-loop control method provided in this application proposes a data-driven force tracking module. This force tracking module is decoupled from the semi-active suspension controller and consists of a data-driven model, boundary constraints, and a force tracking controller obtained based on a deep deterministic policy gradient network. The data-driven model accurately captures the non-stationary response delay and hysteresis characteristics (i.e., raw feature data) of the magnetorheological damper in the semi-active suspension system and uses it for real-time estimation of the damping force. Boundary constraints constrain the desired force. The force tracking controller generates a control current based on the desired and estimated damping forces, completing the mapping from the desired force to the control current in the semi-active suspension system. This force tracking module is independent of the semi-active suspension system and is therefore applicable to any semi-active suspension control, improving the force tracking accuracy of the semi-active suspension system under complex instantaneous impact and other dynamic environments, and enhancing the performance of the semi-active suspension system under complex dynamic environments.

[0063] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data related to the damping force closed-loop control method. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a damping force closed-loop control method.

[0064] Those skilled in the art will understand that Figure 3The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0065] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0066] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0069] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0070] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A closed-loop control method for damping force, characterized in that, include: A data-driven model is built based on convolutional neural networks and long short-term memory neural networks; Acquire characteristic data of a simulated semi-active suspension system; Obtain the simulation expectation force; A data-driven model is used to obtain the simulated estimated damping force based on feature data; boundary constraints are used to obtain the simulated expected damping force based on feature data and simulated expected force. A deep deterministic strategy gradient network is used to obtain the simulation control current based on the simulated expected damping force, the simulated estimated damping force, and characteristic data; the simulation control current is used to control the simulated semi-active suspension system. Acquire feature data from the simulated semi-active suspension system; determine whether the feature data from the simulated semi-active suspension system meets the set requirements; if it does not meet the set requirements, update the parameters of the deep deterministic strategy gradient network based on the feedback feature data, and return to execute the step of using a data-driven model to obtain the simulated estimated damping force based on the feature data, until the set requirements are met, and use the deep deterministic strategy gradient network with updated parameters as the force tracking controller. If the set requirements are met, the deep deterministic policy gradient network will be used as the force tracking controller; Integrate the data-driven model, boundary constraints, and force tracking controller into a force tracking module; Real-time acquisition of raw characteristic data of the semi-active suspension system; acquisition of desired force; A force tracking module is used to obtain the control current based on the original feature data and the desired force; the control current is used to control the semi-active suspension system.

2. The damping force closed-loop control method according to claim 1, characterized in that, Data-driven models are built based on convolutional neural networks and long short-term memory neural networks, including: Acquire historical characteristic data of the semi-active suspension system and the corresponding actual damping force; historical characteristic data includes historical actual current, historical piston displacement, and historical piston speed. An initial data-driven model is constructed based on a convolutional neural network and a long short-term memory neural network; the initial data-driven model is trained using a loss function based on the historical feature data and the actual damping force corresponding to the historical feature data, thus obtaining the data-driven model.

3. The damping force closed-loop control method according to claim 1, characterized in that, The deep deterministic policy gradient network includes a policy network and a value network; A deep deterministic strategy gradient network is used to obtain the simulated control current based on the simulated expected damping force, the simulated estimated damping force, and characteristic data, including: The simulation state variables are constructed based on the simulated expected damping force, the simulated estimated damping force, and the characteristic data; The simulated control current is obtained based on the simulated state variables using a deterministic strategy in the strategy network.

4. The damping force closed-loop control method according to claim 1, characterized in that, The parameters of the deep deterministic policy gradient network are updated based on the feedback feature data, including: The simulation state variables are constructed based on the simulated expected damping force, the simulated estimated damping force, and the characteristic data; The parameters of the policy network are updated based on the feedback feature data and the simulation state variables, thereby updating the parameters of the deep deterministic policy gradient network.

5. The damping force closed-loop control method according to claim 3, characterized in that, The process of determining whether the characteristic data fed back by the simulated semi-active suspension system meets the set requirements includes: The reward value is determined based on the feedback feature data and the reward function in the value network. The cumulative reward value is determined based on the aforementioned reward value; Determine whether the cumulative reward value meets the set requirements.

6. The damping force closed-loop control method according to claim 5, characterized in that, The setting requirement is that the cumulative reward value will no longer increase after a set number of consecutive iterations.

7. The damping force closed-loop control method according to claim 1, characterized in that, The data-driven model includes a convolutional neural network layer, a long short-term memory neural network layer, and a fully connected layer; A data-driven model is used to obtain the simulated estimated damping force based on feature data, including: The convolutional neural network layer is used to extract features based on the feature data; The state features are obtained by using the long short-term memory neural network and the extracted features. Using the fully connected layer, the simulated estimated damping force is obtained based on the state characteristics.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the damping force closed-loop control method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the damping force closed-loop control method according to any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the damping force closed-loop control method according to any one of claims 1-7.

Images