Vehicle suspension control method and system based on control barrier function and storage medium

Abstract

The invention relates to the technical field of automobile engineering, and discloses a vehicle suspension control method and system based on a control barrier function and a storage medium, and the method comprises the following steps: obtaining suspension control data, and determining a state quantity and a control quantity according to the suspension control data; constructing a vehicle seven-degree-of-freedom mathematical model based on the state quantity; according to a control barrier function generation algorithm, a control barrier function is generated in combination with the vehicle seven-degree-of-freedom mathematical model; generating a reward function by using a reward generation algorithm according to the suspension control data; obtaining a control strategy by using a reinforcement learning algorithm based on the reward function; inputting the control strategy into the control barrier function, and outputting the latest control strategy after minimum intrusion security constraint; in order to avoid control behaviors affecting system safety in the reinforcement learning training process, the safety reinforcement learning method based on the control barrier function is used for controlling the vehicle active suspension, and the purpose of controlling the stability of the vehicle suspension is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of automotive engineering, and in particular to a vehicle suspension control method, system, and storage medium based on a control barrier function. Background Art

[0002] Vehicle suspension is the elastic device connecting the vehicle body to the wheels, typically modeled as a mass-spring-damper system. Vehicle suspension control directly impacts ride comfort and safety, and therefore plays a crucial role in the vehicle's overall performance.

[0003] Traditional suspension control usually adopts simple heuristic control algorithms or designs optimal control algorithms based on state-space models.

[0004] The heuristic control algorithm can be a traditional PID control algorithm, which is a control algorithm based on three factors: proportional, integral and differential. The stability of the vehicle suspension can be controlled by adjusting these three parameters. However, this algorithm can only be applied to some single-input and single-output scenarios, and the parameters are difficult to adjust, which is prone to overshoot.

[0005] Designing an optimal control algorithm based on a state-space model can involve using a model-predictive control (MPC) algorithm. This algorithm uses the state-space model to describe and predict the system's future state and uses the predicted results within a finite time domain to optimize control parameters to achieve suspension stability control. This algorithm improves control accuracy and robustness, but its computational speed is slow, which can lead to latency issues.

[0006] Another approach is to use a reinforcement learning-based algorithm, which simulates the entire state dynamics process, gradually adjusts the control strategy, and uses a reward function to evaluate the effectiveness of the control strategy, ultimately achieving suspension stability control. This method offers good adaptability and robustness, but consumes significant computing resources and requires extended training time. Furthermore, because reinforcement learning requires exploring the optimal control strategy through interaction with the environment, its advantage lies in searching for the optimal strategy in the largest possible space. However, during this exploration, some strategies may exceed the physical safety limits of the system. In the worst case, this can directly damage the system, leading to training interruption.

[0007] Heuristic control algorithms are simple to implement but struggle to achieve reliable optimal performance. Designing optimal control algorithms based on state-space models requires an accurate system dynamics model. However, commonly used linear system dynamics models struggle to reflect the vehicle's driving characteristics under various operating conditions. Therefore, a method is needed to prevent control behaviors that could compromise system safety during reinforcement learning training. Summary of the Invention

[0008] In order to avoid control behaviors that affect system safety during reinforcement learning training, the present application provides a vehicle suspension control method, system and storage medium based on a control barrier function.

[0009] In a first aspect, the present application provides a vehicle suspension control method based on a control barrier function, which adopts the following technical solutions:

[0010] A vehicle suspension control method based on a control barrier function comprises the following steps:

[0011] Acquiring suspension control data, and determining a state quantity and a control quantity according to the suspension control data;

[0012] Constructing a seven-degree-of-freedom mathematical model of the vehicle based on the state quantity;

[0013] Generate a control barrier function according to a control barrier function generation algorithm and in combination with the vehicle seven-degree-of-freedom mathematical model;

[0014] Generate a reward function using a reward generation algorithm based on the suspension control data;

[0015] Obtaining a control strategy using a reinforcement learning algorithm based on the reward function;

[0016] The control strategy is input into the control barrier function, and after passing the minimum intrusion safety constraint, the latest control strategy is output.

[0017] By adopting the above technical solution, the reinforcement learning algorithm is applied to the vehicle suspension stability control to reduce the state indicators such as the vehicle body vertical acceleration, pitch angular acceleration and roll angular acceleration, thereby improving the vehicle suspension stability. In order to avoid control behaviors that affect the system safety during the reinforcement learning training process, the safety reinforcement learning method based on the control barrier function is used for vehicle active suspension control to achieve the purpose of vehicle suspension stability control.

[0018] Optionally, the seven-degree-of-freedom mathematical model of the vehicle is in the form of:

[0019] ;

[0020] Among them, X is the state quantity, is the first-order derivative of the state quantity, A and B are the state transfer matrices, and U is the damping force generated by the suspension actuator;

[0021] The state quantity X includes the state quantities of the body-related parameters and the state quantities of the wheel-related parameters;

[0022] The state quantities of body-related parameters include: vehicle acceleration AZ SM , pitch angular acceleration AAx, roll angular acceleration AAy and sideslip angular acceleration AAz;

[0023] The state quantities of wheel-related parameters include: suspension dynamic travel SD, tire dynamic displacement Z L , tire vertical displacement velocity TDV and suspension relative displacement velocity.

[0024] By adopting the above technical solution, the body-related state quantity is the sprung state data, and the wheel-related state quantity is the unsprung state data. The control input U is the damping force generated by the suspension actuator, with each wheel corresponding to an actuator, for a total of four damping forces. The damping force generated by the reinforcement learning strategy can be used to predict the vertical acceleration, pitch acceleration, roll acceleration and other information of the vehicle body at the next moment. This information will be applied to the reinforcement learning reward function, giving reinforcement learning the ability to plan for the future.

[0025] Optionally, the control barrier function generation algorithm includes:

[0026] Define a safe set C as a continuously differentiable function h:R n →R, i.e., C = {x∈R n : superlevel set of h(x)≥0};

[0027] Given a nonlinear control-affine system: x ， =f(x)+g(x)u,

[0028] Where f(x) and g(x) are both locally Lipschitz continuous, x∈X⊂R n is the set of allowed states, u∈ is the set of allowed inputs; where u is the abstract input of the algorithm, is the input set;

[0029] Set the classK function α and satisfy: H ， (x, u) ≥ -α(h(x)) ⇔ C is an invariant set; the classK function α is of the form α(h(x)) = γh(x), γ>0;

[0030] Set the affine constraint on u as Au≤b.

[0031] By adopting the above technical solution, based on the definition of the control barrier function and the dynamic model of the vehicle suspension movement, the parameters that meet the safety control conditions are constrained, and the control barrier function is used as a filter layer connected to the reinforcement learning control strategy to perform safety constraints.

[0032] Optionally, the minimum intrusion safety constraint includes:

[0033] The optimization problem corresponding to the affine constraint is expressed as:

[0034] ;

[0035] The optimization problem is solved by a quadratic programming method.

[0036] By adopting the above technical solution, the action strategy u output by the reinforcement learning agent is modified as little as possible. RL , while ensuring control safety; after adding safety constraints, the safety filtering module will limit the output of the reinforcement learning algorithm, which will affect the training of the reinforcement learning algorithm. This impact is mainly reflected in the definition of the safety reward function.

[0037] Optionally, the reward generation algorithm includes:

[0038] The reward function r includes the performance reward r p and safety rewards s , and r=r p +r s ;

[0039] r p Represents the performance reward function at the current moment, expressed as the weighted square sum of the negative performance indicators, that is, r p = -α1(AZ SM ) 2 -α2(AA x ) 2 -α3(AA y ) 2 , where α1, α2 and α3 are hyperparameters;

[0040] The safety reward rs is defined as:

[0041] .

[0042] By adopting the above technical solution, the performance reward r p The negative sign of each performance index indicates that the three state indicators can be reduced to the greatest extent in the suspension control. The safety constraints are mainly reflected in the following aspects: (1) The upward vertical force on the tire cannot exceed the gravity of the suspension and the wheel, otherwise the tire will lose contact with the ground and cause safety problems; (2) The range of motion of the suspension cannot exceed the dynamic stroke of the suspension, otherwise the suspension movement will cause collision or even jamming.

[0043] Optionally, the reinforcement learning algorithm is a PPO algorithm, and the training process of the PPO algorithm includes the following steps:

[0044] Step a: generating experience data corresponding to states, actions, and rewards from the current control strategy;

[0045] Step b: using the current control strategy to evaluate the quality of the empirical data corresponding to the state and action, and calculating an advantage function, which is used to reflect the average level of the current action relative to all actions in the current state;

[0046] Step c: Calculating a strategy loss; the strategy loss is the minimum value of the sum of the ratio of the producer effect of the current control strategy to the past control strategy and the confrontation loss, wherein the confrontation loss is the smaller value of the ratio of the producer effect of the current control strategy to the past control strategy and a threshold;

[0047] Step d: Calculate the value function loss, which is the mean square error between the value function estimate of the current state and the preset target value;

[0048] Step e: Update the strategy: perform stochastic gradient descent to optimize the parameters of the strategy based on the total loss;

[0049] Step f: Iterate steps a to e multiple times.

[0050] By adopting the above technical solution, it is iterated multiple times until the appropriate policy parameters are found so that the agent performs best in the environment.

[0051] Optionally, the method further comprises the following steps:

[0052] The number of iterations performed is set in positive correlation with the average level.

[0053] By adopting the above technical solutions, the strategy parameters can be more closely matched with the actual vehicle conditions.

[0054] Optionally, the method further comprises the following steps:

[0055] Acquire the state quantity in real time through the vehicle-mounted sensor;

[0056] Filtering the state quantity, using a window sliding filtering algorithm, with the window length being inversely correlated with U;

[0057] Performing noise reduction on the state quantity, wherein the noise reduction adopts a Gaussian filtering algorithm;

[0058] The state quantity is calibrated using a linear compensation algorithm.

[0059] By adopting the above technical solutions, more accurate data can be obtained through filtering, noise reduction and calibration.

[0060] In a second aspect, the present application provides a vehicle suspension control system based on a control barrier function, which adopts the following technical solutions:

[0061] A vehicle suspension control system based on a control barrier function comprises a processor, wherein the processor executes the steps of any one of the above-mentioned vehicle suspension control methods based on a control barrier function.

[0062] In a third aspect, the present application provides a storage medium that adopts the following technical solution:

[0063] A storage medium stores a program, which, when executed by a processor, implements the steps of any one of the above-mentioned vehicle suspension control methods based on a control barrier function.

[0064] In summary, the present application includes at least one of the following beneficial technical effects: it is based on the control barrier function theory, which can add a safety barrier to reinforcement learning, and rely on the theory to ensure the filtering of control actions in reinforcement learning training to avoid safety problems such as excessive tire displacement in the system, and the control barrier function is constructed based on the suspension system dynamics model; reinforcement learning is used as the basic algorithm for active suspension control, which does not rely on accurate system models and has strong adaptability. The optimal control strategy can be obtained through full exploration of the interaction between the intelligent agent and the environment. The PPO algorithm adopted can quickly learn the optimal control strategy; it has modular characteristics, simple implementation and good feasibility. The control barrier function is connected to the reinforcement learning control strategy as a filtering layer for safety constraints. The strategy optimization of reinforcement learning is affected only by changing the reward function (external interface) without changing its internal structure. At the same time, the reward function design comprehensively considers multiple state indicators to achieve detailed and comprehensive suspension stability control. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a framework diagram of a secure reinforcement learning control system.

[0066] Figure 2 It is the algorithm flow chart of the present invention.

[0067] Figure 3 It is the network structure diagram of the PPO algorithm. DETAILED DESCRIPTION

[0068] Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings.

[0069] Throughout this specification, reference to the terms "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0070] A vehicle's suspension is an elastic device connecting the vehicle body to the wheels, typically modeled as a mass-spring-damper system. Vehicle suspension control directly impacts ride comfort and safety, and therefore plays a crucial role in overall vehicle performance. Traditional suspension control typically employs simple heuristic control algorithms or optimal control algorithms designed based on state-space models. The former is simple to implement but struggles to achieve reliable optimal performance. The latter requires an accurate system dynamics model, but commonly used linear system dynamics models struggle to reflect the vehicle's driving characteristics under various operating conditions. Reinforcement learning control is a recently emerging data-driven control approach that treats the controlled object as a "black box" environment and explores the optimal control strategy through continuous interaction with the environment. Reinforcement learning methods are characterized by low model dependence, strong environmental adaptability, and excellent control performance. However, because reinforcement learning algorithms must explore a wide range of possible action spaces under unconstrained conditions to discover the optimal strategy, if intermediate transition strategies are unrestricted, they may produce actions that exceed the physical limitations of the system. For example, in suspension control, if the control strategy causes excessive tire displacement, the vehicle may lift off the ground, posing safety risks.

[0071] To fully leverage the advantages of reinforcement learning control while ensuring control safety, the present invention discloses a vehicle suspension control method based on a control barrier function (CBF). This method proposes a safe reinforcement learning active vehicle suspension control method based on a control barrier function (CBF). This method utilizes a model-free reinforcement learning approach to fully explore suspension control strategies. The resulting control strategy is filtered through a safety module constructed based on the CBF and then input into the control object. A reward signal is also fed back to the control agent to further optimize the control strategy. This method actively controls the suspension while ensuring physical safety, reducing the vehicle's vertical acceleration, pitch acceleration, and roll acceleration during driving.

[0072] Reference Figure 1 and Figure 2 , the method comprises the following steps:

[0073] Suspension control data is acquired and state variables and control variables are determined based on this data. State variables are acquired in real time via onboard sensors. The state variables are filtered using a sliding window filter algorithm, with the window length inversely correlated with U. Noise reduction is performed on the state variables using a Gaussian filter algorithm. The state variables are calibrated using a linear compensation algorithm. This filtering, noise reduction, and calibration process yields more accurate data.

[0074] Construct a seven-degree-of-freedom mathematical model of the vehicle based on state quantities.

[0075] According to the control barrier function generation algorithm, the control barrier function is generated in combination with the vehicle's seven-degree-of-freedom mathematical model.

[0076] Generate a reward function using a reward generation algorithm based on the suspension control data.

[0077] The control strategy is obtained using the reinforcement learning algorithm based on the reward function.

[0078] The control strategy is input into the control barrier function, and after passing the minimum intrusion safety constraint, the latest control strategy is output.

[0079] The reinforcement learning algorithm is applied to the vehicle suspension stability control to reduce the vehicle body's vertical acceleration, pitch acceleration, and roll acceleration and other state indicators, thereby improving the vehicle's suspension stability. In order to avoid control behaviors that affect system safety during the reinforcement learning training process, a safety reinforcement learning method based on the control barrier function is used for vehicle active suspension control to achieve the purpose of vehicle suspension stability control.

[0080] The seven-degree-of-freedom mathematical model of the vehicle is in the form of:

[0081] ;

[0082] Among them, X is the state quantity, is the first-order derivative of the state quantity, A and B are the state transfer matrices, and U is the damping force generated by the suspension actuator.

[0083] The state quantity X includes the state quantities of the body-related parameters and the state quantities of the wheel-related parameters;

[0084] The state quantities of body-related parameters include: vehicle acceleration AZ SM , pitch angular acceleration AAx, roll angular acceleration AAy and sideslip angular acceleration AAz.

[0085] The state quantities of wheel-related parameters include: suspension dynamic travel SD, tire dynamic displacement Z L , tire vertical displacement velocity TDV and suspension relative displacement velocity.

[0086] The body-related state quantity is the sprung state data, and the wheel-related state quantity is the unsprung state data. The control input U is the damping force generated by the suspension actuator. Each wheel corresponds to an actuator, for a total of four damping forces. The damping force generated by the reinforcement learning strategy can be used to predict the vertical acceleration, pitch acceleration, roll acceleration and other information of the vehicle body at the next moment. This information will be applied to the reinforcement learning reward function, giving reinforcement learning the ability to plan for the future.

[0087] The control barrier function generation algorithm includes:

[0088] Define a safe set C as a continuously differentiable function h:R n →R, i.e., C = {x∈R n :h(x)≥0} superlevel set;

[0089] Given a nonlinear control-affine system: x ， =f(x)+g(x)u,

[0090] Where f(x) and g(x) are both locally Lipschitz continuous, x∈X⊂R n is the set of allowed states, u∈ is the set of allowed inputs; where u is the abstract input of the algorithm, is the input set.

[0091] Set the classK function α and satisfy: H ， (x, u) ≥ -α(h(x)) ⇔ C is an invariant set; the classK function α is of the form α(h(x)) = γh(x), γ>0;

[0092] Set the affine constraint on u as Au≤b.

[0093] Based on the definition of the control barrier function and the dynamic model of the vehicle suspension motion, the parameters that meet the safety control conditions are constrained, and the control barrier function is implemented as a filter layer connected to the reinforcement learning control strategy to perform safety constraints.

[0094] Minimally invasive safety constraints include:

[0095] The optimization problem corresponding to the affine constraint is expressed as:

[0096] ;

[0097] Among them, u(x): the final output safety control instruction, which is the control quantity that satisfies the safety constraints after optimization;

[0098] : The original output of the reinforcement learning (RL) algorithm without safety constraints, representing the control strategy under pure performance optimization;

[0099] is the set of allowed inputs;

[0100] : The objective function of "minimum intrusion" is used to measure the difference between the actual control input u and the reference control The square term makes the penalty more significant the larger the deviation, and 1 / 2 is to simplify the quadratic programming derivative calculation, so that the optimized u is as close to the ideal control as possible, reducing the "intrusiveness" of the original control strategy;

[0101] Au≤b: is the mathematical expression of the physical constraints of the suspension actuator, A is the constraint coefficient matrix, b is the constraint boundary vector, and the constraint u must be within the actuator's achievable range;

[0102] The optimization problem is solved by quadratic programming method.

[0103] Modify the action strategy u output by the reinforcement learning agent as little as possible RL , while ensuring control safety; after adding safety constraints, the safety filtering module will limit the output of the reinforcement learning algorithm, which will affect the training of the reinforcement learning algorithm. This impact is mainly reflected in the definition of the safety reward function.

[0104] The reward generation algorithm includes:

[0105] The reward function r includes the performance reward r p and safety rewards s , and r=r p +r s ;

[0106] r p Represents the performance reward function at the current moment, expressed as the weighted square sum of the negative performance indicators, that is, r p = -α1(AZ SM ) 2 -α2(AA x ) 2 -α3(AA y ) 2 , where α1, α2 and α3 are hyperparameters.

[0107] The safety reward rs is defined as:

[0108] ;

[0109] u1, u2: are the different components of the actual control input of the suspension actuator; u1 is the vertical force of the tire, and u2 is the suspension travel;

[0110] : The reference control input corresponding to u1 and u2, representing the control quantity / state quantity under the ideal safety state;

[0111] m sus : The mass of the suspension component is a physical parameter of the suspension system that affects the calculation of the intrusion degree of the safety constraint. The larger the mass, the more significant the impact of the same control deviation on system safety may be;

[0112] m tire Tire mass is involved in the calculation of the physical dimension of safety constraints, reflecting the impact of the overall mass of the suspension-tire system on safety control;

[0113] R sus : Safety margin related parameters of the suspension system, such as suspension dynamic travel limit, actuator arm, etc., characterize the physical characteristics of the suspension safety margin, R sus The larger it is, the higher the safety redundancy is and the larger the adjustable space of the control input is.

[0114] β1, β2: Hyperparameters for the weight of security rewards, which adjust the proportion of different security dimensions in the rewards. The larger the β value, the more significant the impact of the corresponding item on the security reward, guiding the strategy to prioritize the security of this dimension.

[0115] Performance Rewards p The negative sign of each performance index indicates that the three state indicators can be reduced to the greatest extent in the suspension control. The safety constraints are mainly reflected in the following aspects: (1) The upward vertical force on the tire cannot exceed the gravity of the suspension and the wheel, otherwise the tire will lose contact with the ground and cause safety problems; (2) The range of motion of the suspension cannot exceed the dynamic stroke of the suspension, otherwise the suspension movement will cause collision or even jamming.

[0116] Referring to Figure 3, the reinforcement learning algorithm is the PPO algorithm. The training process of the PPO algorithm includes the following steps:

[0117] Step a: Generate data: Generate experience data corresponding to states, actions, and rewards from the current control policy.

[0118] Step b: Calculate the advantage function: Use the current control strategy to evaluate the quality of the empirical data corresponding to the state and action, and calculate the advantage function. The advantage function is used to reflect the average level of the current action relative to all actions in the current state.

[0119] Step c: Calculate Policy-Loss: Calculate the policy loss; the policy loss is the minimum value of the sum of the ratio of the producer effect of the current control policy and the past control policy and the adversarial loss, where the adversarial loss is the smaller value of the ratio of the producer effect of the current control policy and the past control policy and a threshold.

[0120] Step d: Calculate Value-Loss: Calculate the value function loss, which is the mean square error between the estimated value of the current state and the preset target value.

[0121] Step e: Update the policy: Perform stochastic gradient descent to optimize the policy parameters based on the total loss.

[0122] Step f: Iterate steps a through e multiple times. The number of iterations is set in a positive correlation with the average level, so that the strategy parameters are more closely aligned with the actual vehicle conditions.

[0123] Repeated iterations are performed until appropriate policy parameters are found that optimize the agent's performance in the environment. Reinforcement learning modeling uses the vehicle's current motion state as the state space and the vehicle's suspension damping output as the action space. The reward function is a weighted sum of the current and future vehicle vertical acceleration, pitch acceleration, and roll acceleration, optimized using the PPO reinforcement learning algorithm. Reducing these three state metrics effectively improves vehicle comfort and stability while preventing the reinforcement learning control algorithm from outputting unsafe control strategies.

[0124] A theoretically guaranteed control barrier function is used to constrain the control output of reinforcement learning, balancing the safety and optimality of reinforcement learning control. A reinforcement learning algorithm with continuous control capabilities is employed for suspension control, which is suitable for the nonlinear and uncertain characteristics of suspension motion. Vehicle motion is controlled by controlling not only vertical acceleration but also pitch and roll acceleration to stabilize the vehicle's driving state. The method determines the system state and control variables based on the suspension control performance objective, and constructs a system dynamics model to generate the control barrier function. Based on the control barrier function theory, the control barrier function is generated in conjunction with the aforementioned system dynamics model. A reinforcement learning algorithm based on the PPO algorithm is used to design a reasonable reward function based on the suspension control performance requirements and safety constraints. A reinforcement learning control policy is obtained through interactive training in a real or simulated environment. The trained control policy is output to a safety filtering module based on the control barrier function, and the final control output is obtained after minimizing the intrusion into safety constraints. The safety-constrained control policy is then output to the suspension controller for application in a practical vehicle suspension control system. Experimental verification is carried out on a test bench or in an actual road environment to evaluate the control effect and performance of the method under various road conditions. The method can also be continuously optimized and improved based on the experimental results.

[0125] Safety reinforcement learning methods can be divided into two categories: modified optimization criteria and modified exploration processes. Modified optimization criteria methods, such as worst-case criteria, risk-sensitive criteria, and constraint strategy optimization methods, cannot guarantee that unsafe behaviors will not occur. On the other hand, modified exploration processes can always ensure that safe actions are taken through external guidance, such as using a safety filter layer to verify the output of the control algorithm and correct unsafe control behaviors, thereby achieving safe control. Therefore, this method adopts a safety assurance strategy for the modified exploration process, and performs constraint filtering after the reinforcement learning algorithm generates the control action, thereby ensuring that the filtered control action will not exceed the safety range.

[0126] This method, based on the theory of control barrier functions, ensures that the reinforcement learning algorithm does not exceed the physical safety limits of the system while exploring the optimal strategy, thus adding a safety barrier to reinforcement learning. Using reinforcement learning as the foundational algorithm for active suspension control, this method does not rely on an accurate system model and possesses strong adaptability. It fully explores the optimal control strategy through interaction between the agent and the environment. This modular approach is relatively simple to implement and has good feasibility. The control barrier function acts as a filter layer attached to the reinforcement learning control strategy to provide safety constraints. The reinforcement learning strategy optimization is affected only by changing the reward function (the external interface), without altering the internal structure of the reinforcement learning algorithm.

[0127] A theoretically guaranteed control barrier function is used to filter control actions during reinforcement learning training to avoid system safety issues, such as excessive tire displacement causing the tire to lift off the ground. This method constructs a control barrier function based on the dynamic model of the controlled suspension system to constrain or filter the control actions output by the reinforcement learning algorithm. In terms of the reinforcement learning algorithm, this method uses the PPO algorithm for dynamic vehicle control, which can learn the optimal control strategy at a relatively fast speed. In addition, in the design of the reward function, this method comprehensively considers multiple state indicators to achieve detailed and comprehensive suspension stability control.

[0128] An embodiment of the present application further discloses a vehicle suspension control system based on a control barrier function, comprising a processor, wherein the processor executes the steps of the vehicle suspension control method based on a control barrier function as described above.

[0129] An embodiment of the present application further discloses a storage medium storing a program, which, when executed by a processor, implements the steps of any one of the above-mentioned vehicle suspension control methods based on a control barrier function.

[0130] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A vehicle suspension control method based on a control barrier function, characterized in that: The steps include: Acquiring suspension control data, and determining a state quantity and a control quantity according to the suspension control data; Constructing a seven-degree-of-freedom mathematical model of the vehicle based on the state quantity; Generate a control barrier function according to a control barrier function generation algorithm and in combination with the vehicle seven-degree-of-freedom mathematical model; Generate a reward function using a reward generation algorithm based on the suspension control data; Obtaining a control strategy using a reinforcement learning algorithm based on the reward function; The control strategy is input into the control barrier function, and after passing the minimum intrusion safety constraint, the latest control strategy is output.

2. The vehicle suspension control method based on the control barrier function according to claim 1, characterized in that: The seven-degree-of-freedom mathematical model of the vehicle is in the form of: ； Among them, X is the state quantity, is the first-order derivative of the state quantity, A and B are the state transfer matrices, and U is the damping force generated by the suspension actuator; The state quantity X includes the state quantity of the body-related parameters and the state quantity of the wheel-related parameters; The state quantities of body-related parameters include: vehicle acceleration AZ SM , pitch angular acceleration AAx, roll angular acceleration AAy and sideslip angular acceleration AAz; The state quantities of wheel-related parameters include: suspension dynamic travel SD, tire dynamic displacement Z L , tire vertical displacement velocity TDV and suspension relative displacement velocity.

3. The vehicle suspension control method based on the control barrier function according to claim 2, characterized in that: The control barrier function generation algorithm includes: Define a safe set C as a continuously differentiable function h:R n →R, i.e., C = {x∈R n : superlevel set of h(x)≥0}; Given a nonlinear control-affine system: x ， =f(x)+g(x)u, Where f(x) and g(x) are both locally Lipschitz continuous, x∈X⊂R n is the set of allowed states, u∈ is the set of allowed inputs; where u is the abstract input of the algorithm, is the input set; Set the classK function α and satisfy: H ， (x, u) ≥ -α(h(x)) ⇔ C is an invariant set; the classK function α is of the form α(h(x)) = γh(x), γ>0; Set the affine constraint on u as Au≤b.

4. The vehicle suspension control method based on the control barrier function according to claim 3, characterized in that: The minimal intrusion safety constraints include: The optimization problem corresponding to the affine constraint is expressed as: ； Among them, u(x): the final output safety control instruction, which is the control quantity that satisfies the safety constraints after optimization; : The original output of the reinforcement learning (RL) algorithm without safety constraints, representing the control strategy under pure performance optimization; is the set of allowed inputs; : "Minimal intrusion" objective function, which measures the difference between the actual control input u and the reference control The square term makes the penalty more significant the larger the deviation, and 1 / 2 is used to simplify the quadratic programming derivative calculation, so that the optimized u is as close to the ideal control as possible, reducing the "intrusiveness" of the original control strategy. Au≤b: is the mathematical expression of the physical constraints of the suspension actuator, A is the constraint coefficient matrix, b is the constraint boundary vector, and the constraint u must be within the actuator's achievable range; The optimization problem is solved by a quadratic programming method.

5. The vehicle suspension control method based on the control barrier function according to claim 4, characterized in that: The reward generation algorithm includes: The reward function r includes the performance reward r p and safety rewards s , and r=r p +r s ; r p Represents the performance reward function at the current moment, expressed as the weighted square sum of the negative performance indicators, that is, r p = -α1(AZ SM ) 2 -α2(AA x ) 2 -α3(AA y ) 2 , where α1, α2 and α3 are hyperparameters; The safety reward rs is defined as: ； u1, u2: are the different components of the actual control input of the suspension actuator; u1 is the vertical force of the tire, and u2 is the suspension travel; : The reference control input corresponding to u1 and u2, representing the control quantity / state quantity under the ideal safety state; m sus : The mass of the suspension component is a physical parameter of the suspension system that affects the calculation of the intrusion degree of the safety constraint. The larger the mass, the more significant the impact of the same control deviation on system safety may be; m tire Tire mass is involved in the calculation of the physical dimension of safety constraints, reflecting the impact of the overall mass of the suspension-tire system on safety control; R sus : Safety margin related parameters of the suspension system, characterizing the physical characteristics of the suspension safety margin, R sus The larger it is, the higher the safety redundancy is and the larger the adjustable space of the control input is. β1, β2: Hyperparameters for the weight of security rewards, which adjust the proportion of different security dimensions in the rewards. The larger the β value, the more significant the impact of the corresponding item on the security reward, guiding the strategy to prioritize the security of this dimension.

6. The vehicle suspension control method based on the control barrier function according to claim 1, characterized in that: The reinforcement learning algorithm is a PPO algorithm, and the training process of the PPO algorithm includes the following steps: Step a: generating experience data corresponding to states, actions, and rewards from the current control strategy; Step b: using the current control strategy to evaluate the quality of the empirical data corresponding to the state and action, and calculating an advantage function, which is used to reflect the average level of the current action relative to all actions in the current state; Step c: Calculating a strategy loss; the strategy loss is the minimum value of the sum of the ratio of the producer effect of the current control strategy to the past control strategy and the confrontation loss, wherein the confrontation loss is the smaller value of the ratio of the producer effect of the current control strategy to the past control strategy and a threshold; Step d: Calculate the value function loss, which is the mean square error between the value function estimate of the current state and the preset target value; Step e: Update the strategy: perform stochastic gradient descent to optimize the parameters of the strategy based on the total loss; Step f: Iterate steps a to e multiple times.

7. The vehicle suspension control method based on the control barrier function according to claim 6, characterized in that: The method further comprises the steps of: The number of iterations performed is set in positive correlation with the average level.

8. The vehicle suspension control method based on the control barrier function according to claim 7, characterized in that: The method further comprises the steps of: Acquire the state quantity in real time through the vehicle-mounted sensor; The state quantity is filtered using a window sliding filter algorithm, and the window length is set to be inversely correlated with U; U is the damping force generated by the suspension actuator; Performing noise reduction on the state quantity, wherein the noise reduction adopts a Gaussian filtering algorithm; The state quantity is calibrated using a linear compensation algorithm.

9. A vehicle suspension control system based on a control barrier function, characterized in that: The method comprises a processor, wherein the processor executes the steps of the vehicle suspension control method based on the control barrier function according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores a program, and when the program is executed by the processor, the steps of the vehicle suspension control method based on the control barrier function according to any one of claims 1 to 8 are implemented.

Images