Vehicle dynamic stability control method based on reinforcement learning and variable pole placement

Abstract

This invention provides a vehicle dynamic stability control method based on reinforcement learning and variable pole configuration, relating to the field of vehicle dynamics control. This method constructs multi-dimensional criteria for the centroid sideslip angle phase plane and energy phase plane to identify the vehicle's stability state and energy evolution trend in real time. It utilizes a reinforcement learning agent to make online decisions about the expected closed-loop pole positions of the system based on the current state, enabling the configuration of aggressive poles to improve flexibility under stable conditions and conservative poles to ensure stability under unstable conditions. Furthermore, this invention designs a safety monitoring mechanism that includes pole physical domain constraints, enabling the reshaping of the vehicle's dynamic characteristics through feedback control without altering the driver's steering intentions, effectively solving the challenge of real-time trade-off between flexibility and safety in intelligent chassis control.

Description

Technical Field

[0001] This invention relates to the field of vehicle dynamics control, and in particular to a vehicle dynamic stability control method based on reinforcement learning and variable pole configuration. Background Technology

[0002] With the development of automotive chassis electronics and intelligent technologies, Electronic Stability Control (ESC) and Direct Yaw Control (DYC) have become core technologies for improving active safety and handling performance. Traditional control strategies mostly rely on linear two-degree-of-freedom vehicle models and employ control algorithms with fixed gains, such as proportional-integral-derivative (PID) or linear quadratic regulators (LQR). However, vehicles face extremely complex dynamic requirements in actual driving: under normal road conditions, drivers typically seek faster response times and higher steering agility, requiring the system to possess more aggressive dynamic characteristics; while under unstable conditions such as extreme obstacle avoidance or low-adhesion road surfaces, it is essential to prioritize the rapid convergence of the vehicle's lateral state. This trade-off between agility and stability is the fundamental challenge currently facing the field of chassis coordinated control.

[0003] Existing fixed-parameter control methods have significant limitations in dealing with the highly nonlinear dynamic characteristics of vehicles. Fixed closed-loop pole configurations are ill-suited to rapidly changing conditions, such as transitions from low to high speeds or from dry to icy surfaces. This often leads to overly conservative behavior and loss of maneuverability in safe regions, or loss of control due to insufficient stability margins in extreme regions. Furthermore, traditional stability criteria often employ single-variable threshold methods, such as the centroid sideslip angle threshold, which fail to accurately characterize the energy evolution of the vehicle within complex nonlinear regions. While some deep learning-based end-to-end control schemes have emerged in recent years, their internal decision-making processes are often "black boxes," lacking clear physical interpretability. They also suffer from deficiencies in the smoothness of pole switching and fault tolerance in case of algorithm failure, making it difficult to meet the stringent robustness and safety requirements of automotive-grade control systems.

[0004] To overcome the above-mentioned shortcomings, there is an urgent need for a control scheme that can perceive the operating conditions in real time, dynamically adjust driving characteristics, and have a safety backup capability. Summary of the Invention

[0005] Objective: This invention aims to propose a vehicle dynamic stability control method based on reinforcement learning and variable pole configuration. By combining the theories of the centroid sideslip angle phase plane and the energy phase plane, the nonlinear stable state is accurately quantified. Utilizing the online decision-making capability of reinforcement learning, the closed-loop pole positions are adaptively optimized under different operating conditions, effectively achieving dynamic decoupling of vehicle performance across all scenarios. Furthermore, the introduction of a smooth switching function for pole changes ensures a smooth transition of control inputs to the actuators, addressing the challenges of impact and reliability issues in applying artificial intelligence algorithms to physical systems.

[0006] To achieve the above objectives, this invention provides a vehicle dynamic stability control method based on reinforcement learning and variable pole placement, comprising the following steps:

[0007] Step S1: Establish a nonlinear two-degree-of-freedom vehicle model as the control reference, and solve for the lateral forces, vertical loads, and front wheel slip angles of the vehicle's front and rear axles based on the nonlinear two-degree-of-freedom vehicle model.

[0008] Step S2: Construct a composite stability criterion based on the sideslip angle-slip angle velocity phase plane and the energy phase plane, obtain a composite stability factor through the composite stability criterion, and use the composite stability factor to characterize the vehicle's tendency to stabilize.

[0009] Step S3: Design a vehicle controller based on pole placement, establish a linear two-degree-of-freedom generalized dynamic model of the vehicle and construct a state feedback control law, obtain the closed-loop system equation through the state feedback control law, and determine the poles of the closed-loop characteristic equation as the control poles of the vehicle controller based on the closed-loop system equation.

[0010] Step S4: Use a reinforcement learning algorithm to make real-time decisions on the position of the control poles to obtain the dynamically adjusted desired poles, and substitute the desired poles into the closed-loop system equations to complete the feedback control of vehicle dynamic stability.

[0011] As a preferred embodiment, in step S1, the magic tire formula model is introduced to solve for the lateral forces of the front and rear axles, and the vertical loads of the front and rear axles and the front wheel slip angle are solved by a preset mechanical calculation formula.

[0012] As a preferred option, step S2, the process of constructing the composite stability criterion, includes:

[0013] Based on the aforementioned nonlinear two-degree-of-freedom vehicle model, the phase plane of the vehicle's center of mass sideslip angle and center of mass sideslip angular velocity under various working conditions is established. The stable region is identified by the bilinear method, a discrimination function is defined, and the normalized distance from the state point to the boundary line is calculated. The stable state of the vehicle in the phase plane is determined by the normalized distance.

[0014] A physical criterion based on energy evolution rate is introduced to define the total generalized kinetic energy, instantaneous power change rate, and energy index of the system. The energy index is used to determine the energy evolution trend of the vehicle system.

[0015] The composite stability factor is calculated by combining the normalized distance and the energy index with weighting coefficients.

[0016] As a preferred embodiment, in step S3, the vehicle's linear two-degree-of-freedom generalized dynamic model is a state-space equation. The additional front wheel steering angle, rear wheel steering angle, and additional yaw moment are uniformly incorporated into the control input vector of the state-space equation. Depending on the value of the control input vector, the vehicle controller is configured as an active front wheel steering controller, an active rear wheel steering controller, or a direct yaw moment controller.

[0017] As a preferred embodiment, in step S3, the position of the control pole determines the stability and dynamic response characteristics of the closed-loop system equation. The real part of the control pole reflects the convergence speed after the system state deviates from the equilibrium position. The imaginary part of the control pole determines the oscillation characteristics of the system response. The position of the control pole characterizes the stability margin of the vehicle through the damping ratio and the natural frequency.

[0018] As a preferred embodiment, step S4, which involves using a reinforcement learning algorithm to make real-time decisions about the location of the control poles, includes: defining the system state space, defining the system action space, designing a reward function, constructing a reinforcement learning model based on the system state space, the system action space, and the reward function, outputting the offset of the control poles through the reinforcement learning model, and obtaining candidate pole parameters from the offset.

[0019] As a preferred approach, when defining the system action space, the real and imaginary offsets of the control pole are constrained. The output of the real offset is compressed to a preset range using the hyperbolic tangent function to ensure that the control pole does not drift to the right half of the complex plane. The imaginary offset of the control pole is adjusted by the imaginary offset scaling factor.

[0020] As a preferred approach, after obtaining the candidate pole parameters, the candidate pole parameters are processed according to preset logic to obtain the desired pole.

[0021] The preset logic includes complex pole mode and real pole mode:

[0022] When the absolute value of the imaginary part of the candidate pole parameter is greater than the threshold, the complex pole mode is triggered, forcing the real parts of the candidate poles to be equal and the imaginary parts to be opposites of each other, forming a conjugate pair as the desired pole;

[0023] When the imaginary part of the candidate pole parameter approaches 0, the real pole mode is triggered, and the imaginary part of the candidate pole is set to zero, so that the two poles are independently distributed on the real axis as the desired poles.

[0024] As a preferred embodiment, the reward function is a dynamic weighted reward function based on the concept of a potential barrier function. The vehicle stability region is divided according to the magnitude of the composite stability factor. When the vehicle is in the stability region, a positive feedback reward is given. When the vehicle approaches the instability boundary, an exponential penalty is applied. At the same time, an additional penalty is applied to the behavior of the real part of the control pole approaching the preset stability margin.

[0025] As a preferred embodiment, the reinforcement learning algorithm is the DDPG algorithm. The DDPG algorithm adopts an Actor-Critic architecture and constructs a DDPG network model that includes a current policy network, a current value network, a target policy network, and a target value network. Through training steps of action selection and exploration, environmental interaction and reward calculation, and network update, the DDPG network model outputs the optimal control pole offset, thereby realizing the dynamic adjustment of the control pole.

[0026] As a preferred embodiment, during the training of the DDPG network model, random noise is superimposed on the actions output by the policy network to increase exploratory power. The vehicle state, actions, rewards, and the vehicle state at the next time step are stored in the experience replay pool. Samples are sampled from the experience replay pool to update the network parameters. The value network parameters are updated by gradient descent, and the policy network parameters are updated by gradient ascent.

[0027] As a preferred solution, after substituting the desired pole into the closed-loop system equation, the state feedback gain matrix is ​​solved online analytically using the characteristic polynomial matching method. The control continuity is achieved when the control pole changes, combined with the sigmoid-based switching function. At the same time, a safety monitoring mechanism including pole physical domain constraints and computation timeout backoff is introduced.

[0028] Beneficial Effects: This invention proposes a vehicle dynamic stability control method based on reinforcement learning and variable pole configuration. By constructing multi-dimensional criteria for the centroid sideslip angle phase plane and energy phase plane, it identifies the vehicle's stability state and energy evolution trend in real time. A reinforcement learning agent is used to make online decisions about the expected closed-loop pole positions of the system based on the current state, enabling the configuration of aggressive poles to improve flexibility under stable conditions and conservative poles to ensure stability under unstable conditions. The state feedback gain matrix is ​​solved online analytically using the characteristic polynomial matching method, and a sigmoid-based switching function is combined to ensure control continuity during pole changes. Furthermore, this invention designs a safety monitoring mechanism that includes pole physical domain constraints and computational timeout backoff, enabling the reshaping of the vehicle's dynamic characteristics through feedback control without altering the driver's steering intentions. This effectively solves the challenge of real-time trade-off between flexibility and safety in intelligent chassis control. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating the vehicle dynamic stability control method based on reinforcement learning and variable pole configuration of the present invention. Detailed Implementation

[0030] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0031] Researching a general chassis control architecture that balances the self-learning characteristics of artificial intelligence with the stability guarantees of classical control theory is of significant engineering value for improving the dynamic performance of future drive-by-wire chassis systems. Therefore, this embodiment discloses specific details of a vehicle dynamic stability control method based on reinforcement learning and variable pole configuration, see [link to details]. Figure 1 As shown.

[0032] S1: Establish a nonlinear two-degree-of-freedom vehicle model as the control reference, and its motion differential equation is:

[0033]

[0034] According to the equation of motion, we can obtain:

[0035]

[0036] In the formula, m is the total vehicle mass; a and b are the distances from the center of gravity to the front and rear axles; For the whole vehicle to be around Moment of inertia of the shaft; yaw rate is angular velocity; u is longitudinal velocity; It is the centroid sideslip angle; The steering angle of the car's front wheels; This is the yaw acceleration; The angular velocity of the center of mass deflection; This refers to the lateral force on the front axle. This is the lateral force on the rear axle.

[0037] To solve for the lateral forces on the front and rear axles, the magic tire formula model is introduced:

[0038]

[0039] In the formula, The road surface adhesion coefficient; , , These are the fitting coefficients for the front axle tires; , , These are the fitting coefficients for the rear axle tires; , These are the front and rear axle slip angles, respectively. , For the vertical loads on the front and rear axles.

[0040] The vertical loads on the front and rear axles are calculated as follows:

[0041]

[0042] The front wheel slip angle is calculated as follows:

[0043]

[0044] S2. Construct a composite criterion based on the centroid sideslip angle, the centroid sideslip angular velocity phase plane, and the energy phase plane:

[0045] Based on the aforementioned nonlinear two-degree-of-freedom vehicle model, a phase plane for the vehicle's sideslip angle and sideslip angular velocity under various operating conditions is established. The bilinear method is then used to determine the stable region, and its discriminant function is defined as follows:

[0046]

[0047] In the formula, It reflects the slope of the stability region; It reflects the intercept of the stability region.

[0048] Define the point to the boundary line normalized distance :

[0049]

[0050] when The state point is located within the stable region. When The state point has crossed the boundary, and the vehicle has skidded and lost control.

[0051] To capture the essence of vehicle instability under extreme conditions, this invention introduces a physical criterion based on the rate of energy evolution to define the total generalized kinetic energy of the system:

[0052]

[0053] Define the instantaneous power change rate P of the system as the time derivative of the energy function:

[0054]

[0055] Define energy index :

[0056] like The system's energy is dissipating and it is tending towards self-stabilization. .

[0057] like The system is accumulating energy, posing a risk of instability.

[0058]

[0059] That is the critical power point of the system.

[0060] This invention utilizes composite stability factors Ultimately, this determines the vehicle's stability tendency.

[0061]

[0062] in, and These are the weighting coefficients.

[0063] S3. Controller design based on pole placement:

[0064] To achieve coordinated control under different actuator configurations, this invention first establishes a linear two-degree-of-freedom generalized dynamic model of the vehicle. This model incorporates various possible control methods, such as additional front wheel steering angle, rear wheel steering angle, and additional yaw moment, into the control input vector.

[0065] The state-space equation of the generalized system is expressed as:

[0066]

[0067] The system state vector is as follows:

[0068]

[0069]

[0070]

[0071] in, , These are the front and rear axle lateral stiffness, respectively; This is a system disturbance.

[0072] To cover multiple functions, a generalized control input vector is defined. :

[0073]

[0074] When the control vector is taken At this time, the controller is active front wheel steering control (AFS), and matrix B is as follows:

[0075]

[0076] When the control vector is taken At this time, the controller is active rear-wheel steering (ARS), and the B matrix is ​​as follows:

[0077]

[0078] When the control vector is taken At this time, the controller is a direct yaw moment control (DYC), and the B matrix is ​​as follows:

[0079]

[0080] For the established linear two-degree-of-freedom generalized dynamics model of the vehicle, design a state feedback control law. Where K is the feedback gain matrix; For generalized control output, such as additional front wheel steering angle, rear wheel steering angle, and additional yaw moment.

[0081] To perform pole placement and analytically solve for the feedback gain K, we consider the perturbation as an external input that does not participate in the construction of the closed-loop characteristic polynomial. Therefore, when solving for the closed-loop eigenvalues, we set... Then the control law Substitute into the system state-space equations The closed-loop system equations are obtained as follows:

[0082]

[0083] Its stability and dynamic response characteristics are determined by the roots, i.e., the poles, of the closed-loop characteristic equation:

[0084]

[0085] Represents the determinant of a matrix.

[0086] This invention designs the desired poles This allows for the precise definition of the vehicle's dynamic behavior. The location of the poles on the complex plane has a clear physical meaning.

[0087] Real part of the pole This reflects the convergence speed of the system after it deviates from its equilibrium position. The more negative the real part, the faster the vehicle returns to a steady state in terms of sideslip angle and yaw rate after being disturbed or given a steering command.

[0088] Imaginary part of the pole The imaginary part determines the oscillatory characteristics of the system response. The larger the imaginary part, the more sensitive the vehicle's steering response; however, an excessively large imaginary part can cause unexpected swaying or overshooting of the vehicle body.

[0089] Damping ratio With natural frequency The pole position can be further converted into the damping ratio. ,in The angle between the pole vector and the negative real axis. It determines the vehicle's stability margin and prevents skidding and fishtailing.

[0090] S4. Reinforcement learning-driven dynamic pole adjustment strategy:

[0091] This invention employs a reinforcement learning algorithm to make real-time decisions on the location of poles.

[0092] S4.1 Define the system state space as follows:

[0093]

[0094] S4.2, Define the system action space as follows:

[0095]

[0096] In the formula, , Used to adjust the convergence damping of the system. , Used to adjust the system's response frequency;

[0097] To ensure that the closed-loop poles of the system will never drift to the right half of the complex plane, the real part offset is... :

[0098]

[0099] In the formula, The hyperbolic tangent function forces the network output to be compressed to... Interval. The absolute value of the real part of the preset reference pole represents the basic stability of the system. This is a stability margin constant, ensuring that the real part of the poles remains at least [distance] from the imaginary axis even under maximum offset. The physical distance. This represents two extreme points.

[0100] Imaginary part offset :

[0101]

[0102] In the formula, This is the scaling factor for the imaginary part offset.

[0103] First, using the mapped real part offset offset from the imaginary part The candidate pole parameters are obtained:

[0104]

[0105] To ensure that the feedback gain K is a real number, the following logic is executed on the mapped poles:

[0106] Logic A (complex pole mode): If and If the absolute value is greater than the threshold, then the real parts are forced to be equal. Furthermore, their imaginary parts are opposites of each other, forming conjugate pairs.

[0107] when At this point, the system determines that the current operating condition allows or requires a certain frequency of dynamic response. To ensure that the feedback gain matrix K is a real number, the poles must be forced to satisfy the conjugate symmetry relationship:

[0108]

[0109]

[0110] Ultimate pole pair:

[0111]

[0112] Logic B (Real Pole Mode): If the imaginary part approaches 0, then it is allowed. This allows the system to be configured with two distinct real roots under extremely stable conditions, enabling extremely fast and oscillatory convergence.

[0113] when At this point, the system determines that it is currently in an extremely dangerous operating condition or has an extreme requirement for stability. The system discards the imaginary part, allowing the two poles to be independently distributed on the real axis.

[0114]

[0115] Set the imaginary part to zero:

[0116]

[0117] Ultimate pole pair:

[0118]

[0119] Through logic A and logic B, the actions of RL are transformed into specific coordinates on the complex plane.

[0120] Obtain the specific coordinates of the desired closed-loop poles. Then, the order was given:

[0121]

[0122] The closed-loop characteristic polynomial can then be written as:

[0123]

[0124] In the formula Represents the trace of a matrix.

[0125] And from the expected extreme point The constructed target characteristic polynomial is:

[0126]

[0127] By matching the coefficients of the characteristic polynomial, we have:

[0128]

[0129] You can get information about The system of linear equations:

[0130]

[0131]

[0132] Solving equations (39) to (40) yields the control gain matrix. This enables online analytical calculation of the control gain from the desired closed-loop poles. The control law is updated.

[0133] To avoid abrupt changes in control gain, a sigmoid smoothing switching function is introduced. The smoothing weights are defined as follows:

[0134]

[0135] in, To switch the steepness coefficient, This indicates the trigger time for the switch, and k represents the current time.

[0136] The control gain smoothing gain update is then:

[0137]

[0138] In the formula, The gain is controlled for the previous moment.

[0139] Therefore, the control input is updated as follows:

[0140]

[0141] S4.3 Design the reward function:

[0142]

[0143] To guide reinforcement learning agents to pursue sensitivity within safe zones and absolute stability at dangerous boundaries, a dynamic weighted reward mechanism based on the concept of a potential barrier function is designed:

[0144]

[0145] In the formula, The total reward is based on dynamic weighting. This is a positive reward item; This is a penalty term for the composite stability factor; This is a penalty term for extreme point constraints; is the real part of the pole.

[0146] When the vehicle is in a stable region, the system provides continuous positive feedback to encourage the vehicle to stay centered in the phase plane:

[0147]

[0148] In the formula, This is the gain coefficient; This is the threshold for the composite stability factor.

[0149] When a vehicle approaches the boundary of the phase plane or energy begins to accumulate abnormally, i.e. To increase the penalty, it must grow exponentially:

[0150]

[0151] In the formula, Based on the benchmark for punishment.

[0152] To align with the aforementioned action mapping logic, the real part of the pole is approached with the stability margin. Additional penalties will be imposed for such behavior:

[0153]

[0154] In the formula, For pole-constrained gain; To maintain a safe distance; Let be the real part of the i-th pole.

[0155] The following details an embodiment of dynamic pole placement control for vehicle stability based on DDPG. In this embodiment, the DDPG algorithm employs an Actor-Critic (policy-value) architecture to determine the optimal pole offset within a continuous action space.

[0156] Step A1: Construct the DDPG network model:

[0157] The DDPG algorithm contains a total of four neural networks, divided into two groups: the current network and the target network. Each group contains an Actor network (policy network) and a Critic network (value network).

[0158] Step A1.1: Current policy network: Input is the system state vector S, output is a 4-dimensional continuous action vector The network terminal uses the tanh activation function to restrict the output to... Within the interval, it is then mapped to the actual real part offset using formulas (24) and (25). and imaginary part offset Initialize the current policy network parameter set. .

[0159] Step A1.2: Current value network: Inputs are state S and the actions output by the Actor. Output as value This is used to evaluate the contribution of the current pole placement scheme to system stability; and to initialize the value network parameter set. .

[0160] Step A1.3: The target policy network is constructed identically to the policy network and is responsible for replaying the next state sampled from the experience pool. Select the optimal next strategy ; Set the target network parameters ;in The initial target policy network parameters, These are the initial policy network parameters;

[0161] Step A1.4: The target value network is constructed identically to the value network and is responsible for calculating the target. value;

[0162] Let the target network parameters ,in The initial target value network parameters, These are the initial value network parameters.

[0163] The training implementation steps based on DDPG are as follows:

[0164] Step A2, Action Selection and Exploration:

[0165] The Actor network determines the current vehicle status. Calculate actions To increase the exploratory nature of the process, random noise is superimposed on the output action.

[0166]

[0167] In the formula, Let t represent the vehicle state at time t. Here are the policy network parameters at time t; This is motion noise.

[0168] Step A3, Environmental Interaction and Reward Calculation:

[0169] According to the action The final target pole is determined by formula (32). The state feedback gain matrix K is solved analytically, and control input is applied. Calculate the dynamic weighted reward function R, and calculate the composite stability factor. This factor incorporates the phase plane boundary distance. and energy dissipation rate The comprehensive reward is calculated according to formula (42) if the vehicle is in a stable area. Give positive rewards If the area approaches the instability boundary, an exponential penalty is imposed. Ultimately, they will be rewarded. Depending on the vehicle's condition Value gained And the vehicle status at the next moment. Storing datasets .

[0170] Step A4, Network Update:

[0171] A batch of samples is collected from the playback pool, and the target is calculated:

[0172]

[0173] in, The value function for the next time step. This is the discount factor. For policy networks.

[0174] Define the loss function to minimize:

[0175]

[0176] In the formula, For value networks;

[0177] Calculate the gradient:

[0178]

[0179] This is the current projected value. This serves as a reference value for the target price.

[0180] Updating value network parameters using gradient descent :

[0181]

[0182]

[0183] in, For the value network learning rate, Let be the network parameters at time t. The network parameters at time t+1 are the value parameters. The network parameters represent the target value at time t+1;

[0184] Update via gradient ascent :

[0185]

[0186]

[0187] in, For the policy network learning rate, Let be the policy network parameters at time t. The network parameters for the policy at time t+1. The network parameters for the target policy at time t+1.

[0188] Through continuous training of the DDPG algorithm, the system can automatically switch between "complex pole mode (mode A)" and "real pole mode (mode B)" according to changes in vehicle state. Under non-steady-state conditions, DDPG can quickly increase the real part offset, giving the system a very strong convergence speed and preventing sideslip and fishtailing. During stable driving, DDPG optimizes the imaginary part offset, improving the sensitivity of steering response, while ensuring that the closed-loop pole does not drift to the right half of the complex plane.

[0189] The operating mechanisms of the above-disclosed embodiments can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0190] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor.

[0191] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0192] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A vehicle dynamic stability control method based on reinforcement learning and variable pole placement, characterized in that, Includes the following steps: Step S1: Establish a nonlinear two-degree-of-freedom vehicle model as the control reference, and solve for the lateral forces, vertical loads, and front wheel slip angles of the vehicle's front and rear axles based on the nonlinear two-degree-of-freedom vehicle model. Step S2: Construct a composite stability criterion based on the sideslip angle-slip angle velocity phase plane and the energy phase plane, obtain a composite stability factor through the composite stability criterion, and use the composite stability factor to characterize the vehicle's tendency to stabilize. The process of constructing the composite stability criterion includes: Based on the aforementioned nonlinear two-degree-of-freedom vehicle model, the phase plane of the vehicle's center of mass sideslip angle and center of mass sideslip angular velocity under various working conditions is established. The stable region is identified by the bilinear method, a discrimination function is defined, and the normalized distance from the state point to the boundary line is calculated. The stable state of the vehicle in the phase plane is determined by the normalized distance. A physical criterion based on energy evolution rate is introduced to define the total generalized kinetic energy, instantaneous power change rate, and energy index of the system. The energy index is used to determine the energy evolution trend of the vehicle system. The composite stability factor is calculated by combining the normalized distance and the energy index with weighting coefficients. Step S3: Design a vehicle controller based on pole placement, establish a linear two-degree-of-freedom generalized dynamic model of the vehicle and construct a state feedback control law, obtain the closed-loop system equation through the state feedback control law, and determine the poles of the closed-loop characteristic equation as the control poles of the vehicle controller based on the closed-loop system equation. The vehicle's linear two-degree-of-freedom generalized dynamic model is a state-space equation. The additional front wheel steering angle, rear wheel steering angle, and additional yaw moment are uniformly incorporated into the control input vector of the state-space equation. Depending on the value of the control input vector, the vehicle controller is made to correspond to an active front wheel steering controller, an active rear wheel steering controller, or a direct yaw moment controller. The location of the control pole determines the stability and dynamic response characteristics of the closed-loop system equation. The real part of the control pole reflects the convergence speed after the system state deviates from the equilibrium position. The imaginary part of the control pole determines the oscillation characteristics of the system response. The location of the control pole characterizes the vehicle's stability margin through the damping ratio and natural frequency. Step S4: Use a reinforcement learning algorithm to make real-time decisions on the position of the control poles to obtain the dynamically adjusted desired poles, and substitute the desired poles into the closed-loop system equations to complete the feedback control of vehicle dynamic stability.

2. The vehicle dynamic stability control method based on reinforcement learning and variable pole placement according to claim 1, characterized in that, In step S1, the magic tire formula model is introduced to solve the lateral forces of the front and rear axles, and the vertical loads of the front and rear axles and the front wheel slip angle are solved by the preset mechanical calculation formula.

3. The vehicle dynamic stability control method based on reinforcement learning and variable pole placement according to claim 1, characterized in that, In step S4, the process of making real-time decisions on the position of the control pole using a reinforcement learning algorithm includes: defining the system state space, defining the system action space, designing a reward function, constructing a reinforcement learning model based on the system state space, the system action space, and the reward function, outputting the offset of the control pole through the reinforcement learning model, and obtaining candidate pole parameters from the offset.

4. The vehicle dynamic stability control method based on reinforcement learning and variable pole placement according to claim 3, characterized in that, When defining the system action space, the real and imaginary offsets of the control pole are constrained. The output of the real offset is compressed to a preset range by the hyperbolic tangent function to ensure that the control pole does not drift to the right half of the complex plane. The imaginary offset of the control pole is adjusted by the imaginary offset scaling factor.

5. The vehicle dynamic stability control method based on reinforcement learning and variable pole placement according to claim 3, characterized in that, After obtaining the candidate pole parameters, the candidate pole parameters are processed according to preset logic to obtain the desired pole; The preset logic includes complex pole mode and real pole mode: When the absolute value of the imaginary part of the candidate pole parameter is greater than the threshold, the complex pole mode is triggered, forcing the real parts of the candidate poles to be equal and the imaginary parts to be opposites of each other, forming a conjugate pair as the desired pole; When the imaginary part of the candidate pole parameter approaches 0, the real pole mode is triggered, and the imaginary part of the candidate pole is set to zero, so that the two poles are independently distributed on the real axis as the desired poles.

6. The vehicle dynamic stability control method based on reinforcement learning and variable pole placement according to claim 3, characterized in that, The reward function is a dynamic weighted reward function based on the idea of ​​the potential barrier function. The vehicle stability region is divided according to the magnitude of the composite stability factor. When the vehicle is in the stability region, a positive feedback reward is given. When the vehicle approaches the instability boundary, an exponential penalty is applied. At the same time, an additional penalty is applied to the behavior of the real part of the control pole approaching the preset stability margin.

7. The vehicle dynamic stability control method based on reinforcement learning and variable pole placement according to claim 1, characterized in that, The reinforcement learning algorithm is the DDPG algorithm, which adopts an Actor-Critic architecture. It constructs a DDPG network model that includes a current policy network, a current value network, a target policy network, and a target value network. Through training steps such as action selection and exploration, environmental interaction and reward calculation, and network update, the DDPG network model outputs the optimal control pole offset, thereby realizing the dynamic adjustment of the control pole.

Images