A safety-reinforced learning vehicle path tracking control method integrated with active roll

Abstract

This invention discloses a safety reinforcement learning-based vehicle path tracking control method integrating active tilt control, comprising: acquiring vehicle path tracking and tilt state information; constructing a SAC reinforcement learning controller integrating active tilt control, generating candidate front wheel steering angles and candidate active tilt control forces; constructing a model prediction safety filter layer to perform short-time domain state prediction on candidate actions; constructing a dual-input safety channel of front wheel steering angle and active tilt control force, comparing and verifying the predicted state of candidate actions with safety constraints; directly outputting when a candidate action is safe, and online optimizing to find a safe control action that satisfies safety constraints and has the smallest deviation from the candidate action when it is unsafe. This invention leverages the nonlinear decision-making capability of the SAC reinforcement learning controller while ensuring vehicle input constraints, vehicle tilt constraints, and lateral stability constraints through a model prediction safety filter layer, thereby improving the safety and real-time performance of high-speed path tracking.

Description

Technical Field

[0001] This invention belongs to the field of intelligent vehicle motion control technology, specifically a vehicle path tracking control method integrating active tilt control, reinforcement learning control, and model prediction safety filtering. It can be used for lateral path tracking, vehicle attitude adjustment, and stability constraint control of autonomous vehicles in conditions such as high-speed lane changes, obstacle avoidance in double lane changes, and driving on curved paths. This method generates vehicle control actions through reinforcement learning and combines a model prediction safety filtering layer to perform safety checks and corrections on the front wheel steering angle and active tilt control force, thereby improving vehicle path tracking accuracy and high-speed driving stability. Background Technology

[0002] With the development of autonomous driving technology, vehicle path tracking control plays a crucial role in scenarios such as high-speed lane changes, emergency obstacle avoidance, and driving on curved paths. The goal of path tracking control is to enable the vehicle to drive stably along the reference path generated by the planning module, while ensuring the vehicle's lateral stability and driving safety during the tracking process. Under high-speed conditions, vehicles are affected by factors such as tire lateral force nonlinearity, actuator physical constraints, vehicle roll response, and lateral load transfer, which can easily lead to problems such as increased lateral tracking error, severe yaw response, excessive yaw angle, and reduced stability margin. Therefore, vehicle path tracking control not only needs to improve tracking accuracy but also needs to simultaneously consider control input constraints, vehicle state constraints, and roll stability constraints.

[0003] Existing model predictive control (MMCC) methods can predict the future state of a vehicle within a finite prediction time domain and explicitly consider constraints such as front wheel steering angle, steering angle change rate, lateral error, and vehicle stability during the optimization process, thus exhibiting good constraint handling capabilities in vehicle path tracking control. However, these methods are typically highly dependent on the accuracy of the vehicle prediction model. When the vehicle is in complex conditions such as high-speed obstacle avoidance, high-curvature path tracking, or significant roll response, the nonlinearity of vehicle dynamics and model uncertainties increase the difficulty of controller design. Furthermore, MMCC requires solving the optimization problem within each sampling period; if the prediction model and constraints are complex, it can easily increase the online computational burden and affect the real-time performance of the control system.

[0004] Reinforcement learning methods can autonomously learn control strategies through interaction between the agent and the environment, exhibiting strong nonlinear mapping and adaptive control capabilities. For vehicle path tracking tasks, reinforcement learning controllers can directly output control actions based on the vehicle's current state. After training, the control input can be obtained through forward inference via the policy network, demonstrating good online decision-making efficiency. In particular, the SAC algorithm, by introducing a maximum entropy mechanism, can improve policy exploration capabilities and training stability, making it suitable for vehicle path tracking control under continuous state and continuous action conditions. However, reinforcement learning methods primarily rely on reward functions and termination conditions to indirectly constrain control behavior. In the early stages of training, before the policy converges, the agent may output control actions with excessively large amplitudes or rapid changes, causing the vehicle to deviate from the safe zone and even triggering safety risks such as severe yaw response, excessive yaw angle, or increased lateral load transfer rate. Therefore, relying solely on reinforcement learning methods makes it difficult to explicitly guarantee the satisfaction of vehicle input and state constraints in each control cycle.

[0005] Furthermore, during high-speed path tracking, the impact of vehicle roll motion on driving stability becomes more pronounced. When the front wheel angle is large or the path curvature changes rapidly, the vehicle generates significant lateral acceleration, causing body roll and redistributing the vertical load between the left and right wheels. When the lateral load transfer rate is too high, the vehicle may approach the rollover boundary, affecting safety during high-speed obstacle avoidance. Active roll control can improve vehicle stability during high-speed path tracking by applying active suspension to dynamically adjust the vehicle's attitude, reducing roll angle error and lateral load transfer risk. However, introducing active roll control into reinforcement learning path tracking expands the control action from a single front wheel angle to two control variables: front wheel angle and active roll control force. While this increases the degree of control freedom, it also complicates the coupling relationship between control inputs. Simply limiting the amplitude of each control variable individually is insufficient to guarantee that the combined effect will still meet vehicle roll stability and lateral safety constraints.

[0006] Therefore, a safety reinforcement learning vehicle path tracking control method integrating active tilt control needs to be proposed. While leveraging the nonlinear decision-making capability and online inference efficiency of the SAC reinforcement learning controller, a model prediction safety filtering mechanism is introduced to perform short-time domain prediction, constraint verification, and minimum necessary correction on the front wheel steering angle and active tilt control force output by reinforcement learning. This ensures that the final control action applied to the vehicle retains the control intent of the reinforcement learning controller as much as possible while satisfying vehicle input constraints, vehicle body tilt constraints, and lateral load transfer rate constraints, thereby improving the tracking accuracy, real-time control, and driving safety of the vehicle during high-speed path tracking and obstacle avoidance. Summary of the Invention

[0007] The technical problem this invention aims to solve is the shortcomings of existing vehicle path tracking control methods in conditions such as high-speed lane changes, double lane change obstacle avoidance, and curved path driving. These problems include insufficient control safety, high risk of action exploration in the early stages of reinforcement learning training, difficulty in coordinating the coupling relationship between active tilt control and steering control, and difficulty in explicitly guaranteeing vehicle roll stability. This invention provides a safety reinforcement learning vehicle path tracking control method integrating active tilt. This method generates candidate actions for vehicle path tracking through a reinforcement learning controller and sets up a model prediction safety filter layer between the reinforcement learning controller and the vehicle actuators. It performs short-time domain prediction, constraint verification, and minimum necessary correction on the candidate front wheel steering angle and active tilt control force, ensuring that the final control action applied to the vehicle satisfies vehicle input constraints, body roll constraints, and lateral stability constraints while preserving the reinforcement learning control intent as much as possible. This improves the tracking accuracy, control real-time performance, and driving safety of the vehicle during high-speed path tracking.

[0008] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0009] A safety reinforcement learning-based vehicle path tracking control method integrating active tilting includes the following steps:

[0010] Step S1: Collect the vehicle's current driving status and reference path information, and calculate the vehicle path tracking and tilting status information based on the vehicle's current driving status and reference path information;

[0011] The vehicle path tracking and tilt status information includes the vehicle's lateral error, heading error, lateral error change rate, heading error change rate, current front wheel angle, front wheel angle change rate, error between the actual tilt angle and the target tilt angle, and vehicle tilt angular velocity.

[0012] The reference path information includes reference path position, reference heading angle, and reference curvature, which are used to assist in calculating vehicle path tracking and tilting state information; and to provide prediction input for subsequent model prediction of the safety filter layer.

[0013] Step S2: Using the vehicle path tracking and tilt state information obtained in step S1 as the input state of the SAC reinforcement learning agent, and the front wheel steering angle and active tilt control force as the action output of the SAC reinforcement learning agent, construct the SAC reinforcement learning controller.

[0014] Step S3: Input the current vehicle path tracking and tilt state information obtained in step S1 into the trained SAC reinforcement learning controller, and output candidate control actions, including candidate front wheel steering angle and candidate active tilt control force.

[0015] Step S4: Set up a model prediction safety filter layer between the SAC reinforcement learning controller and the vehicle actuator. Using the candidate front wheel steering angle and candidate active yaw control force output in step S3 as reference inputs, and combining the current vehicle state and the vehicle prediction model, predict the changes in vehicle state and control input caused by the candidate control action in the short time domain in the future, and obtain the predicted vehicle state and predicted control input sequence in the future prediction time domain.

[0016] The current vehicle status includes at least the vehicle's lateral error, heading error, lateral velocity, yaw rate, body roll angle, and roll rate.

[0017] Step S5: Based on the vehicle's lateral dynamics model, roll dynamics model, and lateral load transfer relationship, construct a two-dimensional coupled safe feasible domain, i.e., a dual-input safe channel, which is jointly determined by the front wheel steering angle and the active roll control force under the current vehicle state. The dual-input safe channel is a set of all control actions that simultaneously satisfy the safety constraints under the current vehicle state: the safety constraints include vehicle input constraints, input rate of change constraints, vehicle state constraints, and lateral stability constraints; the control actions are composed of the safe front wheel steering angle and the safe active roll control force.

[0018] Within the dual-input safety channel, the current candidate front wheel steering angle and the candidate active yaw control force satisfy the vehicle input constraints and the rate of change of input constraints, and the combination of the front wheel steering angle and the active yaw control force will not cause any vehicle state constraints or lateral stability constraints to be violated in the prediction time domain.

[0019] Step S6: Compare the candidate front wheel steering angle and candidate active yaw control force at the current moment, as well as the predicted vehicle state and predicted control input sequence obtained in step S4, with the safety constraints described in step S5 one by one.

[0020] If the candidate front wheel steering angle and candidate active sway control force satisfy the vehicle input constraints and input rate of change constraints, and all predicted vehicle states and predicted control input sequences in the prediction time domain satisfy the safety constraints, then the candidate control action output in step S3 is located in the dual-input safety channel in step S5. The candidate control action is not modified, and the candidate front wheel steering angle and candidate active sway control force are directly output as the final control action.

[0021] If the candidate front wheel steering angle or candidate active sway control force does not meet the vehicle input constraints and input rate of change constraints, or if any predicted vehicle state or predicted control input sequence in the prediction time domain does not meet the safety constraints, then the candidate control action output in step S3 is determined to be not located in the dual-input safety channel described in step S5. The safety control action that is closest to the candidate control action and meets the safety constraints is solved through online optimization, and the corrected safety front wheel steering angle and safety active sway control force are output as the final control action.

[0022] Step S7: Apply the final control action output in step S6 to the vehicle actuator;

[0023] In the next sampling period, the current driving state of the vehicle is reacquired and steps S1 to S7 are repeated to form a rolling updated safety reinforcement learning path tracking control process.

[0024] As a further improvement to the present invention, the construction of the SAC reinforcement learning controller in step S2 specifically includes:

[0025] Step S2.1: Construct the SAC strategy network. The SAC strategy network takes the vehicle path tracking and tilt state information described in step S1 as input, outputs the normalized front wheel steering angle and the normalized active tilt control force, and converts them into actual candidate front wheel steering angle and candidate active tilt control force through proportional mapping.

[0026] Step S2.2: Construct a SAC dual-critic network, which includes a first critic network and a second critic network. The first critic network and the second critic network have the same structure but their network parameters are independent of each other. They are used to evaluate the comprehensive action value of performing a dual-input action under the current vehicle state. The smaller value between the output value of the first critic network and the output value of the second critic network is used as the action value estimation result to reduce the problem of overestimation of action value.

[0027] Step 2.3: Construct the reward function, which is as follows:

[0028] ;

[0029] in, For lateral error, For heading error, This represents the rate of change of the lateral error. The rate of change of heading error. This represents the error between the actual sway angle and the target sway angle of the vehicle body. For the maximum tilt angle, The angular velocity of the vehicle body tilt. For the current front wheel steering angle, This is the upper limit of the front wheel steering angle. For the current active tilt control force, This is the upper limit of the active tilt control force. As an additional penalty, an additional penalty is triggered when the absolute value of the lateral error exceeds the lateral error safety threshold, or the absolute value of the vehicle body sway angle exceeds the sway angle safety threshold, or the absolute value of the lateral load transfer rate exceeds the lateral load transfer rate safety threshold; otherwise, no additional penalty is applied.

[0030] As a further improvement of the present invention, the model prediction security filtering layer in step S4 includes:

[0031] Step S4.1: Establish a vehicle prediction model: Based on the vehicle's lateral dynamics model, path error model, and roll dynamics model, establish a vehicle prediction model for short-time-domain state prediction. Using the current vehicle state and candidate control actions as inputs, recursively calculate the predicted vehicle state and predicted control input sequence for several future sampling times within a finite prediction time domain.

[0032] The predicted vehicle state includes the vehicle's lateral error, heading error, lateral velocity, yaw rate, body roll angle, roll rate, and lateral load transfer rate.

[0033] The predictive control input sequence includes the front wheel steering angle and active yaw control force for each future prediction step;

[0034] Step S4.2: Construct safety constraints, which include at least the following constraints: front wheel steering angle amplitude constraint, front wheel steering angle change rate constraint, active yaw control force amplitude constraint, active yaw control force change rate constraint, lateral error constraint, heading error constraint, lateral velocity constraint, yaw rate constraint, vehicle body yaw angle constraint, yaw rate constraint, and lateral load transfer rate constraint.

[0035] Step S4.3: The candidate front wheel steering angle and candidate active yaw control force output by the SAC reinforcement learning controller are used as safety verification objects and substituted into the safety filter layer prediction model established in step S4.1. The future state of the vehicle and the control input are recursively calculated within the finite prediction time domain to obtain the predicted vehicle state and predicted control input sequence.

[0036] As a further improved technical solution of the present invention, the vehicle input constraints in step S5 specifically include front wheel steering angle amplitude constraints and active yaw control force amplitude constraints; the input rate of change constraints specifically include front wheel steering angle rate of change constraints and active yaw control force rate of change constraints; the vehicle state constraints specifically include lateral error constraints, heading error constraints, lateral velocity constraints, yaw rate constraints, vehicle body yaw angle constraints, and yaw rate constraints; and the lateral stability constraints specifically include lateral load transfer rate constraints.

[0037] As a further improvement to the present invention, step S6, which involves online optimization to find the safety control action that is closest to the candidate control action and meets the safety constraints, specifically includes:

[0038] Step 6.1: Construct safety action correction and optimization goals:

[0039] With the goal of making the corrected front wheel steering angle and active yaw control force as close as possible to the candidate front wheel steering angle and candidate active yaw control force output by the SAC reinforcement learning controller, a safety action correction optimization objective function is constructed.

[0040] The objective function for safety action correction and optimization is:

[0041] ;

[0042] Where J is the objective function; D u This is a normalization matrix for the motion amplitude, used to unify the dimensions of the front wheel steering angle and the active roll control force; 𝑘+j|𝑘 The control input for the j-th step predicted at the current time k includes the front wheel steering angle and active yaw control force; RL (k) represents the candidate control action output by the SAC reinforcement learning controller at the current time k; D 𝛥𝑢 To control the incremental normalization matrix, used to unify the dimensions of the front wheel steering angle change and the active roll control force change; 𝛥𝑢 𝑘+j|𝑘 For the change in control input in the j-th step in the future; N c To control the length of the time domain, let represent the number of control input steps that need to be solved in the online optimization; where k represents the current sampling time, and j represents the j-th control step in the control time domain. This is the action deviation weight matrix. To control the incremental weight matrix, As slack variables, The penalty coefficient for slack variables;

[0043] Step 6.2: Construct a constrained quadratic programming problem:

[0044] Using vehicle safety constraints as optimization constraints and the safety action correction optimization objective described in step 6.1 as the optimization objective, a constrained quadratic programming problem is constructed.

[0045] Step 6.3, Solve for safety control actions:

[0046] By solving the quadratic programming problem online, the optimal control sequence that satisfies the vehicle safety constraints and minimizes the deviation from the candidate action output by the SAC reinforcement learning controller is obtained in the prediction time domain at the current time k.

[0047] The optimal control sequence includes multiple sets of control inputs in the current time and several future control time domains. Each set of control inputs consists of the front wheel steering angle and the active tilt control force.

[0048] Step 6.4: Output the final control action:

[0049] The first set of control inputs in the optimal control sequence obtained in step 6.3 is the safety control action at the current moment, including the safe front wheel steering angle and the safe active sway control force. The safe front wheel steering angle and the safe active sway control force are used as the final control action.

[0050] The beneficial effects of this invention are as follows:

[0051] 1. To address the issues of existing model predictive control methods' heavy reliance on the accuracy of vehicle prediction models and significant online computational burden, this invention employs a SAC reinforcement learning controller to generate candidate control actions. After training, the control input can be obtained through forward inference via a policy network, demonstrating good online decision-making efficiency. Furthermore, this invention combines the nonlinear decision-making capability of the reinforcement learning controller with a model predictive safety filtering mechanism. While leveraging the strong nonlinear mapping and adaptive control capabilities of the SAC algorithm, a safety filtering layer constrains and verifies candidate actions, balancing control real-time performance and safety.

[0052] 2. To address the problem that relying solely on reinforcement learning methods makes it difficult to explicitly guarantee the satisfaction of vehicle input and state constraints within each control cycle, this invention establishes a model prediction safety filter layer between the reinforcement learning controller and the vehicle actuators. This safety filter layer uses candidate front wheel steering angles and candidate active yaw control forces as reference inputs, and the current vehicle state as the initial prediction state. It combines the vehicle prediction model to predict the changes in vehicle state and control input caused by candidate control actions in the short future time domain. When a candidate action satisfies the safety constraints, the safety filter layer is non-interventional, and the final control action equals the candidate action output by the SAC policy network. When a candidate action may lead to a violation of safety constraints, the safety filter layer performs minimum necessary corrections through online optimization, ensuring that the final control action satisfies the safety constraints while preserving as much of the original control intent of the SAC controller as possible. This mechanism effectively solves the problem of the agent outputting high-risk actions in the early stages of reinforcement learning training when the policy has not yet converged.

[0053] 3. To address the issues of complex control input coupling and the difficulty in ensuring combined safety with simple amplitude limiting after introducing reinforcement learning path tracking in active roll control, this invention constructs a dual-input safety channel of front wheel steering angle and active roll control force. This safety channel is not a simple combination of the independent safety ranges of the front wheel steering angle and the active roll control force, but rather a joint safe feasible region formed after considering their combined effect on the vehicle's dynamic response. Within the dual-input safety channel, the combination of front wheel steering angle and active roll control force will not cause lateral error, heading error, yaw rate, vehicle roll angle, or lateral load transfer rate to exceed safety constraints in the prediction time domain. Through the construction of this two-dimensional coupled safe feasible region, this invention effectively coordinates the coupling relationship between steering control and active roll control, avoiding safety vulnerabilities caused by simple amplitude limiting.

[0054] 4. To address the significant impact of vehicle roll motion on high-speed stability and the potential for rollover due to excessive lateral load transfer rate, this invention introduces active roll control force as an independent control variable into the path tracking control framework. An equivalent roll control torque is generated through an active suspension system or active roll actuator to actively adjust the vehicle roll response and lateral load transfer state. The safety constraints explicitly include vehicle roll angle constraints, roll angular velocity constraints, and lateral load transfer rate constraints, enabling early prediction and active limitation of vehicle roll attitude and roll stability risks in the prediction time domain. Simulation results show that this invention effectively reduces vehicle roll angle errors and lateral load transfer risks while maintaining good path tracking performance, improving vehicle attitude stability and roll safety during high-speed path tracking.

[0055] 5. To address the issue of high online computational burden in model predictive control, the model predictive safety filter layer in this invention only performs safety checks and necessary corrections on the candidate actions output by reinforcement learning, rather than completely replacing the reinforcement learning controller for path tracking decisions. The safety filter layer employs short-time domain prediction and rolling time-domain control, using only the first set of control inputs from the solved optimal control sequence as the actual safe control action for the current time step, effectively reducing the computational load of online optimization while ensuring safety. Furthermore, the optimization objective of the safety filter layer is to minimize the deviation from the candidate actions rather than completely regenerating the control actions, further reducing the computational burden. This architecture enables this invention to explicitly satisfy vehicle input constraints, vehicle roll constraints, and lateral stability constraints while maintaining high control real-time performance.

[0056] 6. This invention is applicable to various operating conditions such as high-speed double lane change, emergency lane change, curve path tracking, and obstacle avoidance path tracking. During vehicle path tracking, it simultaneously achieves lateral path error control, vehicle tilt attitude control, and lateral load transfer risk suppression. Simulation results show that the Safe-SAC active tilt method proposed in this invention, while maintaining the nonlinear decision-making capability of the SAC controller, uses a model prediction safety filter layer to perform safety verification and minimum correction on candidate front wheel steering angles and active tilt control forces. This allows the vehicle's lateral displacement to better follow the desired path, the front wheel steering angle to be smoother, and the vehicle tilt angle error and lateral load transfer rate to be effectively suppressed. Attached Figure Description

[0057] Figure 1 This is a diagram of the vehicle's kinematics model.

[0058] Figure 2 This is a flowchart illustrating the overall process of the safety reinforcement learning vehicle path tracking control method integrating active tilting, as described in this invention.

[0059] Figure 3 This is a structural diagram of the Safe-SAC path tracking control system that integrates active tilt control in this invention.

[0060] Figure 4 This is a schematic diagram illustrating the working principle of the SAC reinforcement learning controller.

[0061] Figure 5 This is a schematic diagram illustrating the working principle of the model prediction security filtering layer.

[0062] in, Figure 5 Figure (a) in the diagram represents the output / state prediction channel. Figure 5 Figure (b) in the diagram illustrates the input security channel and the current action correction process.

[0063] Figure 6 This is a schematic diagram of a dual-input safety channel for front wheel steering angle and active tilt control force.

[0064] Figure 7 A diagram illustrating the non-intervention approach when candidate actions are safe.

[0065] Figure 8 This is a schematic diagram of projection correction when the candidate action is unsafe.

[0066] Figure 9 This is a comparison diagram of lateral displacement using different control methods in this invention.

[0067] Figure 10 This is a comparison chart of the path tracking performance of different control methods in this invention.

[0068] Figure 11 This is a comparison diagram of vehicle body sway angles for different control methods in this invention.

[0069] Figure 12 This is a comparison chart of the vehicle body tilt angle errors of different control methods in this invention.

[0070] Figure 13 This is a comparison chart of the lateral load transfer rates of different control methods in this invention. Detailed Implementation

[0071] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0072] This invention provides a safety reinforcement learning-based vehicle path tracking control method integrating active tilting, as shown in Figure 2. Figure 2 This invention presents an overall flowchart of an integrated active tilt safety reinforcement learning vehicle path tracking control method. The method mainly consists of a reference path and road environment, a SAC reinforcement learning controller, a model prediction safety filter layer, and vehicle dynamics and path tracking environment. The reference path and road environment provide the target path the vehicle needs to track and acquire vehicle path tracking and tilt state information. The SAC reinforcement learning controller outputs candidate control quantities (i.e., candidate actions) based on the vehicle path tracking and tilt state information. Internally, the SAC reinforcement learning controller includes an Actor network (i.e., the SAC policy network) and a dual Q-Critic network (i.e., a first critic network and a second critic network). The Actor network outputs candidate control quantities based on the state; the dual Q-Critic network evaluates the value corresponding to the state and action, reduces Q-value overestimation, and improves training stability. In this invention, the candidate control quantities output by the SAC reinforcement learning controller include candidate front wheel steering angle and candidate active tilt control force.

[0073] The model prediction safety filter layer is used to perform safety checks and necessary corrections on candidate control variables output by the SAC. This safety filter layer includes four stages: short-time domain prediction, safety constraint check, safety feasibility determination, and minimum correction or non-intervention pass. Short-time domain prediction is used to predict the changes in vehicle state and control input caused by candidate actions in the future prediction time domain; safety constraint check is used to determine whether front wheel steering angle, active roll control force, lateral error, body roll angle, and lateral load transfer rate meet the constraints; safety feasibility determination is used to determine whether the candidate action is within the dual-input safety channel; minimum correction or non-intervention pass is used to determine whether to correct the candidate action.

[0074] The above method will be described in detail below, including the following steps:

[0075] Step 1: Obtain vehicle path tracking and tilt status information.

[0076] Kinematic model, magic tire model and six-degree-of-freedom vehicle roll model were established respectively.

[0077] To describe the planar motion of the vehicle, a ground coordinate system is established. With the vehicle body coordinate system ,like Figure 1 As shown.

[0078] In this system, the vehicle coordinate system has its origin at the vehicle's center of mass, with the X-axis along the vehicle's forward direction and the Y-axis pointing to the left side of the vehicle. The ground coordinate system is used to describe the vehicle's position and attitude information in the global environment. Let the vehicle's position in the ground coordinate system be (X, Y), and the yaw angle be... The longitudinal speed of the vehicle is lateral velocity is By transforming between the two coordinate systems, a connection can be established between the vehicle's dynamic state and its trajectory geometry. In the ground coordinate system, the vehicle's center-of-gravity velocity can be obtained from the velocity in the vehicle body coordinate system through coordinate transformation, i.e.:

[0079] ;

[0080] The lateral force expression for the magic tire model can be written as:

[0081] ;

[0082] In the formula, , representing the front and rear wheels of the vehicle, respectively; The road surface adhesion coefficient; For the lateral force of the tire, when and hour, These correspond to the front wheel lateral forces in a two-degree-of-freedom vehicle model. and rear wheel lateral force . , , , These are the stiffness factor, shape factor, peak factor, and curvature factor, respectively. , , , .also, This is the corrected sideslip angle. This represents the horizontal drift of the curve. This represents the vertical drift of the curve. For the vertical load of the tire, This refers to the tire slip angle; This refers to the tire camber angle; The parameters for the Magic Formula tire model are empirical fitting parameters. This invention refers to the fitting results based on CarSim tire data in existing studies and sets the parameters for the Magic Formula tire model. The specific parameters are shown in Table 1.

[0083] Table 1. Estimated Parameters of Lateral Force in Magic Tire Model:

[0084] parameter Value parameter Value <![CDATA[A0]]> 2.186 <![CDATA[A7]]> 0.9916 <![CDATA[A1]]> -9.129 <![CDATA[A8]]> 0.002289 <![CDATA[A2]]> 974 <![CDATA[A9]]> -0.00152 <![CDATA[A3]]> 3741 <![CDATA[A 10 ]]> -0.14 <![CDATA[A4]]> 24.37 <![CDATA[A 11 ]]> 19.1656 <![CDATA[A5]]> 0.005 <![CDATA[A 12 ]]> 39.54 <![CDATA[A6]]> 0.000056 <![CDATA[A 13 ]]> 19.14

[0085] The magic tire model can approximately degenerate into a linear tire model under small slip angle conditions. In this case, the tire lateral force can be expressed as:

[0086] ;

[0087] Tire lateral stiffness, usually measured in N / rad, represents the change in lateral force a tire can generate when the tire slip angle changes by a unit. α represents the tire slip angle, usually measured in rad, which is the angle between the tire's rolling direction and the direction of the tire's center of gravity velocity; it describes the deviation between the tire's actual direction of motion and the wheel's pointing direction.

[0088] The six-degree-of-freedom vehicle prediction model can be summarized as: lateral motion, yaw motion, sprung mass vertical motion, sprung mass tilting motion, and left and right unsprung mass vertical motion.

[0089] Assume the sprung mass of the vehicle is The unsprung masses on the left and right are respectively and The roll inertia of the sprung mass about the longitudinal axis of the vehicle body is Let the vertical displacement of the center of mass of the spring-loaded mass be... The body tilt angle is The vertical displacements of the left and right unsprung masses are respectively and Additionally, let h be the equivalent height from the center of mass of the sprung mass to the roll center, and let h be the lateral distance from the center of mass of the sprung mass to the points of action of the left and right suspensions. The stiffnesses of the left and right suspension springs are respectively and The damping coefficients of the left and right suspension shock absorbers are respectively and The vertical stiffness of the left and right tires are respectively and The left and right road surface input displacements are respectively and The left and right active suspensions act as the power source. and Under small tilt angle conditions, the roll dynamic equations are established. The vertical motion equations and roll motion equations of the sprung mass are as follows:

[0090] ;

[0091] in:

[0092] ;

[0093] The dynamic equation of unsprung mass:

[0094] ;

[0095] in:

[0096] ;

[0097] in, and These are the forces acting on the left and right suspensions, respectively, and are composed of the spring forces and damping forces of the left and right suspensions. This represents the vertical displacement of the center of mass of the spring-loaded mass; and These represent the vertical displacements of the sprung mass at the left and right suspension connection points, respectively.

[0098] The formula shows that the left and right unsprung masses are coupled to the vehicle body through the suspension on one hand, and maintain vertical contact with the road surface through the tires on the other. When the vehicle rolls, the compression of the left and right suspensions is different, which in turn causes a difference in the vertical response of the left and right unsprung masses.

[0099] When smaller, there are The following explanation is provided:

[0100] Left suspension extension: ;

[0101] Right side suspension extension: ;

[0102] The force generated by the spring on the left: ;

[0103] The force generated by the spring on the right: ;

[0104] The force generated by the left shock absorber:

[0105] The force generated by the right-side shock absorber: ;

[0106] The force difference generated by the spring: ;

[0107] Force difference caused by damping: ;

[0108] The yaw control moment of the active suspension: ;

[0109] The main parameters of the vehicle model are shown in Table 2.

[0110] Table 2. Main parameters of the vehicle model:

[0111] Parameter name symbol Value Overall vehicle quality m 1500 kg Yaw moment of inertia <![CDATA[I Z ]]> 3000 kg·m² Distance from center of gravity to front axle <![CDATA[l f ]]> 1.2 m Distance from center of gravity to rear axle <![CDATA[l r ]]> 1.6 m wheelbase L 2.8 m gravitational acceleration g 9.8 m / s² Front wheel lateral stiffness <![CDATA[C f ]]> 80000 N / rad Rear wheel lateral stiffness <![CDATA[C r ]]> 80000 N / rad Static front axle normal load <![CDATA[F zf ]]> 8410 N Static rear axle normal load <![CDATA[F zr ]]> 6310 N

[0112] In the simulation verification phase, the current motion state and reference path information of the vehicle can be predicted by combining the kinematic model, the magic tire model and the six-degree-of-freedom vehicle prediction model.

[0113] During actual vehicle operation, the vehicle's current motion state and reference path information can be obtained by vehicle sensors, state estimation module, and upper-level planning module.

[0114] The vehicle's current motion status includes the vehicle's lateral velocity, yaw rate, current position, heading angle, body roll angle, and roll rate; the reference path information includes reference path coordinates, reference heading angle, and reference curvature.

[0115] Based on the geometric relationship between the vehicle's current position and the reference path, the lateral error and heading error of the vehicle relative to the reference path are calculated. The lateral error characterizes the degree of lateral deviation of the vehicle's center of gravity from the reference path, while the heading error characterizes the deviation between the vehicle's actual heading angle and the tangential direction of the reference path.

[0116] Finally, vehicle path tracking and tilt status information were obtained, including the vehicle's lateral error, heading error, rate of change of lateral error, rate of change of heading error, current front wheel angle, rate of change of front wheel angle, error between the actual tilt angle and the target tilt angle, and vehicle tilt angular velocity.

[0117] Step 2: Calculate the desired tilt angle of the vehicle and determine the achievable tilt target.

[0118] During high-speed path tracking, the vehicle experiences lateral acceleration as it travels along a curved path, leading to body roll. To reduce vehicle roll response and lateral load transfer risks, this embodiment calculates the desired yaw angle based on the vehicle's longitudinal velocity and the yaw rate of the reference path. The desired yaw angle can be... The calculation formula is:

[0119] ;

[0120] in, The desired yaw rate, under small angle conditions, satisfies:

[0121] ;

[0122] ;

[0123] Due to limitations in the capabilities of vehicle actuators and suspension structure, the yaw angle achievable by a real vehicle is not unlimited. Let the maximum permissible yaw angle be... The actual target tilt angle used for control is:

[0124] ;

[0125] in, The target tilt angle (the achievable tilt angle under constraints) is taken as the maximum tilt angle in this paper. This value is used to limit the maximum attitude adjustment range during the vehicle's active tilting process, preventing the theoretically desired tilt angle from being too large under high-speed or high-curvature conditions, which could lead to the actuators being in a saturated state for a long time or causing excessive body attitude changes. Therefore, the target tilt angle tracked by the active tilt controller is an achievable tilt angle that meets the actual actuation capabilities of the vehicle. .

[0126] In this embodiment, the reinforcement learning observation vector can be represented as:

[0127] ;

[0128] In the formula, For lateral error, For heading error, and These are the rates of change of lateral error and the rates of change of heading error, respectively. The current front wheel steering angle, This represents the rate of change of the front wheel steering angle. This represents the error between the actual sway angle and the target sway angle of the vehicle body. The angular velocity of the vehicle body tilt. and The introduction of this feature is used to reflect the steering execution state and prevent the agent from outputting abrupt steering angles during continuous control. and The introduction of this feature reflects the effectiveness of active tilt control, enabling the agent to learn how to track the desired tilt angle and suppress vehicle attitude oscillations through active tilt control forces. Therefore, the defined state space can simultaneously describe trajectory tracking performance, control input continuity, and vehicle tilt stability.

[0129] The lateral load transfer ratio (LTR) can be calculated based on the relationship between vehicle roll dynamics and lateral acceleration, and its expression can be written as:

[0130] .

[0131] Step 3: Construct a SAC reinforcement learning controller that integrates active tilt control.

[0132] like Figure 3 As shown in the diagram, the Safe-SAC path tracking control system mainly consists of a reference input module, a vehicle dynamics model module, a SAC controller, a model prediction safety filter layer, and a vehicle state feedback module.

[0133] The reference input module provides the reference trajectory, vehicle speed, and road surface input required for vehicle path tracking. The reference trajectory is used to calculate lateral and heading errors; vehicle speed is used for vehicle dynamics prediction and controller state calculation; and road surface input reflects the road excitation effects experienced by the vehicle during operation. The vehicle dynamics model module includes a two-degree-of-freedom steering model, a four-degree-of-freedom roll model, and a magic tire model. The two-degree-of-freedom steering model and the four-degree-of-freedom roll model constitute a six-degree-of-freedom vehicle prediction model. The two-degree-of-freedom steering model describes the vehicle's lateral and yaw motions; the four-degree-of-freedom roll model describes the vehicle's roll response and suspension motion; and the magic tire model describes the nonlinear relationship between tire lateral force and tire slip angle. These models allow for the calculation of the vehicle's lateral motion state, yaw state, vehicle roll state, and tire lateral force changes under the action of front wheel steering angle and active roll control force. The SAC controller outputs candidate control actions based on the current vehicle path tracking and roll state information. These candidate control actions include a candidate front wheel steering angle δ. RL and candidate active tilt control force F RL The candidate front wheel steering angle is used to adjust the vehicle's lateral path tracking motion, and the candidate active yaw control force is used to adjust the vehicle's yaw response and lateral load transfer state. A model prediction safety filter layer is set between the SAC controller and the vehicle dynamics model to perform safety checks and necessary corrections on the candidate front wheel steering angle and candidate active yaw control force output by the SAC. It should be noted that... Figure 3 In the system block diagram shown, the vehicle dynamics model module is a unified abstract representation of the vehicle actuators and the vehicle's own dynamics. Therefore, the model prediction safety filter layer is located between the SAC controller and the actual vehicle actuators. The safety control actions output by the safety filter layer ultimately act on the steering actuator and the active tilt actuator, respectively. When a candidate action meets the safety constraints, the safety filter layer does not make any corrections and directly outputs the candidate action; when a candidate action has a risk of constraint violation, the safety filter layer solves for the safety control action that meets the safety constraints and has the smallest deviation from the candidate action through online optimization. The model prediction safety filter layer ultimately outputs the safe front wheel steering angle δ. safe and safe active tilt control force F safe This is then applied to the vehicle steering actuator and the active tilt actuator, respectively. The vehicle dynamics model updates the vehicle state under the safety control actions and sets the current state S... tFeedback is sent to the SAC controller and the model prediction safety filtering layer for candidate action generation, state prediction, and safety verification at the next sampling time. Therefore, Figure 3 The system structure shown embodies a closed-loop control process: "SAC generates candidate actions, the model predicts safety and the safety filter layer performs safety corrections, and the vehicle dynamics environment executes safety actions and provides feedback on the status." This structure can leverage the nonlinear decision-making capabilities of the SAC controller while ensuring that the front wheel steering angle, active roll control force, vehicle roll state, and lateral stability indicators meet safety constraints through the model predictive safety filter layer.

[0134] This embodiment uses the SAC reinforcement learning algorithm to construct a vehicle path tracking controller. Figure 4 As shown, the SAC reinforcement learning controller (hereinafter referred to as the SAC controller) takes the vehicle path tracking and tilt state information obtained in step 1 as input, and the front wheel angle and active tilt control force as action outputs. In this embodiment, the Actor policy network in the SAC controller outputs two continuous action quantities according to the input state, corresponding to the front wheel angle channel and the active tilt control force channel, respectively. To facilitate network training and action constraint processing, the Actor policy network can first output normalized actions, and then map them to actual control quantities according to the upper limit of the front wheel angle and the upper limit of the active tilt control force.

[0135] The front wheel steering angle and active tilt control force are not fixed control quantities pre-defined by humans, but are calculated in real time by the trained SAC policy network based on the vehicle's current path tracking and tilt state information. The candidate front wheel steering angle is used to adjust the vehicle's lateral movement, reducing lateral error, heading error, and anticipation lateral error; the candidate active tilt control force is used to adjust the vehicle's tilt response, suppressing excessive tilt angle and lateral load transfer rate. The candidate front wheel steering angle and candidate active tilt control force output by the SAC controller are further input into the model prediction safety filter layer. After safety verification and necessary corrections by the safety filter layer, they are used as the final control actions applied to the vehicle's actuators. The action vector is defined as: . and The normalized actions output by the agent are the normalized candidate front wheel steering angle and candidate active yaw control force, respectively. To facilitate neural network training and ensure that the two action quantities are learned on the same scale, the SAC action outputs are restricted to the following values ​​during the learning process: The actual control input is then obtained through proportional mapping:

[0136] ;

[0137] , These are the actual action values ​​input to the vehicle.

[0138] The reward function is set as follows:

[0139] ;

[0140] In the formula, For lateral error, For heading error, and These are the rates of change of lateral error and the rates of change of heading error, respectively. This represents the error between the actual yaw angle of the vehicle body and the achievable desired (target) yaw angle. The angular velocity of the tilt is... For the front wheel steering angle, For active tilt control force. This is the upper limit of the front wheel steering angle. This represents the upper limit of the active tilt control force. Based on the controller settings in this paper, the front wheel steering angle constraint is set to [value missing]. The upper limit of the active tilt control force is set to .

[0141] To enhance the agent's ability to avoid dangerous situations, additional penalty terms are set during the training process. When the vehicle's lateral error, yaw angle error, or control input exceeds physical constraints, a significant penalty is imposed. During training, the agent learns to reduce trajectory errors and actively avoids unsafe behaviors such as excessive yaw angles, excessive control forces, and rapid attitude changes. ;

[0142] The units are 1.5𝑚 and 0.21𝑟𝑎d.

[0143] This penalty mechanism prevents the agent from achieving short-term tracking results in the early stages of training through excessive steering angles or active yaw control forces, while simultaneously encouraging it to learn the safe control boundaries as early as possible. The reward function not only guides the agent to reduce trajectory errors but also explicitly constrains the vehicle's yaw attitude and control input amplitude, enabling the SAC to learn the appropriate division of labor between front wheel steering and active yaw control forces. If the vehicle's lateral load transfer rate is too high, it indicates that the vehicle is approaching the rollover risk boundary. Therefore, during training, a termination condition is constructed by combining LTR and lateral error to further constrain vehicle roll safety.

[0144] Step 4: Set up a safety filter layer for the active tilt model prediction.

[0145] A model prediction safety filter layer is set between the SAC reinforcement learning controller and the vehicle actuator. The model prediction safety filter layer does not directly replace the SAC controller in making path tracking decisions, but performs safety verification and necessary corrections on the candidate actions output by the SAC.

[0146] Figure 5 This is a schematic diagram illustrating the working principle of the model-predicted security filtering layer. Figure 5 Figure (a) in the diagram represents the output / state prediction channel. Figure 5 Figure (b) in the diagram illustrates the input security channel and the current action correction process.

[0147] like Figure 5 As shown in Figure (a), the horizontal axis represents time, the vertical dashed line represents the current time k, the left side of the dashed line represents the vehicle state or actual output that has occurred in the past, and the right side of the dashed line represents the future prediction time domain. The black solid line and black dots in the figure represent the actual output or actual state changes of the vehicle at past times. The vertical axis "output y / state" in the figure does not refer to the vehicle's lateral displacement alone, but refers to the predicted output or state quantities that the safety filter layer needs to constrain, such as lateral error, heading error, yaw rate, vehicle body tilt angle, and lateral load transfer rate. In different constraint analyses, y can correspond to different constrained state quantities. The blue dashed line represents the reference trajectory or expected output, and the blue upper and lower boundaries represent the upper and lower boundaries of the output safety channel. The output safety channel represents the safe range within which the predicted state of the vehicle is allowed to change. The curve formed by the blue dots represents the safe predicted output obtained under the action of the safety control action. This curve always lies within the output safety channel, indicating that the prediction result meets the safety constraints. The red dashed line represents the candidate predicted output obtained when the SAC candidate action is directly applied. This predicted output may cross the output boundary at some point in the future, thus posing a risk of constraint violation. By comparing the red candidate prediction output with the blue safety channel, it can be determined whether the SAC candidate action may cause the vehicle state to exceed the limit.

[0148] like Figure 5 As shown in Figure (b), the horizontal axis represents time, and the vertical axis represents the front wheel angle input. The black stepped lines in the figure represent the front wheel angle control variables executed in the past. The right side of the vertical dashed line is the prediction time domain, and the blue upper and lower boundaries represent the upper and lower boundaries of the input safety channel, i.e., the range of allowable changes in the front wheel angle within the future prediction time domain. The red hollow dots in the figure represent the candidate front wheel angle output by the SAC at the current moment. This candidate action is located outside the input safety channel, indicating that it may exceed the allowable range of the actuator or cause subsequent state constraints to be violated. The red arrows indicate the direction in which the model prediction safety filter layer corrects the candidate actions, and the blue solid dots represent the safe front wheel angle obtained after correction. The blue stepped lines represent the future control input sequence obtained by the safety filter layer.

[0149] It should be noted that the figure Figure 5Figure (b) illustrates the input safety channel and action correction process using the front wheel steering angle δ as an example. In the active tilt dual-input control method of this invention, the control input also includes the active tilt control force F. Therefore, the model prediction safety filter layer not only needs to determine whether the front wheel steering angle δ is within the safe range, but also needs to determine whether the active tilt control force F and its change satisfy the safety constraints. That is to say, Figure 5 The input safety channel in Figure (b) can be further extended into a dual input safety channel consisting of the front wheel steering angle and the active tilt control force.

[0150] pass Figure 5 As can be seen, the role of the model prediction safety filter layer is as follows: at the current time k, based on the vehicle's current state, reference trajectory, and prediction model, it predicts the changes in vehicle state caused by SAC candidate actions in the future prediction time domain; if both the predicted output and control input are within the safe channel, the candidate action is passed directly; if the candidate action may cause input exceedance or state out of bounds, the safety filter layer corrects it into a safe action within the safe channel through online optimization. This yields the final control action that satisfies input constraints, state constraints, and stability constraints, thereby avoiding safety risks when SAC candidate actions directly act on the vehicle.

[0151] Step 5: Construct a dual-input safety channel for front wheel steering angle and active tilt control force within the model prediction safety filter layer.

[0152] like Figure 6 As shown in the figure, the horizontal axis represents the front wheel steering angle δ, and the vertical axis represents the active tilt control force F. Therefore Figure 6 It represents the two-dimensional motion space formed by the front wheel steering angle and the active tilt control force.

[0153] Figure 6 In and These represent the lower and upper limits of the front wheel steering angle, respectively, used to limit the range of front wheel steering angles that the steering actuator is allowed to output; and These boundaries represent the lower and upper limits of the active tilt control force, respectively. They are used to indicate the range of physical inputs allowed by the vehicle actuator, i.e., the front wheel angle and the active tilt control force cannot exceed the maximum output capacity of the actuator.

[0154] Figure 6 The green area in the diagram represents the current vehicle status x. k The dual-input safety channel is not directly obtained from the simple limiting of the front wheel steering angle δ and the active roll control force F, but is jointly determined by the vehicle's lateral dynamics, roll dynamics, lateral load transfer relationships, and safety constraints. Specifically, for the current vehicle state x... kThe model predicts safety filters using different combinations of front wheel steering angles δ and active yaw control forces F as candidate inputs. The vehicle prediction model predicts changes in lateral error, heading error, yaw rate, vehicle roll angle, and lateral load transfer rate, as well as changes in control inputs, over a future period. If a set of control inputs simultaneously satisfies vehicle input constraints, input rate of change constraints, vehicle state constraints, and lateral stability constraints within the prediction time domain, then this set of control inputs belongs to the dual-input safety channel. If it might cause any constraint to be violated, then this set of control inputs is located outside the dual-input safety channel.

[0155] Therefore, the dual-input safety channel specifically refers to the current vehicle state x. k The set of all safe and feasible actions is defined as a dual-input control action consisting of front wheel steering angle and active yaw control force. This set must simultaneously satisfy at least the following constraints: front wheel steering angle amplitude constraint, active yaw control force amplitude constraint, front wheel steering angle rate of change constraint, active yaw control force rate of change constraint, lateral error constraint, heading error constraint, yaw rate constraint, vehicle body yaw angle constraint, and lateral load transfer rate constraint.

[0156] Figure 6 The black dots in the diagram represent candidate actions output by the reinforcement learning controller. When a candidate action is located outside the green safety channel, it indicates that directly executing the action may cause the vehicle input to exceed the limit, the lateral error to increase, the vehicle body tilt angle to be too large, or the lateral load transfer rate to be too high. Therefore, it cannot be directly applied to the vehicle actuator.

[0157] Figure 6 The blue dot in the diagram represents the safe and feasible action obtained after correction by the model prediction safety filter layer. This point is located inside the green safety channel, indicating that the corrected control action meets the input constraints, vehicle tilt constraints, and lateral stability constraints under the current vehicle state, and can be used as the final control action output.

[0158] Figure 6 The green boundary in the diagram represents the dual-input safety passage boundary. This boundary is determined jointly by vehicle dynamics predictions and safety constraints, and it changes with the current vehicle state x. k The shape and extent of the safety passage vary depending on factors such as the curvature of the reference path, vehicle speed, and roll condition. In other words, the shape and extent of the safety passage are not fixed under different vehicle conditions, but are determined in real time by the model-predicted safety filter layer based on the current conditions.

[0159] pass Figure 6It can be seen that the dual-input safety channel in this invention does not restrict the front wheel steering angle or the active yaw control force separately, but rather makes a safety judgment on the combination of the front wheel steering angle and the active yaw control force. Only when the combined effect of the two does not cause the vehicle path tracking error, body yaw response, and lateral load transfer rate to exceed the safe range can the control combination be considered a dual-input safety channel. Figure 6 This is consistent with the definition of "front wheel steering angle - active tilt control force dual input safety channel" in the invention, and also corresponds to the process of the model prediction safety filter layer performing safety verification and minimum correction on candidate control actions in the specific implementation.

[0160] The dual-input safety channel is a set of safe and feasible actions built and used within the model prediction safety filter layer. The logical relationship between the two is as follows: The model prediction safety filter layer first receives the candidate front wheel steering angle and candidate active yaw control force output by the SAC controller and uses them as candidate control actions; then, based on the current vehicle state, the vehicle prediction model, and the finite prediction time domain, it predicts the changes in vehicle state and control input caused by the candidate control actions in the future short time domain, obtaining the predicted vehicle state and predicted control input sequence in the future prediction time domain; wherein, the current vehicle state includes at least the vehicle's lateral error, heading error, lateral velocity, yaw rate, body roll angle, and roll rate. The predicted vehicle state includes the vehicle's lateral error, heading error, lateral velocity, yaw rate, body roll angle, roll rate, and lateral load transfer rate. The predicted control input sequence includes the front wheel steering angle and active roll control force for each future prediction step. Subsequently, the candidate front wheel steering angle and candidate active roll control force at the current moment, as well as the predicted vehicle state and predicted control input sequence, are compared one by one with the safety constraints. The combination of front wheel steering angle and active roll control force that simultaneously satisfies the safety constraints constitutes the dual-input safety channel for the current vehicle state. The safety constraints include vehicle input constraints, input change rate constraints, vehicle state constraints, and lateral stability constraints. The control action consists of the safe front wheel steering angle and the safe active roll control force. The vehicle input constraints specifically include front wheel steering angle amplitude constraints and active yaw control force amplitude constraints; the input rate of change constraints specifically include front wheel steering angle rate of change constraints and active yaw control force rate of change constraints; the vehicle state constraints specifically include lateral error constraints, heading error constraints, lateral velocity constraints, yaw rate constraints, body roll angle constraints, and roll rate constraints; the lateral stability constraints specifically include lateral load transfer rate constraints.

[0161] Therefore, the model prediction safety filter layer is the overall mechanism for achieving safety verification and action correction. The dual-input safety channel is the constrained feasible region used in this mechanism to determine whether a candidate action is safe. When the candidate action output by SAC is within the dual-input safety channel, it means that the candidate action will not cause the input or state constraints to be violated within the finite prediction time domain. The model prediction safety filter layer does not intervene and directly outputs the candidate action. When the candidate action is outside the dual-input safety channel, it means that directly executing the candidate action may lead to the vehicle state exceeding the limit or the stability risk increasing. At this time, the online optimization module in the safety filter layer solves for the safe action that is closest to the candidate action and is located within the dual-input safety channel through online optimization, and uses the corrected front wheel steering angle and active yaw control force as the final control input.

[0162] This embodiment uses the front wheel steering angle and active yaw control force as coupled two-dimensional control inputs, rather than simply limiting each separately. This is because the front wheel steering angle directly affects the vehicle's lateral acceleration and yaw response, thus influencing the yaw angle and lateral load transfer rate; while the active yaw control force primarily affects vehicle attitude, it also influences tire vertical load distribution and vehicle stability through roll dynamics and load transfer relationships. Therefore, during high-speed path tracking, the front wheel steering angle and active yaw control force jointly determine whether the vehicle is in a safe state.

[0163] The SAC agent outputs normalized binary actions, and the processed candidate actions output to the vehicle model can be represented as follows: .in, Indicates the front wheel angle of the candidate. This represents the candidate active tilt control force. The actual action executed after correction by the model-predicted safety filter layer is: The safe passage is a two-dimensional coupled safe passage jointly determined by the current vehicle state, front wheel steering angle, and the dynamics of the active suspension. Its joint safe feasible region is defined as:

[0164] ;

[0165] In the formula, k represents the current sampling time, and i represents the number of prediction steps in the prediction time domain; 𝑘+𝑖|𝑘 This represents the future vehicle state at the current time k, obtained based on the vehicle prediction model; 𝑢 𝑘+𝑖|𝑘 This represents the predicted or optimized control input for the i-th future step at the current time k; 𝛥𝑢 𝑘+𝑖|𝑘 This represents the change in control input at the i-th future step, used to describe the changes in control input between adjacent prediction steps. To predict the time domain, For a set of safe states, To control the set of input constraints, To control the incremental constraint set. Because and They all affect the vehicle's condition, therefore they usually include: There is a coupling relationship between the front wheel steering angle and the active yaw control force. Therefore, the dual-input safety channel cannot be simply regarded as a direct combination of the front wheel steering angle safety range and the active yaw control force safety range. This represents the feasible region under the corresponding action. Therefore, the essence of this chapter is the two-dimensional coupled safe feasible region. Searching for distance SAC candidate actions Recent safety actions .

[0166] Front wheel steering angle and active roll control force are not two independent limiting variables, but rather jointly influence vehicle state through vehicle lateral dynamics, roll dynamics, and lateral load transfer relationships. Therefore, when constructing a dual-input safety channel, the relationship between front wheel steering angle and vehicle lateral motion state is first established based on the vehicle lateral dynamics model (the establishment process uses an existing two-degree-of-freedom model). Specifically, the front wheel steering angle changes the front wheel slip angle and front wheel lateral force, thereby affecting the vehicle's lateral velocity, yaw rate, lateral error, and heading error. Then, the relationship between active roll control force and vehicle roll state is established based on the roll dynamics model. Specifically, the active roll control force generates an equivalent roll control torque through the active roll actuator or active suspension system, thereby affecting the vehicle roll angle and roll rate. Finally, based on the lateral load transfer relationship, the relationship between vehicle lateral acceleration, vehicle roll motion, active roll control force, and lateral load transfer rate is established to determine whether the vehicle has roll stability risks. Front wheel steering angle directly alters a vehicle's lateral acceleration and yaw response, thus affecting body roll angle and lateral load transfer rate. Active roll control force indirectly affects tire lateral force and lateral stability by adjusting the vehicle's attitude and altering the vertical load distribution of the left and right wheels. Therefore, the safety filter layer needs to simultaneously determine the safety of the combination of these two factors within a two-dimensional control space, rather than independently saturating the front wheel steering angle and active roll control force separately.

[0167] The purpose of establishing the above relationships is to link the two control inputs, front wheel steering angle and active yaw control force, with the vehicle's future lateral motion state, body yaw state, and lateral load transfer rate, providing a basis for state prediction and safety constraint verification for the model prediction safety filter layer. Specifically, the front wheel steering angle affects lateral velocity, yaw rate, lateral error, and heading error through vehicle lateral dynamics; the active yaw control force affects body yaw angle and yaw rate through roll dynamics; and vehicle lateral acceleration and body yaw motion further affect the lateral load transfer rate. By establishing the above relationships between inputs and states, the model prediction safety filter layer (within which the vehicle prediction model) can predict the vehicle state changes caused by SAC candidate actions within a finite prediction time domain and determine whether the candidate action satisfies vehicle input constraints, input change rate constraints, vehicle state constraints, and lateral load transfer rate constraints. If the candidate action does not meet the safety constraints, the safe front wheel steering angle and safe active yaw control force closest to the candidate action are solved in the subsequent online optimization process, thereby achieving the minimum necessary correction to the candidate action.

[0168] Step 6: Determine whether the SAC candidate actions meet the safety constraints.

[0169] The candidate actions output by the SAC are substituted into the model prediction safety filter layer. The vehicle prediction model then predicts the changes in vehicle state and control input caused by these candidate actions within a finite prediction time domain. Specifically, the prediction process is as follows: using the current vehicle state as the initial prediction state, the vehicle prediction model uses the candidate front wheel steering angle and candidate active yaw control force output by the SAC as prediction inputs. Combined with the reference path curvature, reference heading angle, and vehicle longitudinal velocity, it progressively extrapolates the future vehicle state within a finite prediction time domain. In this embodiment, the predicted future vehicle state of the vehicle prediction model in the model prediction safety filter layer includes lateral error, heading error, vehicle lateral velocity, yaw rate, body roll angle, body roll rate, and lateral load transfer rate. The predicted control input sequence includes the front wheel steering angle and active yaw control force for each future prediction step. Within the prediction time domain, the front and rear wheel slip angles are first calculated based on the front wheel steering angle, lateral velocity, yaw rate, and vehicle longitudinal velocity, and the lateral forces of the front and rear wheels are calculated using the tire model. Then, the lateral velocity change rate and yaw rate change rate are calculated based on the vehicle's lateral dynamics model. Next, the lateral error change rate and heading error change rate are calculated based on the path error model. Subsequently, the vehicle roll acceleration under active roll control force is calculated based on the roll dynamics model, and the vehicle roll angle and roll rate are recursively obtained. Finally, the lateral load transfer rate is calculated based on the predicted lateral acceleration and vehicle roll state. When using a discretized form, the future state of the vehicle can be uniformly represented as:

[0170] ;

[0171] In the formula, f d (·) is a prediction function composed of the vehicle lateral dynamics model, path error model and roll dynamics model; This represents the external information required for the prediction process, including reference path curvature, reference heading angle, and vehicle longitudinal velocity. 𝑘+𝑖|𝑘 Let represent the predicted vehicle state at the current time k for the i-th future step. 𝑘+𝑖|𝑘 This represents the predicted future control input at the current time k.

[0172] The reference heading angle is primarily used to calculate the vehicle's current heading error, serving as part of the initial state prediction. The reference path curvature is mainly used for heading error prediction, representing the degree of curvature of the reference path in the prediction time domain. The vehicle's longitudinal velocity is used for tire slip angle calculation, lateral error prediction, heading error prediction, and lateral acceleration calculation. Therefore, the reference path curvature, reference heading angle, and vehicle longitudinal velocity do not form a separate prediction model, but rather participate in the calculation as external inputs or known parameters in the vehicle state prediction process.

[0173] The lateral load transfer rate is calculated based on the predicted lateral acceleration and vehicle roll state, and is used to evaluate the risk of vehicle roll stability. It can be expressed as:

[0174] LTR k+i|k = [I x θ ddot(k+i|k) + m s ha y(k+i|k) − m s hg θ k+i|k ] / (m s gd);

[0175] In the formula, LTR k+i|k Let I be the lateral load transfer rate predicted at the current time k for the i-th future step; x θ is the roll inertia of the sprung mass about the longitudinal axis of the vehicle body; ddot(k+i|k) The predicted vehicle body yaw angle acceleration; θ k+i|k The predicted vehicle body yaw angle; m s denoted as ρs, where h is the equivalent height from the center of mass of the sprung mass to the center of tilt; as is the sprung mass. y(k+i|k) denoted as , where g is the predicted lateral acceleration of the vehicle; g is the acceleration due to gravity; and d is the lateral distance from the center of mass of the sprung mass to the points of action of the left and right suspensions.

[0176] The vehicle lateral dynamics model is used to predict the changes in vehicle lateral velocity and yaw rate under the action of front wheel steering angle; the path error model is used to predict the changes in vehicle lateral error and heading error; the roll dynamics model is used to predict the changes in vehicle body roll angle and roll rate under the action of active roll control force; the lateral load transfer relationship is used to calculate the lateral load transfer rate based on the predicted lateral motion state and vehicle body roll state.

[0177] In this embodiment, the vehicle state changes within the finite prediction time domain include at least lateral error, heading error, lateral velocity, yaw rate, vehicle body roll angle, roll rate, and lateral load transfer rate. Specifically, lateral error and heading error reflect the vehicle's tracking deviation from the reference path; lateral velocity and yaw rate reflect the vehicle's lateral and yaw motion states; vehicle body roll angle and roll rate reflect changes in vehicle body roll attitude; and the lateral load transfer rate reflects the degree of vertical load transfer between the left and right wheels and the risk of roll stability.

[0178] The safety constraints in step 6 refer to the constraints used to define the dual-input safety channel. These constraints include at least the following: front wheel angle amplitude constraint, active yaw control force amplitude constraint, front wheel angle change rate constraint, active yaw control force change rate constraint, lateral error constraint, heading error constraint, lateral velocity constraint, yaw rate constraint, vehicle yaw angle constraint, yaw rate constraint, and lateral load transfer rate constraint. The front wheel angle amplitude constraint limits the candidate front wheel angle at the current moment and the front wheel angle at each prediction step within the prediction time domain from exceeding the upper limit of the front wheel angle. The active yaw control force amplitude constraint limits the candidate active yaw control force at the current moment and the active yaw control force at each prediction step within the prediction time domain from exceeding the upper limit of the active yaw control force. The upper limit of the front wheel angle and the upper limit of the active yaw control force are preset based on the physical capabilities of the vehicle's actuators. The front wheel steering angle change rate constraint limits the change in the candidate front wheel steering angle at the current moment relative to the actual front wheel steering angle at the previous moment to not exceed the maximum allowable front wheel steering angle change rate, and also limits the change in front wheel steering angle between adjacent prediction steps within the prediction time domain to not exceed the maximum allowable front wheel steering angle change rate; the active yaw control force change rate constraint limits the change in the candidate active yaw control force at the current moment relative to the actual active yaw control force at the previous moment to not exceed the maximum allowable active yaw control force change rate, and also limits the change in active yaw control force between adjacent prediction steps within the prediction time domain to not exceed the maximum allowable active yaw control force change rate; the lateral error constraint limits the lateral error of each prediction step within the prediction time domain to not exceed the maximum allowable lateral error. The heading error constraint is used to limit the heading error of each prediction step in the prediction time domain from not exceeding the maximum allowable heading error; the lateral velocity constraint is used to limit the lateral velocity of each prediction step in the prediction time domain from not exceeding the maximum allowable lateral velocity; the yaw rate constraint is used to limit the yaw rate of each prediction step in the prediction time domain from not exceeding the maximum allowable yaw rate; the vehicle roll angle constraint is used to limit the vehicle roll angle of each prediction step in the prediction time domain from not exceeding the maximum allowable vehicle roll angle; the roll rate constraint is used to limit the roll rate of each prediction step in the prediction time domain from not exceeding the maximum allowable roll rate; the lateral load transfer rate constraint is used to limit the absolute value of the lateral load transfer rate of each prediction step in the prediction time domain from not exceeding the maximum allowable lateral load transfer rate.

[0179] After completing state prediction within a finite prediction time domain, the model prediction safety filter layer compares the control input and predicted vehicle state obtained at each prediction step with the aforementioned safety constraints. Specifically: it determines whether the candidate front wheel steering angle meets the front wheel steering angle amplitude constraint; it determines whether the front wheel steering angle at each prediction step within the prediction time domain meets the front wheel steering angle amplitude constraint; it determines whether the candidate active yaw control force meets the active yaw control force amplitude constraint; it determines whether the active yaw control force at each prediction step within the prediction time domain meets the active yaw control force amplitude constraint; it determines whether the change in the candidate front wheel steering angle at the current moment relative to the actual front wheel steering angle at the previous moment meets the front wheel steering angle change rate constraint; it determines whether the change in the front wheel steering angle between adjacent prediction steps within the prediction time domain meets the front wheel steering angle change rate constraint; and it determines whether the change in the candidate active yaw control force at the current moment relative to the actual active yaw control force at the previous moment meets the active yaw control force change rate constraint. The system determines whether the change in active tilt control force between adjacent prediction steps within the prediction time domain satisfies the active tilt control force change rate constraint; whether the lateral error of each prediction step within the prediction time domain satisfies the lateral error constraint; whether the heading error of each prediction step within the prediction time domain satisfies the heading error constraint; whether the lateral velocity of each prediction step within the prediction time domain satisfies the lateral velocity constraint; whether the yaw rate of each prediction step within the prediction time domain satisfies the yaw rate constraint; whether the vehicle body yaw angle of each prediction step within the prediction time domain satisfies the vehicle body yaw angle constraint; whether the yaw rate of each prediction step within the prediction time domain satisfies the yaw rate constraint; and whether the lateral load transfer rate of each prediction step within the prediction time domain satisfies the lateral load transfer rate constraint.

[0180] If the candidate front wheel angle and candidate active yaw control force satisfy the vehicle input constraints and input rate of change constraints, and all predicted vehicle states and predicted control input sequences in the prediction time domain satisfy the safety constraints, then the candidate control action output by SAC is determined to be within the dual-input safety channel. No correction is made to the candidate control action, and the candidate front wheel angle and candidate active yaw control force are directly used as the final control action output.

[0181] If the candidate front wheel steering angle or candidate active yaw control force does not meet the vehicle input constraints and input rate of change constraints, or if any predicted vehicle state or predicted control input sequence in the prediction time domain does not meet the aforementioned safety constraints, then the candidate control action output by the SAC is determined to be not located in the dual-input safety channel. The safe control action that is closest to the candidate control action and meets the safety constraints is solved online, and the corrected safe front wheel steering angle and safe active yaw control force are output as the final control action.

[0182] When the candidate action is located within the dual-input safety channel, the model predicts that the safety filter layer does not make additional corrections to the SAC output. The final executed action is equal to the candidate action, which is the final safety control action applied to the vehicle. This process reflects the non-intervention nature of the safety filter layer, that is, when the reinforcement learning output meets the safety requirements, the safety filter layer will not change the original control intent of the SAC controller.

[0183] Figure 7 This is a non-interventional diagram illustrating the safety of candidate maneuvers. The horizontal axis δ represents the front wheel steering angle, and the vertical axis F represents the active yaw control force. Therefore, this diagram represents a two-dimensional motion space composed of the front wheel steering angle and the active yaw control force.

[0184] Figure 7 In and These represent the lower and upper limits of the front wheel steering angle, respectively, used to limit the range of front wheel steering angles that the steering actuator is allowed to output; and These represent the lower and upper limits of the active yaw control force, respectively, used to limit the range of control force allowed to be output by the active yaw actuator. The enclosed blue dashed rectangle represents the physical amplitude constraint range of the control input, that is, the front wheel angle and the active yaw control force cannot exceed the allowable range of the actuator. The blue solid box D0(x(k)) in the figure represents the dual-input safety channel under the current vehicle state x(k), which can also be understood as the set of safe actions. This area is not simply composed of the upper and lower limits of the front wheel angle and the active yaw control force, but is a safe action range obtained after further considering the vehicle's lateral dynamics, roll dynamics, and lateral load transfer constraints. In other words, only when the combination of the front wheel angle and the active yaw control force does not lead to excessive lateral error, excessive yaw rate, excessive vehicle yaw angle, or excessive lateral load transfer rate in the prediction time domain, does the action belong to D0(x(k)).

[0185] Figure 7 u in cand (k) represents the candidate control action output by the SAC controller at the current time k. This candidate action consists of the candidate front wheel steering angle and the candidate active yaw control force. Figure 7 u in safe (k) represents the final safety control action output after verification by the model-predicted safety filter layer. The solid green dot indicates the location of the final output safety control action.

[0186] exist Figure 7 in, u cand (k) and u safeIf (k) is located at the same position and this point is inside the blue solid box D0(x(k)), it indicates that the candidate action output by SAC already satisfies the dual-input safety constraints under the current vehicle state. Therefore, the model prediction safety filter layer does not need to modify the candidate action, but directly uses the candidate action output by SAC as the final control action output. At this time, we have:

[0187] u safe (k)= u cand (k)

[0188] therefore, Figure 7 This reflects the non-interventional nature of the model prediction safety filter layer, that is, when the SAC candidate action itself is safe, the safety filter layer does not change the action, so as to preserve the original control intent of the SAC controller as much as possible.

[0189] Step 7: Perform online optimization and correction when candidate actions do not meet safety constraints.

[0190] like Figure 8 As shown, Figure 8 This is a schematic diagram of projection correction when the candidate action is unsafe.

[0191] Figure 8 The red hollow dot u in cand (k) represents the candidate control action output by the SAC controller at the current moment. Although this point may still be within the input physical amplitude constraint range, it is located outside the blue solid box D0(x(k)), indicating that this candidate action may lead to a violation of safety constraints in the future prediction time domain, such as causing excessive vehicle yaw angle, excessive yaw rate, or excessive lateral load transfer rate. Therefore, this candidate action cannot be directly applied to the vehicle actuators.

[0192] Figure 8 The black arrows in the diagram indicate the direction in which the model predicts the safe filtering layer corrects for unsafe candidate actions. The safe filtering layer uses the candidate action u output by the SAC as an example. cand (k) serves as the reference action, and the nearest safe action is found within the dual-input safe channel D0(x(k)). The solid green dot u in the diagram... safe (k) represents the safety control action obtained after online optimization and correction. This action is located inside or on the boundary of D0(x(k)), indicating that it satisfies the vehicle input constraints, input rate of change constraints, vehicle state constraints, and lateral stability constraints under the current vehicle state.

[0193] therefore, Figure 8This reflects the online minimum correction mechanism of the model prediction safety filter layer. When a candidate SAC action does not meet the safety constraints, the safety filter layer does not completely regenerate the control action. Instead, it makes minimum necessary corrections to the candidate front wheel steering angle and candidate active yaw control force while still satisfying the safety constraints, so that the final output u... safe (k) as close to u as possible cand (k). This avoids unsafe actions directly affecting the vehicle while preserving the control intent of the SAC controller to the greatest extent possible.

[0194] It should be noted that, Figure 7 and Figure 8 D0(x(k)) in the above text is related to the dual-input secure channel Ω. u (x k The meanings are basically the same, both representing a set of dual-input actions that meet safety constraints under the current vehicle state.

[0195] When the candidate action output by the SAC is not located outside the dual-input safety channel, it indicates that the action may lead to excessive front wheel steering angle, excessive active roll control force, excessive vehicle roll angle, excessive yaw rate, or excessive lateral load transfer rate. In this case, the model prediction safety filter layer does not directly adopt the candidate action output by the SAC, but instead solves for a control action that is closest to the SAC candidate action and satisfies the safety constraints through online optimization. It should be noted that this process cannot be simply understood as "substituting the candidate action output by the SAC into the objective function to directly obtain the safe action". More accurately, the candidate action output by the SAC is used as a reference action in the objective function. The model prediction safety filter layer uses the control input sequence in the prediction time domain as the variable to be solved, and under the constraints of input constraints, input rate of change constraints, vehicle roll constraints, and lateral stability constraints, it finds the safe control action closest to the SAC candidate action through online optimization.

[0196] The objective function is written as:

[0197] ;

[0198] In the formula, J is the objective function; D u This is a normalization matrix for the motion amplitude, used to unify the dimensions of the front wheel steering angle and the active roll control force; 𝑘+j|𝑘 The control input for the j-th step predicted at the current time k includes the front wheel steering angle and active yaw control force; RL (k) represents the candidate control action output by the SAC at the current time k; D 𝛥𝑢 To control the incremental normalization matrix, used to unify the dimensions of the front wheel steering angle change and the active roll control force change; 𝛥𝑢 𝑘+j|𝑘 For the change in control input in the j-th step in the future; Nc To control the time domain length, let represent the number of control input steps that need to be solved in the online optimization. Here, k represents the current sampling time, and j represents the j-th control step within the control time domain. In the optimal control sequence obtained through online optimization, the first control input is the final safe control action output at the current time, namely, the front wheel steering angle and active yaw control force that satisfy the safety constraints. This is the action deviation weight matrix. To control the incremental weight matrix, As slack variables, The slack variable penalty coefficient is due to the front wheel steering angle. and active tilt control force The dimensions differ significantly, so a normalization matrix is ​​introduced into the objective function. and The amplitude of the motion and the control increment are scaled uniformly. This prevents the active yaw control force from dominating the optimization result due to its large magnitude, allowing the safety filter layer to reasonably balance the front wheel angle correction and the active yaw control force correction.

[0199] The online optimization problem of the active tilt safety filter layer is used to make the minimum necessary correction to the candidate control actions output by the SAC when the candidate actions do not meet the safety constraints. This optimization problem does not simply substitute the candidate actions into the objective function to obtain the result directly. Instead, it uses the SAC candidate actions as reference inputs, and the front wheel angle and active tilt control force sequence in the control time domain, along with relaxation variables, as the variables to be solved. Under the joint constraints of the vehicle prediction model, input constraints, input rate of change constraints, state constraints, and lateral load transfer rate constraints, it solves for the safe control action that satisfies the safety constraints and has the smallest deviation from the candidate actions. After optimization, the optimal control input sequence is obtained. A rolling time domain control method is adopted, where only the first set of control inputs in the sequence is used as the actual output safe control action at the current moment, i.e., the safe front wheel angle and safe active tilt control force; the remaining predictive step control quantities are only used for prediction and optimization and do not directly act on the vehicle actuators. The online optimization problem (quadratic programming problem) of the active tilt safety filter layer can be uniformly expressed as:

[0200]

[0201] In the formula, The candidate dual-input control action output by the SAC controller at the current time k. =[δ SAC(k) ,F SAC(k) ]^T;u 0|k and u 1|k These are the control inputs for step 0 and step 1 predicted by the model for the security filter layer at the current time k, respectively, where u i|k=[δ i|k ,F i|k ]^T;u safe(k−1) The safety filter layer output at the previous sampling time and the actual safety control action applied to the vehicle; x k+i|k f represents the vehicle state predicted at the current time k for the i-th future step; d (·) is a prediction function composed of the vehicle lateral dynamics model, path error model and roll dynamics model; This represents known information such as reference trajectory curvature, reference heading angle, and longitudinal velocity; U is the set of control input constraints, including front wheel steering angle magnitude constraints and active yaw control force magnitude constraints; ΔU is the set of control input rate of change constraints, including front wheel steering angle rate of change constraints and active yaw control force rate of change constraints; X safe For vehicle state safety sets; LTR k+i|k The predicted lateral load transfer rate; LTR max ε represents the maximum permissible lateral load transfer rate; ε is a non-negative relaxation variable used to relax the lateral load transfer rate constraint in extreme cases, avoiding infeasibility of the optimization problem. u R is the candidate action tracking weight matrix. u To control the input change penalty weight matrix, q δ q F q Δδ and q ΔF represent the weighting coefficients corresponding to the front wheel steering angle, active yaw control force, and their changes, respectively. In the formula, , It should be noted that slack variables are mainly used for moderate relaxation of state constraints and LTR constraints, while the front wheel steering angle, active yaw control force and its rate of change constraints are still treated as hard constraints to ensure that the final output action does not exceed the physical range of the actuator.

[0202] Step 8: Apply the safety control action to the vehicle's actuators.

[0203] After the model predicts the safety filter layer's output safety control action, the front wheel steering angle δ will be adjusted. safe The safe active tilt control force F is sent to the vehicle steering actuator. safe The signal is sent to the active tilt actuator. Under this control action, the vehicle completes path tracking and vehicle attitude adjustment within the current sampling period.

[0204] In the next sampling period, the vehicle's current state and reference path information are reacquired, and steps 1 to 8 are repeated to form a rolling updated safety reinforcement learning path tracking control process. In this way, the vehicle can continuously update its safety actions based on real-time status under conditions such as high-speed lane changes, double lane change obstacle avoidance, and curved path tracking, ensuring that the path tracking control process simultaneously meets tracking accuracy requirements and vehicle stability constraints.

[0205] Step 9: Perform simulation verification.

[0206] In this embodiment, a dual lane-change path can be selected as the reference path to compare SAC, NMPC, and the Safe-SAC method with integrated active sway described in this invention. During the simulation, the vehicle travels at a set longitudinal speed, and the controller outputs the front wheel steering angle and active sway control force according to the reference path. The effectiveness of the method of this invention is verified by comparing indicators such as vehicle lateral displacement, lateral error, yaw angle error, front wheel steering angle, active sway control force, vehicle body sway angle, and lateral load transfer rate.

[0207] Simulation results can be used to illustrate that the method described in this invention can perform safety checks and minimum necessary corrections on high-risk actions output by SAC while maintaining path tracking accuracy, reducing the impact of unsafe control actions in the early stages of training or under complex conditions; at the same time, the active tilt control force can improve the vehicle's roll response, reduce the risk of lateral load transfer, and improve the vehicle's driving stability during high-speed path tracking.

[0208] Figure 9 This is a simulation diagram of the active tilting vehicle path tracking control in this invention. Figure 9 This graph compares simulation results of vehicle lateral displacement under different control methods. The horizontal axis represents time, and the vertical axis represents vehicle lateral displacement. In the graph, the black curve represents the desired path, the red curve represents the SAC passive suspension control result, the blue curve represents the SAC active roll control result, the green curve represents the NMPC active roll control result, and the purple curve represents the Safe-SAC active roll control result of this invention. Figure 9It can be seen that all control methods can enable the vehicle to track the desired path, and the trend of the vehicle's lateral displacement change is basically consistent with the desired path, indicating that the designed control methods all have a certain path tracking capability. In the stage where the path curvature changes significantly, there is a certain deviation between the SAC passive suspension control results and the desired path, indicating that when using SAC control alone without introducing active yaw adjustment, the vehicle is easily affected by changes in vehicle attitude and lateral dynamic coupling during high-speed lane changes or double lane shifts, leading to a decrease in tracking accuracy. Compared with SAC passive suspension, SAC active yaw control can better follow the desired path, indicating that active yaw control helps improve the vehicle's attitude response and lateral stability during rapid path changes. The lateral displacement curves of NMPC active yaw and Safe-SAC active yaw are closer to the desired path, indicating that both can effectively suppress tracking errors. A comprehensive comparison shows that the Safe-SAC active yaw method, while maintaining the nonlinear decision-making capability of the SAC controller, uses a model prediction safety filter layer to perform safety checks and minimum corrections on candidate front wheel steering angles and active yaw control forces, enabling the vehicle's lateral displacement to follow the desired path better. Therefore, Figure 9 This invention demonstrates that the Safe-SAC active tilt control method proposed in this invention can improve lateral tracking accuracy during high-speed path tracking while also taking into account vehicle driving stability.

[0209] Figure 10 This image shows a comparison of simulation results for front wheel steering angle under different control methods, primarily comparing the steering control effects of SAC, NMPC, and Safe-SAC during vehicle path tracking. Figure 10It can be seen that during double lane change or lane-change path tracking, the front wheel steering angle changes of the three control methods are basically consistent. All three methods can generate corresponding steering inputs based on changes in the curvature of the reference path, indicating that SAC, NMPC, and Safe-SAC can all complete the vehicle path tracking control task. When the curvature of the path ahead changes, the front wheel steering angle first increases to one side, then changes in the opposite direction to complete lane changing and attitude correction, finally gradually returning to near zero, indicating that the vehicle enters a stable straight-line driving state after completing path tracking. Comparing the three control methods, it can be found that the front wheel steering angle fluctuation of the SAC control method is relatively more obvious, and large steering angle peaks and local abrupt changes are prone to occur during rapid changes in path curvature. This indicates that while reinforcement learning control can learn a certain path tracking strategy, the candidate actions may be too aggressive during high-speed lane changes or high-curvature path tracking, resulting in relatively insufficient control input smoothness and constraint satisfaction. The front wheel steering angle change of the NMPC control method is smoother and can better suppress abrupt steering input changes. This is because NMPC explicitly considers constraints such as the magnitude and rate of change of the front wheel steering angle during the optimization process, thus its control input exhibits good constraint satisfaction and stability. The front wheel steering angle variation trend of the Safe-SAC control method is basically consistent with SAC, but compared to SAC, its peak angle and local fluctuations are reduced, resulting in a smoother control input. This indicates that the model prediction safety filter layer can, while preserving the original control intent of the SAC controller, make minimal necessary corrections to candidate front wheel steering angles that may cause constraint violations or overreactions, thereby improving the safety and smoothness of the control input. Therefore, Figure 10 This demonstrates that the Safe-SAC method proposed in this invention can suppress sudden changes in front wheel steering angle by using a model prediction safety filter layer while maintaining the strong nonlinear decision-making capability of SAC, making the final output steering control quantity smoother and safer, and meeting the physical constraint requirements of the vehicle actuator.

[0210] Figure 11-13 This is a comparison chart of vehicle body sway angle and lateral load transfer rate for different control methods in this invention.

[0211] Figure 11 This section presents a comparison of vehicle yaw angles (i.e., roll angles). The black curve represents the desired yaw angle, the blue curve represents the SAC control result, the green curve represents the NMPC control result, and the purple curve represents the Safe-SAC control result. From... Figure 11 As can be seen, all three control methods can make the vehicle body yaw angle follow the desired yaw angle, but the tracking performance of different methods varies. The body yaw angle under SAC control exhibits a certain lag and fluctuation during the rapid change phase of the path curvature; NMPC and Safe-SAC can better follow the changes in the desired yaw angle, indicating that they have better adjustment capabilities for the vehicle body yaw attitude.

[0212] Figure 12 The results show a comparison of vehicle roll angle (i.e., side roll angle) errors. It can be seen that the SAC control method exhibits significant roll angle errors during rapid lane changes or double lane changes, especially with noticeable error peaks during rapid changes in path direction. The NMPC and Safe-SAC methods show smaller overall roll angle errors, indicating that model predictive control and model predictive safety filtering mechanisms effectively suppress vehicle roll angle deviations. Specifically, Safe-SAC reduces the impact of unsafe or overly aggressive control actions on the vehicle roll response by performing safety checks and minimum necessary corrections on SAC candidate actions.

[0213] Figure 13 This section presents the comparison results of lateral load transfer rates. The lateral load transfer rate characterizes the degree of vertical load transfer between the left and right wheels of a vehicle; a larger absolute value indicates a higher risk to vehicle roll stability. As shown in the figure, the SAC control method exhibits significant fluctuations and a relatively high peak value during changes in the lateral load transfer rate, indicating that relying solely on reinforcement learning control can easily lead to substantial load transfer risks during high-speed obstacle avoidance or high-curvature path tracking. In contrast, NMPC and Safe-SAC better limit the lateral load transfer rate, keeping the vehicle within a safer roll stability range.

[0214] comprehensive Figure 11-13 It is evident that while SAC possesses a certain degree of active roll control capability, it is prone to significant roll angle errors and lateral load transfer fluctuations during periods of intense vehicle dynamic response. NMPC exhibits superior constraint handling capabilities, effectively controlling vehicle roll angle and lateral load transfer rate. Safe-SAC, while retaining the nonlinear decision-making capabilities of SAC, uses a model prediction safety filter layer to constrain and minimize candidate front wheel steering angles and active roll control forces, thereby reducing the risk of vehicle roll angle errors and lateral load transfer while maintaining good path-following performance. Therefore, Figure 11-13 This invention demonstrates that the Safe-SAC method proposed in this invention can improve vehicle attitude stability and roll safety during high-speed path tracking. The scope of protection of this invention includes, but is not limited to, the above embodiments. The scope of protection of this invention is defined by the claims. Any substitutions, modifications, or improvements to this technology that are readily conceived by those skilled in the art fall within the scope of protection of this invention.

Claims

1. A safety reinforcement learning-based vehicle path tracking control method integrating active tilting, characterized in that, Includes the following steps: Step S1: Collect the vehicle's current driving status and reference path information, and calculate the vehicle path tracking and tilting status information based on the vehicle's current driving status and reference path information; The vehicle path tracking and tilt status information includes the vehicle's lateral error, heading error, lateral error change rate, heading error change rate, current front wheel angle, front wheel angle change rate, error between the actual tilt angle and the target tilt angle, and vehicle tilt angular velocity. The reference path information includes reference path position, reference heading angle, and reference curvature, which are used to assist in calculating vehicle path tracking and tilt state information; And it provides predictive input for subsequent model predictions of the security filtering layer; Step S2: Using the vehicle path tracking and tilt state information obtained in step S1 as the input state of the SAC reinforcement learning agent, and the front wheel steering angle and active tilt control force as the action output of the SAC reinforcement learning agent, construct the SAC reinforcement learning controller. Step S3: Input the current vehicle path tracking and tilt state information obtained in step S1 into the trained SAC reinforcement learning controller, and output candidate control actions, including candidate front wheel steering angle and candidate active tilt control force. Step S4: Set up a model prediction safety filter layer between the SAC reinforcement learning controller and the vehicle actuator. Using the candidate front wheel steering angle and candidate active yaw control force output in step S3 as reference inputs, and combining the current vehicle state and the vehicle prediction model, predict the changes in vehicle state and control input caused by the candidate control action in the short time domain in the future, and obtain the predicted vehicle state and predicted control input sequence in the future prediction time domain. The current vehicle status includes at least the vehicle's lateral error, heading error, lateral velocity, yaw rate, body roll angle, and roll rate. Step S5: Based on the vehicle's lateral dynamics model, roll dynamics model, and lateral load transfer relationship, construct a two-dimensional coupled safe feasible domain, i.e., a dual-input safe channel, which is jointly determined by the front wheel steering angle and the active roll control force under the current vehicle state. The dual-input safe channel is a set of all control actions that simultaneously satisfy the safety constraints under the current vehicle state: the safety constraints include vehicle input constraints, input rate of change constraints, vehicle state constraints, and lateral stability constraints; the control actions are composed of the safe front wheel steering angle and the safe active roll control force. Within the dual-input safety channel, the current candidate front wheel steering angle and the candidate active yaw control force satisfy the vehicle input constraints and the rate of change of input constraints, and the combination of the front wheel steering angle and the active yaw control force will not cause any vehicle state constraints or lateral stability constraints to be violated in the prediction time domain. Step S6: Compare the candidate front wheel steering angle and candidate active yaw control force at the current moment, as well as the predicted vehicle state and predicted control input sequence obtained in step S4, with the safety constraints described in step S5 one by one. If the candidate front wheel steering angle and candidate active sway control force satisfy the vehicle input constraints and input rate of change constraints, and all predicted vehicle states and predicted control input sequences in the prediction time domain satisfy the safety constraints, then the candidate control action output in step S3 is located in the dual-input safety channel in step S5. The candidate control action is not modified, and the candidate front wheel steering angle and candidate active sway control force are directly output as the final control action. If the candidate front wheel steering angle or candidate active sway control force does not meet the vehicle input constraints and input rate of change constraints, or if any predicted vehicle state or predicted control input sequence in the prediction time domain does not meet the safety constraints, then the candidate control action output in step S3 is determined to be not located in the dual-input safety channel described in step S5. The safety control action that is closest to the candidate control action and meets the safety constraints is solved through online optimization, and the corrected safety front wheel steering angle and safety active sway control force are output as the final control action. Step S7: Apply the final control action output in step S6 to the vehicle actuator; In the next sampling period, the current driving state of the vehicle is reacquired and steps S1 to S7 are repeated to form a rolling updated safety reinforcement learning path tracking control process.

2. The integrated active tilting safety reinforcement learning vehicle path tracking control method according to claim 1, characterized in that, The construction of the SAC reinforcement learning controller in step S2 specifically includes: Step S2.1: Construct the SAC strategy network. The SAC strategy network takes the vehicle path tracking and tilt state information described in step S1 as input, outputs the normalized front wheel steering angle and the normalized active tilt control force, and converts them into actual candidate front wheel steering angle and candidate active tilt control force through proportional mapping. Step S2.2: Construct a SAC dual-critic network, which includes a first critic network and a second critic network. The first critic network and the second critic network have the same structure but their network parameters are independent of each other. They are used to evaluate the comprehensive action value of performing a dual-input action under the current vehicle state. The smaller value between the output value of the first critic network and the output value of the second critic network is used as the action value estimation result to reduce the problem of overestimation of action value. Step S2.3: Construct the reward function, wherein the reward function is: ； in, For lateral error, For heading error, This represents the rate of change of the lateral error. The rate of change of heading error. This represents the error between the actual yaw angle and the target yaw angle of the vehicle body. For the maximum tilt angle, The angular velocity of the vehicle body tilt. For the current front wheel steering angle, This is the upper limit of the front wheel steering angle. For the current active tilt control force, This is the upper limit of the active tilt control force. As an additional penalty, an additional penalty is triggered when the absolute value of the lateral error exceeds the lateral error safety threshold, or the absolute value of the vehicle body sway angle exceeds the sway angle safety threshold, or the absolute value of the lateral load transfer rate exceeds the lateral load transfer rate safety threshold; otherwise, no additional penalty is applied.

3. The integrated active tilting safety reinforcement learning vehicle path tracking control method according to claim 1, characterized in that, The model prediction safety filtering layer in step S4 includes: Step S4.1: Establish a vehicle prediction model: Based on the vehicle's lateral dynamics model, path error model, and roll dynamics model, establish a vehicle prediction model for short-time-domain state prediction. This model uses the current vehicle state and candidate control actions as inputs to recursively calculate the predicted vehicle state and predicted control input sequence for several future sampling times within a finite prediction time domain. The predicted vehicle state includes the vehicle's lateral error, heading error, lateral velocity, yaw rate, body roll angle, roll rate, and lateral load transfer rate. The predictive control input sequence includes the front wheel steering angle and active yaw control force for each future prediction step; Step S4.2: Construct safety constraints, which include at least the following constraints: front wheel steering angle amplitude constraint, front wheel steering angle change rate constraint, active yaw control force amplitude constraint, active yaw control force change rate constraint, lateral error constraint, heading error constraint, lateral velocity constraint, yaw rate constraint, vehicle body yaw angle constraint, yaw rate constraint, and lateral load transfer rate constraint. Step S4.3: The candidate front wheel steering angle and candidate active yaw control force output by the SAC reinforcement learning controller are used as safety verification objects and substituted into the vehicle prediction model established in step S4.

1. The future state of the vehicle and the control input are recursively calculated within the finite prediction time domain to obtain the predicted vehicle state and predicted control input sequence.

4. The integrated active tilting safety reinforcement learning vehicle path tracking control method according to claim 1, characterized in that, The vehicle input constraints in step S5 specifically include front wheel steering angle amplitude constraints and active yaw control force amplitude constraints; the input rate of change constraints specifically include front wheel steering angle rate of change constraints and active yaw control force rate of change constraints; the vehicle state constraints specifically include lateral error constraints, heading error constraints, lateral velocity constraints, yaw rate constraints, vehicle body yaw angle constraints, and yaw rate constraints; the lateral stability constraints specifically include lateral load transfer rate constraints.

5. The integrated active tilting safety reinforcement learning vehicle path tracking control method according to claim 1, characterized in that, The step S6, which involves finding the safe control action that is closest to the candidate control action and meets the safety constraints through online optimization, specifically includes: Step S6.1: Construct safety action correction and optimization goals: With the goal of making the corrected front wheel steering angle and active yaw control force as close as possible to the candidate front wheel steering angle and candidate active yaw control force output by the SAC reinforcement learning controller, a safety action correction optimization objective function is constructed. The objective function for safety action correction and optimization is: ； Where J is the objective function; D u This is a normalization matrix for the motion amplitude, used to unify the dimensions of the front wheel steering angle and the active roll control force; 𝑘+j|𝑘 The control input for the j-th step predicted at the current time k includes the front wheel steering angle and active yaw control force; RL (k) represents the candidate control action output by the SAC reinforcement learning controller at the current time k; D 𝛥𝑢 To control the incremental normalization matrix, used to unify the dimensions of the front wheel steering angle change and the active roll control force change; 𝛥𝑢 𝑘+j|𝑘 For the change in control input in the j-th step in the future; N c To control the length of the time domain, let represent the number of control input steps that need to be solved in the online optimization; where k represents the current sampling time, and j represents the j-th control step in the control time domain. This is the action deviation weight matrix. To control the incremental weight matrix, As slack variables, The penalty coefficient for slack variables; Step S6.2: Construct a constrained quadratic programming problem: Using vehicle safety constraints as optimization constraints and the safety action correction optimization objective described in step S6.1 as the optimization objective, a constrained quadratic programming problem is constructed. Step S6.3, Solve for safety control actions: By solving the quadratic programming problem online, the optimal control sequence that satisfies the vehicle safety constraints and minimizes the deviation from the candidate action output by the SAC reinforcement learning controller is obtained in the prediction time domain at the current time k. The optimal control sequence includes multiple sets of control inputs in the current time and several future control time domains. Each set of control inputs consists of the front wheel steering angle and the active tilt control force. Step S6.4: Output the final control action: The first set of control inputs in the optimal control sequence obtained in step S6.3 is the safety control action at the current moment, including the safe front wheel steering angle and the safe active sway control force. The safe front wheel steering angle and the safe active sway control force are used as the final control action.

Images