A contradiction-driven collaborative control method and system for reinforcement learning of a chassis

Abstract

This invention discloses a chassis reinforcement learning collaborative control method and system based on contradiction-driven control, belonging to the field of vehicle dynamics control. The method includes: acquiring the vehicle state, calculating the ideal yaw rate, defining a stability boundary function based on the phase plane, and calculating the contradiction index; combining the vehicle state, contradiction index, and underlying commands into a full-scale state input feature vector; constructing an Actor-Critic-based reinforcement learning network model including an encoder, a master policy head, and a dual explorer head, training it to maximize cumulative rewards and employing a conflict-avoidance gradient descent algorithm, and removing the dual explorer head after convergence; during actual vehicle operation, inputting the state vector into the model, outputting the rear wheel steering angle residual compensation and yaw moment residual compensation, superimposing them onto the underlying commands, and distributing them to each wheel motor after amplitude limiting. This invention achieves contradiction-driven collaborative control of active rear wheel steering and direct yaw moment, improving vehicle stability and ride comfort under extreme conditions.

Description

Technical Field

[0001] This invention relates to the field of vehicle dynamics control technology, specifically to a chassis reinforcement learning cooperative control method and system based on contradiction-driven processes. Background Technology

[0002] In the evolution of modern automotive engineering, the integration and intelligence of chassis control systems have become the core of measuring vehicle active safety and ride quality. With the widespread adoption of steer-by-wire technology, Active Rear Steering (ARS) and Direct Yaw Moment Control (DYC), as the actuators that directly control the vehicle's yaw dynamics, directly determine the vehicle's handling limits through their synergistic effectiveness.

[0003] Traditional chassis control logic typically employs independent design or switching rules based on fixed thresholds. This architecture performs reasonably well under simple conditions where the tires are in a linear range, but at the limits of dynamics, such as high-speed emergency obstacle avoidance or road surface conditions with abrupt changes in adhesion coefficient, severe coupling conflicts arise between the ARS and DYC. The ARS obtains lateral force through rear wheel steering angle to adjust vehicle attitude; while the DYC generates additional yaw moment through torque vector distribution. According to the tire friction circle principle, when the vehicle is on the verge of instability and the DYC intervenes forcefully, outputting large drive torque for attitude correction, it drastically depletes the longitudinal adhesion limit of the relevant wheels, causing a significant decrease or even complete saturation of the available lateral force upon which the ARS relies. Without dynamic coordination, the ARS is prone to entering a nonlinear control region, resulting in tire slippage; simultaneously, the rear wheel steering angle generated by the ARS directly changes the vector direction of the DYC's longitudinal driving force, causing the actual yaw moment output of the DYC to deviate from the expected direction. The mutual interference between the two in utilizing lateral and longitudinal forces not only fails to achieve better control but can also lead to loss of vehicle control.

[0004] Existing collaborative control research, such as model predictive control or traditional end-to-end reinforcement learning, has made some progress in multi-objective optimization, but still faces three major technical bottlenecks: First, methods such as MPC rely too heavily on accurate vehicle mathematical models, making it difficult to adapt to the extremely strong nonlinear saturation characteristics of tires under extreme conditions; Second, traditional reinforcement learning collaboration often attempts to directly replace the output physical quantities of the underlying controller, which compromises safety and, due to the large trial-and-error space, is extremely difficult to converge in real-world vehicle deployments; Third, the allocation of permissions between controllers often relies on static rule dead zones, making it impossible to quantify and resolve coupling conflicts in real time based on vehicle instability conditions. Summary of the Invention

[0005] The purpose of this invention is to provide a chassis reinforcement learning-based cooperative control method and system based on contradiction-driven design, in order to solve the problems in existing chassis cooperative control, such as the severe coupling conflict between active rear wheel steering and direct yaw torque under extreme conditions, reliance on accurate models, and static and rigid permission allocation, which makes it difficult to balance stability and ride comfort.

[0006] To achieve the above objectives, the technical solution provided by this invention is: a chassis reinforcement learning collaborative control method based on contradiction-driven mechanisms, comprising the following steps: S1: Obtain real-time vehicle status parameters and calculate the ideal yaw rate; define a stable boundary function based on the phase plane of the center of mass sideslip angle-center of mass sideslip rate, and calculate the contradiction intensity index according to the degree of deviation of the phase plane trajectory from the stable boundary. S2: Obtain the rear wheel steering angle command output by the underlying active rear wheel steering system controller and the yaw moment command output by the underlying direct yaw moment system controller; combine the vehicle's real-time state parameters, ideal yaw rate, contradiction strength index, rear wheel steering angle command, and yaw moment command into a full-scale state input feature vector; the active rear wheel steering system adjusts the vehicle's attitude by changing the rear wheel steering angle, and the direct yaw moment system generates additional yaw moment by distributing the longitudinal driving force of the wheels; S3: Construct a reinforcement learning network model including an Actor network and a Critic network. The Actor network includes a shared feature encoder, a master policy head, and a dual explorer head. During model training, the gradient vectors of the master policy head objective function and the dual explorer head objective function with respect to the shared feature encoder are calculated respectively. The inner product of the two gradient vectors is used to determine whether there is a gradient conflict. If there is a conflict, the conflict avoidance gradient descent algorithm is used to calculate the convex combination of the two gradients as the fused gradient. The shared feature encoder is updated by the fused gradient, while the master policy head and the dual explorer head are updated by their respective independent gradients. After training converges, the dual explorer head is removed, and the shared feature encoder and the master policy head are retained. S4: In actual vehicle operation, the full state input feature vector is input into the reinforcement learning network model, and the residual compensation of the rear wheel steering angle and the residual compensation of the yaw moment are output. The residual compensation of the rear wheel steering angle is superimposed on the rear wheel steering angle command, and the residual compensation of the yaw moment is superimposed on the yaw moment command to obtain the composite command. The composite command is then subjected to rate of change limit and absolute amplitude limit in sequence to generate the final control command. The torque optimization allocation function is used to allocate the final control command to each wheel drive motor.

[0007] To optimize the above technical solution, the specific measures also include: In step S1, the process of acquiring real-time vehicle state parameters and calculating the ideal yaw rate is as follows: The vehicle's real-time status parameters include longitudinal speed, actual yaw rate, yaw rate deviation, center of gravity sideslip angle, center of gravity sideslip rate, lateral acceleration, road adhesion coefficient, and vehicle front wheel steering angle; Establish a two-degree-of-freedom vehicle dynamics reference model and calculate the ideal yaw rate: ; Furthermore, considering road surface adhesion limitations, the ideal yaw rate is constrained: ; The expression for the ideal yaw rate is: ; in, Ideal yaw rate; The road surface adhesion coefficient, It is the acceleration due to gravity; Longitudinal velocity; For vehicle stability factors, ; These are the front and rear axle lateral stiffness, respectively. Wheelbase , This is the distance from the center of mass to the front and rear axles; This refers to the steering angle of the vehicle's front wheels.

[0008] In step S1, the stability boundary function is defined based on the phase plane of the center of mass sideslip angle and the center of mass sideslip angular velocity. The contradiction intensity index is calculated according to the degree to which the phase plane trajectory deviates from the stability boundary. The specific process is as follows: Define stable boundary functions The expression is: ; Contradiction Intensity Index The calculation formula is as follows: ; in, This is the actual sideslip angle of the vehicle's center of gravity. This refers to the vehicle's actual sideslip angular velocity at its center of gravity. The slope parameter of the phase plane boundary; The preset instability and contradiction triggers a safety threshold; This represents the maximum phase plane divergence value under the vehicle's physical limits.

[0009] In step S3, the reinforcement learning network model is trained with the objective of maximizing the cumulative reward function. Rewards from yaw stability Lateral stability reward Dynamic intervention smoothness reward Actuator control energy consumption penalty And instability and boundary crossing termination of punishment The weighted composition is as follows: ; Furthermore, yaw stability bonus for: ; Lateral stability reward for: ; Dynamic intervention smoothness reward for: ; Actuator control energy consumption penalty for: ; Instability and boundary crossing will result in termination of penalty. for: ; in, These are the dimensionless weight coefficients for each reward sub-item; For the first The actual yaw rate of each control cycle; For the first The deviation between the actual yaw rate and the expected yaw rate in each control cycle; These are the sideslip angle penalty coefficient and the sideslip angular velocity penalty coefficient, respectively. For the first Yaw moment residual compensation amount for each control cycle; For the first intelligent agent -1 is the residual compensation amount of the yaw moment output during the control cycle; For the first The residual compensation amount of the rear wheel steering angle in each control cycle; These are the ARS action penalty coefficient and the DYC action penalty coefficient, respectively. This is a preset fixed out-of-bounds penalty constant.

[0010] In step S3, the step of calculating the gradient vectors of the main policy head objective function and the dual exploration head objective function with respect to the shared feature encoder is specifically as follows: The gradient vector of the main policy head objective function with respect to the shared feature encoder The calculation formula is: ; Furthermore, the main strategy head objective function for: ; The gradient vector of the dual exploration head objective function with respect to the shared feature encoder The calculation formula is: ; Dual exploration head objective function for: ; in, For network weight parameters of the shared feature encoder; The network weight parameters are the main strategy head; For mathematical expectation operators; For the first The full state input feature vector of each control cycle; To reinforce the experience replay pool for learning; For the first The action vector output by the agent in each control cycle; The probability distribution of actions output by the main strategy head; The Q-score is the result of evaluations by a network of action value critics. For the weight parameters of the critic network; Entropy regularization weights; For the network weight parameters of the dual explorer head; The action probability distribution of the dual exploration head output; The set boundary exploration penalty amplifies the weight constant.

[0011] In step S4, the composite command is sequentially subjected to rate-of-change limiting and absolute amplitude limiting to generate the final control command. The specific process is as follows: The rate of change limit is calculated by taking the difference between the composite instruction and the final control instruction of the previous cycle, and then constraining this difference within the maximum action rate allowed by the hardware to obtain the smoothed instruction. ; ; Furthermore, the absolute amplitude limiting method forces the smoothed command to be confined within the physical limit range of the actuator, resulting in the final control command: ; ; in, This is the standard truncation and amplitude limiting function; For the first The instruction increment after the rate of change is limited for each control cycle; For the first Smoothing instructions after rate of change limiting for each control cycle; For the first The final control command for each control cycle; This represents the physical maximum operating rate boundary of the actuator. and The physical maximum value of the rear wheel steering angle that the vehicle can provide; The physical extreme value of the yaw moment that the vehicle can provide; For the first A composite instruction for each control cycle; To control the sampling period; For the first The final control command output by the active rear wheel steering system controller in each control cycle; For the first The smooth command output by the active rear wheel steering system controller during the control cycle; For the first The final control command output by the bottom-level direct yaw moment system controller in each control cycle; For the first Each control cycle directly outputs a smooth command from the yaw moment system controller.

[0012] In step S4, the final control command is distributed to each wheel drive motor using a torque optimization allocation function. Specifically, the torque optimization allocation function is as follows:

[0013] in, This represents the peak torque of the motor. Vertical load for a single tire; The radius of the tire; The total torque required by the driver; and These are the track widths of the front and rear wheels, respectively. For the desired driving torque of a single wheel, These represent the front left wheel, front right wheel, rear left wheel, and rear right wheel, respectively.

[0014] As another important technical solution, the present invention also provides a chassis reinforcement learning collaborative control system based on contradiction-driven mechanisms, comprising: The vehicle state perception and contradiction index calculation module is used to acquire real-time vehicle state parameters and calculate the ideal yaw rate; based on the phase plane of the center of mass sideslip angle-center of mass sideslip rate, a stable boundary function is defined, and the contradiction intensity index is calculated according to the degree of deviation of the phase plane trajectory from the stable boundary. The underlying command acquisition and state stitching module is used to acquire the rear wheel steering angle command output by the underlying active rear wheel steering system controller and the yaw moment command output by the underlying direct yaw moment system controller; it combines the vehicle's real-time state parameters, ideal yaw rate, contradiction intensity index, rear wheel steering angle command, and yaw moment command into a full-quantity state input feature vector; the active rear wheel steering system adjusts the vehicle attitude by changing the rear wheel steering angle, and the direct yaw moment system generates additional yaw moment by distributing the longitudinal driving force of the wheels; The dual-head reinforcement learning training module is used to construct a reinforcement learning network model including an Actor network and a Critic network. The Actor network includes a shared feature encoder, a master policy head, and a dual explorer head. During model training, the gradient vectors of the objective functions of the master policy head and the dual explorer head with respect to the shared feature encoder are calculated respectively. The inner product of the two gradient vectors is used to determine whether there is a gradient conflict. If there is a conflict, a conflict-avoiding gradient descent algorithm is used to calculate the convex combination of the two gradients as the fused gradient. The shared feature encoder is updated by the fused gradient, while the master policy head and the dual explorer head are updated by their respective independent gradients. After training converges, the dual explorer head is removed, and the shared feature encoder and the master policy head are retained. The residual compensation and safety limiting execution module is used to input the full state input feature vector into the reinforcement learning network model during actual vehicle operation, and output the residual compensation amount of the rear wheel steering angle and the residual compensation amount of the yaw moment. The residual compensation amount of the rear wheel steering angle is superimposed on the rear wheel steering angle command, and the residual compensation amount of the yaw moment is superimposed on the yaw moment command to obtain a composite command. The composite command is then subjected to rate of change limiting and absolute amplitude limiting in sequence to generate the final control command. The final control command is distributed to each wheel drive motor using a torque optimization allocation function.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention is the first to introduce a conflict-driven mechanism into chassis cooperative control. It quantifies the competition between active rear-wheel steering (ARS) and direct yaw moment (DYC) on the tire friction circle in real time through the phase plane of the center-of-gravity sideslip angle-center-of-gravity sideslip velocity, and explicitly incorporates the conflict intensity index into the reinforcement learning state space. When the vehicle approaches its physical limits, the conflict index automatically activates the DYC compensation authority and reduces the ARS compensation amount, achieving a smooth and seamless transition from smooth tracking to extreme anti-skid, fundamentally solving the chassis torque vibration and loss-of-control risks caused by traditional rule switching.

[0016] This invention employs a reinforcement learning network model and the Conflict Avoidance Gradient Descent (CAGrad) algorithm, eliminating the need for a complex tire nonlinear mathematical model. During training, the main policy head pursues global comprehensive benefits and smoothness, while the dual exploration head specializes in exploring sideslip boundaries. The gradient conflicts generated by the two in the shared feature encoding layer are transformed into fused gradients by the CAGrad algorithm, enabling the shared layer to simultaneously acquire both conservative and aggressive complementary knowledge. After training convergence, the dual exploration head is removed, retaining only the lightweight main policy network. This preserves the core capability against extreme instability while avoiding the deployment challenges of an excessively large trial-and-error space in end-to-end reinforcement learning.

[0017] In this invention, reinforcement learning does not directly output the full physical action, but instead performs minor residual compensation on the independent instructions of the underlying ARS / DYC controller; the underlying controller serves as a physical safety net, always providing basic stability control; the agent only fine-tunes in the linear region or superimposes compensation in the extreme region, significantly reducing the safety risks of deployment in real vehicles; the output instructions are limited by the rate of change and the absolute amplitude, strictly constrained within the hardware mechanical limits and the physical boundaries of the motor, ensuring that no over-limit instructions are issued under any circumstances.

[0018] In this invention, the cumulative reward function comprehensively considers yaw tracking accuracy, centroid side slip angle suppression, motion rate of change smoothness, actuator energy consumption, and instability termination penalty, and flexibly adjusts various indicators through weight coefficients; the trained strategy can automatically adjust the intervention depth of ARS and DYC according to the contradiction intensity index: when the contradiction is low, ARS is used first for smooth adjustment, and when the contradiction is high, DYC large torque correction is decisively called, thereby achieving the best balance between stability and ride comfort under all working conditions. Attached Figure Description

[0019] Figure 1 : A schematic diagram of the process in an embodiment of the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below through specific embodiments, but it should not be construed as limiting the scope of the subject matter of the present invention to the following embodiments. All technologies implemented based on the above content of the present invention fall within the scope of the present invention.

[0021] In the description of this invention, it should also be noted that: In some embodiments, the cooperative control architecture proposed in this invention includes the following underlying physical actuators for cooperation: Active Rear Wheel Steering (ARS): Changes the rear wheel steering angle via an electric motor. It uses the lateral force of the tires to adjust the center of gravity sideslip angle, providing counter-steering at low and medium speeds to enhance maneuverability, and providing in-phase steering at high speeds to smoothly maintain vehicle tracking stability.

[0022] Direct Yaw Moment (DYC) system based on drive control: By applying longitudinal differential torque to the left and right wheels through a distributed drive motor or torque vectoring system, a yaw moment is generated directly around the center of mass. This is to correct the vehicle's posture.

[0023] The basic control execution unit (ECU) in the chassis domain controller calculates and outputs baseline control commands to maintain the vehicle's basic dynamic performance based on real-time vehicle status information, driver input, and a reference model during vehicle operation. In this invention, the basic control algorithms of the Active Rear Wheel Steering (ARS) system, the Direct Yaw Moment (DYC) system, and the top-level collaborative control module are all integrated within the same ECU. Specifically, the ECU includes the Active Rear Wheel Steering system, the Direct Yaw Moment system, and the top-level collaborative control module. The Active Rear Wheel Steering system outputs basic rear wheel steering angle commands, the Direct Yaw Moment system outputs basic yaw moment commands, and the top-level collaborative control module further outputs rear wheel steering angle residual compensation and yaw moment residual compensation based on the basic commands, thereby achieving coordinated control of the conflicting forces between the Active Rear Wheel Steering and Direct Yaw Moment.

[0024] In some implementations, this invention employs a residual control architecture, meaning that reinforcement learning does not directly output the full physical action, but instead relies on the initial output of the underlying controller for superposition compensation. To maximize the versatility of this system, this invention does not exhaustively enumerate or limit the specific control laws (such as PID, LQR, MPC, or lookup table methods) of the underlying ARS and DYC. The ECU only needs to independently output initial control commands to maintain basic dynamic performance based on the current vehicle state and reference model. Specifically: Active rear-wheel steering system is based on the current longitudinal speed of the vehicle. Front wheel steering angle of the vehicle Based on the vehicle status, output the basic rear wheel steering angle command. .

[0025] Direct yaw moment system is based on actual yaw rate yaw rate of the target The deviation will trigger the output of the basic yaw moment command. .

[0026] Basic instructions calculated by the ECU and This will be directly used as part of the known environment variables, i.e., system state input. These basic instructions are fed into the reinforcement learning agent of the top-level collaborative control module. By perceiving these instructions, the agent identifies the underlying local control intentions and, in the event of actuator coupling conflicts, accurately calculates and outputs the residual compensation amount for the rear wheel steering angle. Yaw moment residual compensation .

[0027] In some implementations, such as Figure 1 As shown, this invention provides a chassis reinforcement learning-based collaborative control method driven by contradictions, comprising the following steps: S1: Obtain real-time vehicle status parameters and calculate the ideal yaw rate; define a stable boundary function based on the phase plane of the center of mass sideslip angle-center of mass sideslip rate, and calculate the contradiction intensity index according to the degree of deviation of the phase plane trajectory from the stable boundary. The vehicle's real-time status parameters include longitudinal speed, actual yaw rate, yaw rate deviation, center of gravity sideslip angle, center of gravity sideslip rate, lateral acceleration, road adhesion coefficient, and vehicle front wheel steering angle; Establish a two-degree-of-freedom vehicle dynamics reference model and calculate the ideal yaw rate: ; In some implementations, the ideal yaw rate is constrained to take into account road surface adhesion limitations: ; Ideal yaw rate The expression is: ; in, The road surface adhesion coefficient, It is the acceleration due to gravity; Longitudinal velocity; For vehicle stability factors, ; These are the front and rear axle lateral stiffness, respectively. Wheelbase , This is the distance from the center of mass to the front and rear axles; This refers to the steering angle of the vehicle's front wheels.

[0028] In some implementations, ARS primarily establishes or corrects lateral forces by adjusting the rear wheel steering angle, thereby smoothly adjusting the vehicle's attitude; DYC primarily generates additional yaw moment through the longitudinal differential torque between the left and right wheels to quickly suppress yaw deviation. When the vehicle is under extreme conditions such as high speed, large steering angle, or low adhesion, the deep intervention of DYC will preferentially occupy the tire's longitudinal adhesion margin, thereby compressing the lateral adhesion space available to ARS; conversely, the change in tire force direction caused by ARS will also affect the actual effect of DYC yaw moment. To enable the controller to promptly identify and resolve such coupling conflicts when the vehicle approaches the instability boundary, this invention introduces a center-of-gravity sideslip angle-center-of-gravity sideslip angular velocity phase plane, uniformly mapping the vehicle stability margin and the degree of actuator conflict into a quantifiable conflict intensity index, and dynamically adjusting the compensation magnitude of ARS and DYC accordingly.

[0029] In some implementations, the sideslip angle-slip velocity (SLAG) is used. The phase plane method is used to quantify this contradiction, and a stable boundary function is defined. The expression is: ; Define the contradiction intensity index The calculation formula is as follows: ; in, This is the actual sideslip angle of the vehicle's center of gravity. This refers to the vehicle's actual sideslip angular velocity at its center of gravity. The slope parameter of the phase plane boundary; The preset instability and contradiction triggers a safety threshold; This represents the maximum phase plane divergence value under the vehicle's physical limits. The larger the value, the further the current vehicle state deviates from the origin in the phase plane, meaning it is closer to the edge of losing control.

[0030] In some implementations, when At this time: This indicates that the vehicle is in an absolutely stable region, and the desired state can be tracked solely by ARS, while DYC is in a dormant state. Both perform their respective functions, and there is no friction circle competition. Time: This indicates that the vehicle is approaching its physical limits, and there is a contradiction between ARS and DYC in their utilization of lateral and longitudinal adhesion. The larger the value, the more intense the contradiction.

[0031] S2: Obtain the rear wheel steering angle command output by the ARS controller and the yaw moment command output by the DYC controller; combine the real-time vehicle state parameters, ideal yaw rate, contradiction intensity index, rear wheel steering angle command and yaw moment command into a full state input feature vector; ARS adjusts the vehicle attitude by changing the rear wheel steering angle, and DYC generates additional yaw moment by distributing the longitudinal driving force of the wheels; In order for the agent of the top-level collaborative control module to accurately assess the current level of vehicle instability and the saturation of the lower-level controller, in the... Each control cycle constructs a full-state input feature vector containing multi-dimensional features. : ; in, For the first Longitudinal vehicle speed per control cycle; For the first The actual yaw rate of each control cycle; For the first The deviation between the actual yaw rate and the expected yaw rate in each control cycle; For the vehicle in the The actual centroid sideslip angle for each control cycle; For the vehicle in the The actual centroid sideslip angular velocity of each control cycle; For the first Lateral acceleration per control cycle; For the first The road surface adhesion coefficient for each control cycle (estimated by the road surface observer); For the first The front wheel steering angle input by the driver in each control cycle; For the first The basic yaw moment command for each control cycle; For the first The basic rear wheel steering angle command for each control cycle; superscript Indicates vector transpose; For the first Phase plane contradiction intensity index for each control cycle.

[0032] The intelligent agent in the top-level collaborative control module does not directly control the actuators, but instead outputs correction and compensation amounts for the underlying instructions. The action output during each control cycle is defined as a two-dimensional continuous vector: ; in, For the first Rear wheel steering angle residual compensation for each control cycle: Under normal low-speed or slight slip conditions, the agent fine-tunes this value and adds it to the control cycle. Furthermore, the linear lateral force potential of the rear wheels is utilized to improve tracking of the desired state. For the first Yaw moment residual compensation for each control cycle: When the vehicle enters extreme operating conditions and the ARS lateral force saturates and fails, the agent outputs this compensation amount, which is superimposed on the... Above that, relying on the longitudinal differential torque of the distributed drive motors, a yaw moment is generated to pull the vehicle back to the stable region.

[0033] In some implementations, through this residual action design, the actual instructions ultimately issued to the hardware consist of bottom-level basic instructions and top-level residual compensation. That is, the agent of the top-level collaborative control module does not bypass the bottom-level ARS / DYC controller to directly generate full control instructions. Instead, it makes small-scale modifications based on the safety-guaranteed control instructions already provided by the bottom-level controller. This preserves the stable control capability of the bottom-level controller under normal operating conditions while enabling the agent to implement targeted collaborative compensation when conflicts escalate or approach extreme boundaries. This balances control safety, interpretability, and feasibility for deployment in real vehicles. Specifically: ; ; in, For the first The final expected output angle for each control cycle; For the first The final expected output yaw torque for each control cycle.

[0034] S3: Construct a reinforcement learning network model including an Actor network and a Critic network. The Actor network consists of a shared feature encoder, a master policy head, and a dual explorer head. The Critic network is a critic network. This reinforcement learning network model is trained with the objective of maximizing the cumulative reward function. During training, the gradient vectors of the master policy head objective function and the dual explorer head objective function with respect to the shared feature encoder are calculated. When the inner product of the two gradient vectors is negative, it is determined to be a gradient conflict. A conflict-avoiding gradient descent algorithm is used to calculate the convex combination of the two gradients as the fused gradient. The shared feature encoder is updated through the fused gradient, while the master policy head and the dual explorer head are updated through their respective independent gradients. After training converges, the dual explorer head is removed, and the shared feature encoder and the master policy head are retained. In some implementations, an experience replay pool is constructed to support reinforcement learning training. It is used to store sample quadruples of state, action, reward and next state for each control cycle. ,in, For the first The full state input feature vector of each control cycle For the first Action vector for each control cycle For the first Instant rewards within each control cycle for The full state input feature vector of the control cycle. The shared feature encoder, as the shared front end of the reinforcement learning network model, receives the full state input feature vector and jointly encodes the vehicle operating state, underlying basic commands, and conflict intensity index. It extracts shared hidden layer features that characterize the vehicle's dynamic evolution and actuator coupling relationships, which are then used by the main strategy head and the dual exploration head for subsequent decision-making. Its network weight parameters are defined as follows: .

[0035] The main strategy head is used to output a coordinated control strategy that takes into account yaw stability, sideslip stability, and control smoothness. It represents the vehicle's control requirements under normal driving and slight slip conditions. Its network weight parameters are defined as follows: The dual exploration head is used to enhance the model's ability to explore conditions near the physical boundary of vehicle instability, representing the anti-skid control requirements under extreme contradictory conditions. Its network weight parameters are defined as follows: .

[0036] During the training phase, a critic network is also set up as a value evaluation network to cooperate with the two-headed actor network. The critic network receives the first... Full state input feature vector of each control cycle With action vectors Output the action value function value of the corresponding state-action pair. ,in, These are the weight parameters of the commentator network. The commentator network is used to evaluate the long-term cumulative gain that can be obtained after performing corresponding actions in the current state, and to provide value assessment signals for the parameter updates of the main policy head and the dual explorer head. When the main policy head is updated, the commentator network evaluates the value of actions sampled from the main policy head; when the dual explorer head is updated, the commentator network evaluates the value of actions sampled from the dual explorer head, thus establishing a correspondence between the commentator network and the dual-head Actor design.

[0037] In some implementations, the critic network is trained by minimizing the temporal difference error of the state-action value function, with the loss function expressed as: ; The value of the target action is expressed as: ; in, Discount factor; Entropy regularization weights are used to balance exploration and exploitation; This represents the probability distribution of actions corresponding to the current policy head being updated, when the main policy head is updated. When updating the dual explorer head ; The output of the commentator network is the action value function value of the state-action pair; For mathematical expectation operators.

[0038] In some implementations, the gradient vector of the main policy head objective function with respect to the shared feature encoder is calculated as follows: ; The primary strategy objective function aims to maximize the action value function and policy entropy, focusing on obtaining the overall global benefit while also considering yaw stability, skew stability, and control smoothness. Its expression is: ; The formula for calculating the gradient vector of the dual explorer head objective function with respect to the shared feature encoder is as follows: ; The objective function of the dual explorer head is: ; in, The probability distribution of actions corresponding to the main strategy head; This represents the probability distribution of actions corresponding to the dual exploration head; The boundary exploration penalty is amplified by a constant weight. This is a bonus item for lateral stability. To Gradient operator for partial derivatives; This is the gradient conflict flag; a value of 1 indicates an obtuse angle conflict, and 0 indicates directional cooperation.

[0039] In some implementations, the gradient vectors of the master strategy head and the dual explorer head relative to the parameters of the shared feature encoder are calculated separately during backpropagation, and the inner product of the two gradient vectors is used to determine whether there is a gradient conflict.

[0040] when When this occurs, it indicates a conflict between the two gradients in the update direction of the shared feature layer, and a gradient conflict flag is defined. ; when When, define This indicates that the two gradient directions cooperate. In some implementations, when At this time, a conflict-avoiding gradient descent algorithm is used to fuse gradients, and an optimal fused gradient is found in the local neighborhood. To avoid mutual cancellation: ; in, The optimal fusion gradient vector after resolving the conflict; These are the gradient weight coefficients of the principal policy head obtained by the conflict avoidance gradient descent algorithm; These are the dual explorer head gradient weight coefficients obtained by the conflict avoidance gradient descent algorithm; and Constraints must be met: ; and ; when At that time, the parameters of the shared feature encoder can be updated along the shared direction.

[0041] Update the shared layer using the merged gradient, and update the dual-head parameters using their respective independent gradients:

[0042] in, This indicates a parameter assignment / update operation; The learning rate set for the shared feature encoder; The learning rate set for the main strategy head; The learning rate set for the dual explorer head; To Gradient operator for partial derivatives; To Gradient operator for finding partial derivatives.

[0043] In some implementations, the cumulative reward function Rewards from yaw stability Lateral stability reward Dynamic intervention smoothness reward Actuator control energy consumption penalty And instability and boundary crossing termination of punishment The weighted composition is as follows: ; Yaw stability bonus To penalize the deviation between the actual yaw rate and the target value, ensuring that the vehicle's yaw response converges to the expected stable value, the expression is: ; Lateral stability reward The expression used to suppress the divergence of the centroid sideslip angle and its rate of change, thereby suppressing vehicle sideslip, is: ; Dynamic intervention smoothness reward The expression used to penalize the step change rate of the DYC compensation amount, preventing vehicle jerking caused by high-frequency oscillations and sudden longitudinal slip changes, is: ; Actuator control energy consumption penalty The expression used to constrain the absolute amplitude of the residual action output by the agent, preventing the actuator from overacting, is: ; Instability and boundary crossing will result in termination of penalty. Used when a vehicle is detected to have entered an irreversible state of loss of control (such as...) , When the set out-of-control threshold is reached, a fixed negative constant penalty is applied and the current training round is terminated early. The expression is: ; in, These are the dimensionless weight coefficients for each reward sub-item; These are the sideslip angle penalty coefficient and the sideslip angular velocity penalty coefficient, respectively. For the first intelligent agent The residual compensation amount of the yaw moment output in each control cycle; These are the ARS action penalty coefficient and the DYC action penalty coefficient, respectively. This is a preset fixed out-of-bounds penalty constant.

[0044] In some implementations, the dual exploration head serves only as an auxiliary task during training, used to force the shared feature encoding layer to extract high-dimensional anti-skid features, and is stripped away after training convergence. After model training convergence, the weights of the shared feature encoder and the main strategy head are stripped and solidified, and deployed in the microprocessor of the vehicle chassis domain controller. The main strategy head relies on the fusion of the contradiction intensity index. Its high-dimensional state characteristics enable it to continuously and smoothly output composite instructions for all operating conditions with a single network structure. This not only significantly saves the valuable computing power of automotive-grade microprocessors, but also completely avoids the chassis torque chattering problem caused by hard switching of multiple network rules from a physical perspective.

[0045] S4: In actual vehicle operation, the full state input feature vector is input into the reinforcement learning model, and the residual compensation amounts of the rear wheel steering angle and yaw moment are output. The residual compensation amounts of the rear wheel steering angle are superimposed on the rear wheel steering angle command, and the residual compensation amounts of the yaw moment are superimposed on the yaw moment command to obtain a composite command. The composite command is then subjected to rate of change limiting and absolute amplitude limiting in sequence to generate the final control command. The final control command is distributed to each wheel drive motor using a torque optimization allocation function to achieve the contradictory drive-coordinated control of active rear wheel steering and direct yaw moment.

[0046] In some implementations, in the first One control cycle, the system control sampling cycle is set to... Within the vehicle, the following closed-loop calculation cycle is executed: The electronic control unit independently calculates the basic physical control commands without AI intervention, while the ARS controller calculates the basic rear wheel steering angle. The DYC controller calculates the basic yaw moment. .

[0047] Will The input is fed into the fixed reinforcement learning network model. After forward matrix multiplication and activation function operations, the reinforcement learning model directly outputs the rear wheel steering angle and yaw moment residual compensation. ; The composite instruction for the current cycle is obtained by algebraically adding the basic instruction to the residual compensation amount. The composite command is sequentially limited by both rate of change and absolute amplitude to generate the final control command, preventing the agent from outputting commands that could cause mechanical overshoot or motor overload. The specific process is as follows: The rate of change limit is calculated by taking the difference between the composite instruction and the final control instruction of the previous cycle, and then constraining this difference within the maximum action rate allowed by the hardware to obtain the smoothed instruction. ; ; Absolute amplitude limiting forces the smoothed command to be confined within the physical limit range of the actuator, resulting in the final control command: ; ; in, This is the standard truncation and amplitude limiting function; For the first The instruction increment after the rate of change is limited for each control cycle; For the first Smoothing instructions after rate of change limiting for each control cycle; For the first The final control command for each control cycle; This represents the physical maximum operating rate boundary of the actuator. and The physical maximum value of the rear wheel steering angle that the vehicle can provide; The physical extreme value of the yaw moment that the vehicle can provide; To control the sampling period; For the first The final control command output by the active rear wheel steering system controller in each control cycle; For the first The smooth command output by the active rear wheel steering system controller during the control cycle; For the first The final control command output by the bottom-level direct yaw moment system controller in each control cycle; For the first Each control cycle directly outputs a smooth command from the yaw moment system controller.

[0048] In some implementations, a torque optimization allocation function is used to distribute the final control command to each wheel drive motor. Specifically, the torque optimization allocation function is as follows:

[0049] in, This represents the peak torque of the motor. Vertical load for a single tire; The radius of the tire; The total torque required by the driver; and These are the track widths of the front and rear wheels, respectively. For the desired driving torque of a single wheel, These represent the front left wheel, front right wheel, rear left wheel, and rear right wheel, respectively.

[0050] In another embodiment of the present invention, a chassis reinforcement learning cooperative control system based on contradiction-driven mechanisms is proposed, comprising: The vehicle state perception and contradiction index calculation module is used to acquire real-time vehicle state parameters and calculate the ideal yaw rate; based on the phase plane of the center of mass sideslip angle-center of mass sideslip rate, a stable boundary function is defined, and the contradiction intensity index is calculated according to the degree of deviation of the phase plane trajectory from the stable boundary. The underlying command acquisition and state stitching module is used to acquire the rear wheel steering angle command output by the underlying active rear wheel steering system controller and the yaw moment command output by the underlying direct yaw moment system controller; it combines the vehicle's real-time state parameters, ideal yaw rate, contradiction intensity index, rear wheel steering angle command, and yaw moment command into a full-quantity state input feature vector; the active rear wheel steering system adjusts the vehicle attitude by changing the rear wheel steering angle, and the direct yaw moment system generates additional yaw moment by distributing the longitudinal driving force of the wheels; The dual-head reinforcement learning training module is used to construct a reinforcement learning network model including an Actor network and a Critic network. The Actor network includes a shared feature encoder, a master policy head, and a dual explorer head. During model training, the gradient vectors of the objective functions of the master policy head and the dual explorer head with respect to the shared feature encoder are calculated respectively. The inner product of the two gradient vectors is used to determine whether there is a gradient conflict. If there is a conflict, a conflict-avoiding gradient descent algorithm is used to calculate the convex combination of the two gradients as the fused gradient. The shared feature encoder is updated by the fused gradient, while the master policy head and the dual explorer head are updated by their respective independent gradients. After training converges, the dual explorer head is removed, and the shared feature encoder and the master policy head are retained. The residual compensation and safety limiting execution module is used to input the full state input feature vector into the reinforcement learning network model during actual vehicle operation, and output the residual compensation amount of the rear wheel steering angle and the residual compensation amount of the yaw moment. The residual compensation amount of the rear wheel steering angle is superimposed on the rear wheel steering angle command, and the residual compensation amount of the yaw moment is superimposed on the yaw moment command to obtain a composite command. The composite command is then subjected to rate of change limiting and absolute amplitude limiting in sequence to generate the final control command. The final control command is distributed to each wheel drive motor using a torque optimization allocation function.

[0051] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent substitutions, and improvements made by those skilled in the art to the above embodiments without departing from the scope of the technical solution of the present invention, based on the technical essence of the present invention, shall still fall within the protection scope of the technical solution of the present invention.

Claims

1. A chassis reinforcement learning-based collaborative control method driven by contradictions, characterized in that, Includes the following steps: S1: Obtain real-time vehicle status parameters and calculate the ideal yaw rate; Based on the phase plane of the centroid sideslip angle and the centroid sideslip angular velocity, a stable boundary function is defined, and the contradiction intensity index is calculated according to the degree of deviation of the phase plane trajectory from the stable boundary. S2: Obtain the rear wheel steering angle command output by the underlying active rear wheel steering system controller and the yaw moment command output by the underlying direct yaw moment system controller; combine the vehicle's real-time state parameters, ideal yaw rate, contradiction strength index, rear wheel steering angle command, and yaw moment command into a full-scale state input feature vector; the active rear wheel steering system adjusts the vehicle's attitude by changing the rear wheel steering angle, and the direct yaw moment system generates additional yaw moment by distributing the longitudinal driving force of the wheels; S3: Construct a reinforcement learning network model including an Actor network and a Critic network. The Actor network includes a shared feature encoder, a master policy head, and a dual explorer head. During model training, the gradient vectors of the master policy head objective function and the dual explorer head objective function with respect to the shared feature encoder are calculated respectively. The inner product of the two gradient vectors is used to determine whether there is a gradient conflict. If there is a conflict, the conflict avoidance gradient descent algorithm is used to calculate the convex combination of the two gradients as the fused gradient. The shared feature encoder is updated by the fused gradient, while the master policy head and the dual explorer head are updated by their respective independent gradients. After training converges, the dual explorer head is removed, and the shared feature encoder and the master policy head are retained. S4: In actual vehicle operation, the full state input feature vector is input into the reinforcement learning network model, and the residual compensation of the rear wheel steering angle and the residual compensation of the yaw moment are output. The residual compensation of the rear wheel steering angle is superimposed on the rear wheel steering angle command, and the residual compensation of the yaw moment is superimposed on the yaw moment command to obtain the composite command. The composite command is then subjected to rate of change limit and absolute amplitude limit in sequence to generate the final control command. The torque optimization allocation function is used to allocate the final control command to each wheel drive motor.

2. The chassis reinforcement learning collaborative control method based on contradiction-driven approach according to claim 1, characterized in that: In step S1, the process of acquiring real-time vehicle state parameters and calculating the ideal yaw rate is as follows: The vehicle's real-time status parameters include longitudinal speed, actual yaw rate, yaw rate deviation, center of gravity sideslip angle, center of gravity sideslip rate, lateral acceleration, road adhesion coefficient, and vehicle front wheel steering angle; Establish a two-degree-of-freedom vehicle dynamics reference model and calculate the ideal yaw rate: ； in, Ideal yaw rate; The road surface adhesion coefficient, It is the acceleration due to gravity; Longitudinal velocity; For vehicle stability factors; This refers to the wheelbase; This refers to the steering angle of the vehicle's front wheels.

3. The chassis reinforcement learning collaborative control method based on contradiction-driven approach according to claim 1, characterized in that: In step S1, the stability boundary function is defined based on the phase plane of the center of mass sideslip angle and the center of mass sideslip angular velocity. The contradiction intensity index is calculated according to the degree to which the phase plane trajectory deviates from the stability boundary. The specific process is as follows: Define stable boundary functions The expression is: ； Contradiction Intensity Index The calculation formula is as follows: ； in, This is the actual sideslip angle of the vehicle's center of gravity. This refers to the vehicle's actual sideslip angular velocity at its center of gravity. The slope parameter of the phase plane boundary; The preset instability and contradiction triggers a safety threshold; This represents the maximum phase plane divergence value under the vehicle's physical limits.

4. The chassis reinforcement learning collaborative control method based on contradiction-driven approach according to claim 1, characterized in that: In step S3, the reinforcement learning network model is trained with the objective of maximizing the cumulative reward function. Rewards from yaw stability Lateral stability reward Dynamic intervention smoothness reward Actuator control energy consumption penalty And instability and boundary crossing termination of punishment The weighted composition is as follows: ； in, These are the dimensionless weighting coefficients for each reward sub-item.

5. The chassis reinforcement learning collaborative control method based on contradiction-driven approach according to claim 1, characterized in that: In step S3, the step of calculating the gradient vectors of the main policy head objective function and the dual exploration head objective function with respect to the shared feature encoder is specifically as follows: The gradient vector of the main policy head objective function with respect to the shared feature encoder The calculation formula is: ； Main strategy head objective function for: ； The gradient vector of the dual exploration head objective function with respect to the shared feature encoder The calculation formula is: ； Dual exploration head objective function for: ； in, For network weight parameters of the shared feature encoder; The network weight parameters are the main strategy head; For mathematical expectation operators; For the first The full state input feature vector of each control cycle; To reinforce the experience replay pool for learning; For the first The action vector output by the agent in each control cycle; The probability distribution of actions output by the main strategy head; The Q-score is the result of evaluations by a network of action value critics. For the weight parameters of the critic network; Entropy regularization weights; For the network weight parameters of the dual explorer head; The action probability distribution of the dual exploration head output; The boundary exploration penalty is amplified by a constant weight. This is a reward for lateral stability.

6. The chassis reinforcement learning collaborative control method based on contradiction-driven approach according to claim 1, characterized in that: In step S4, the composite command is sequentially subjected to rate-of-change limiting and absolute amplitude limiting to generate the final control command. The specific process is as follows: The rate of change limit is calculated by taking the difference between the composite instruction and the final control instruction of the previous cycle, and then constraining this difference within the maximum action rate allowed by the hardware to obtain the smoothed instruction. ； ； Absolute amplitude limiting forces the smoothed command to be confined within the physical limit range of the actuator, resulting in the final control command: ； ； in, This is the standard truncation and amplitude limiting function; For the first The instruction increment after the rate of change is limited for each control cycle; For the first Smoothing instructions after rate of change limiting for each control cycle; For the first The final control command for each control cycle; This represents the physical maximum operating rate boundary of the actuator. and The physical maximum value of the rear wheel steering angle that the vehicle can provide; The physical extreme value of the yaw moment that the vehicle can provide; For the first A composite instruction for each control cycle; To control the sampling period; For the first The final control command output by the active rear wheel steering system controller in each control cycle; For the first The smooth command output by the active rear wheel steering system controller during the control cycle; For the first The final control command output by the bottom-level direct yaw moment system controller in each control cycle; For the first Each control cycle directly outputs a smooth command from the yaw moment system controller.

7. The chassis reinforcement learning collaborative control method based on contradiction-driven approach according to claim 1, characterized in that: In step S4, the final control command is distributed to each wheel drive motor using a torque optimization allocation function. Specifically, the torque optimization allocation function is as follows: in, This represents the peak torque of the motor. The road surface adhesion coefficient; Vertical load for a single tire; The radius of the tire; The total torque required by the driver; and These are the track widths of the front and rear wheels, respectively. For the desired driving torque of a single wheel, These represent the front left wheel, front right wheel, rear left wheel, and rear right wheel, respectively.

8. A chassis reinforcement learning collaborative control system based on contradiction-driven mechanisms, characterized in that, include: The vehicle state perception and conflict index calculation module is used to obtain real-time vehicle state parameters and calculate the ideal yaw rate. Based on the phase plane of the centroid sideslip angle and the centroid sideslip angular velocity, a stable boundary function is defined, and the contradiction intensity index is calculated according to the degree of deviation of the phase plane trajectory from the stable boundary. The underlying command acquisition and state stitching module is used to acquire the rear wheel steering angle command output by the underlying active rear wheel steering system controller and the yaw moment command output by the underlying direct yaw moment system controller; it combines the vehicle's real-time state parameters, ideal yaw rate, contradiction intensity index, rear wheel steering angle command, and yaw moment command into a full-quantity state input feature vector; the active rear wheel steering system adjusts the vehicle attitude by changing the rear wheel steering angle, and the direct yaw moment system generates additional yaw moment by distributing the longitudinal driving force of the wheels; The dual-head reinforcement learning training module is used to construct a reinforcement learning network model including an Actor network and a Critic network. The Actor network includes a shared feature encoder, a master policy head, and a dual explorer head. During model training, the gradient vectors of the objective functions of the master policy head and the dual explorer head with respect to the shared feature encoder are calculated respectively. The inner product of the two gradient vectors is used to determine whether there is a gradient conflict. If there is a conflict, a conflict-avoiding gradient descent algorithm is used to calculate the convex combination of the two gradients as the fused gradient. The shared feature encoder is updated by the fused gradient, while the master policy head and the dual explorer head are updated by their respective independent gradients. After training converges, the dual explorer head is removed, and the shared feature encoder and the master policy head are retained. The residual compensation and safety limiting execution module is used to input the full state input feature vector into the reinforcement learning network model during actual vehicle operation, and output the residual compensation amount of the rear wheel steering angle and the residual compensation amount of the yaw moment. The residual compensation amount of the rear wheel steering angle is superimposed on the rear wheel steering angle command, and the residual compensation amount of the yaw moment is superimposed on the yaw moment command to obtain a composite command. The composite command is then subjected to rate of change limiting and absolute amplitude limiting in sequence to generate the final control command. The final control command is distributed to each wheel drive motor using a torque optimization allocation function.

Images