Vehicle intelligent chassis cooperative control method and system based on deep reinforcement learning

Abstract

The application discloses a vehicle intelligent chassis cooperative control method and system based on deep reinforcement learning, relates to the technical field of joint control of vehicle subsystems, and comprises the following steps: acquiring key information of a vehicle and a surrounding environment and generating a characteristic state vector; inputting the characteristic state vector into a preset deep reinforcement learning algorithm model to obtain a weight increment double-channel control action; combining benchmark execution data to calculate brake final execution instructions, suspension final execution instructions and steering final execution instructions; and updating network parameters through comprehensive rewards, experience tuples and batch data to obtain a control strategy for intelligent chassis subsystem cooperative control in multiple scenarios, so that dynamic optimization of chassis subsystem cooperative control is realized.

Description

Technical Field

[0001] This invention relates to the field of joint control technology for vehicle subsystems, specifically to a vehicle intelligent chassis cooperative control method and system based on deep reinforcement learning. Background Technology

[0002] The intelligent chassis is a crucial component for achieving precise vehicle motion control in intelligent connected vehicles. It coordinates the lateral, longitudinal, and vertical movements of the vehicle through subsystems such as steer-by-wire, brake-by-wire, and active suspension. Current intelligent chassis collaborative control systems often employ an architecture where an upper-level supervisor allocates weights and lower-level sub-controllers execute independently. Specifically, the upper level assigns control weights to the steering, braking, and suspension subsystems based on data such as vehicle speed, slip angle, yaw rate, and scenario type. The lower-level brake controller, suspension controller, and steering controller each output execution quantities according to their respective control rules to achieve coordinated control between the chassis systems.

[0003] However, in various scenarios such as high-speed cruising, emergency obstacle avoidance, low-adhesion road surfaces, and complex extreme conditions, there is strong physical coupling between vehicle steering, braking, and suspension. If a single weight is directly mapped to the execution quantity, it will simultaneously bear the task of expressing two types of information: "scenario judgment" and "action intensity," resulting in significant differences in the actual execution quantity corresponding to the same weight value in different scenarios. Furthermore, existing weight allocation relies on preset thresholds and analytical formulas, which are prone to failure in uncovered complex conditions. Independent execution by each sub-controller may also generate unexpected coupling oscillations or control conflicts. Therefore, existing technologies suffer from the technical problem of insufficient adaptability of chassis system collaborative control in multi-scenario scenarios due to the coupling between the degree of subsystem intervention and the specific physical action quantity of the actuators. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a vehicle intelligent chassis cooperative control method and system based on deep reinforcement learning.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] In a first aspect, this invention discloses a vehicle intelligent chassis cooperative control method based on deep reinforcement learning, comprising the following steps: acquiring key information about the vehicle and its surrounding environment, integrating the states, and generating a state space vector corresponding to the current control cycle; inputting the state space vector into a preset deep reinforcement learning algorithm model to generate control actions, obtaining a weighted incremental dual-channel control action corresponding to the current control cycle, the weighted incremental dual-channel control action including subsystem intervention weights and execution increments; acquiring the baseline execution data output by the brake controller, suspension controller, and steering controller respectively, and co-calculating them with the subsystem intervention weights and execution increments to obtain the final braking execution index. The system executes the final commands for braking, suspension, and steering. Based on the state space vector, the final commands for braking, suspension, and steering, a dynamic reward function is calculated to obtain a comprehensive reward. The state space vector, the weighted incremental dual-channel control action, the comprehensive reward, and the state space vector corresponding to the next control cycle are combined to form an experience tuple, which is then stored in an experience replay pool. Batch data is sampled from the experience replay pool, and the network parameters of the deep reinforcement learning algorithm model are updated based on this batch data to obtain a control strategy for collaborative control of intelligent chassis systems in multiple scenarios. The network parameters include policy network parameters, value network parameters, and target network parameters.

[0007] Secondly, this invention discloses a vehicle intelligent chassis cooperative control system based on deep reinforcement learning, comprising the following modules: a state acquisition module, used to acquire key information about the vehicle and its surrounding environment, and perform state integration to generate a state space vector corresponding to the current control cycle; an initial action generation module, used to input the state space vector into a preset deep reinforcement learning algorithm model to generate control actions, obtaining a weighted incremental dual-channel control action corresponding to the current control cycle; and a cooperative calculation module, used to acquire the baseline execution data output by the brake controller, suspension controller, and steering controller respectively, and perform cooperative calculation with the subsystem intervention weights and execution increments to obtain the final braking execution command. The system comprises: a suspension final execution command and a steering final execution command; a comprehensive reward analysis module, used to calculate a dynamic reward function based on the state space vector, braking final execution command, suspension final execution command, and steering final execution command to obtain a comprehensive reward; an experience tuple storage module, used to assemble the state space vector, weighted incremental dual-channel control action, comprehensive reward, and the state space vector corresponding to the next control cycle into an experience tuple, and store the experience tuple in an experience playback pool; and a collaborative strategy generation module, used to sample batch data from the experience playback pool and update the network parameters of the deep reinforcement learning algorithm model based on the batch data to obtain a control strategy for collaborative control of the intelligent chassis system in multiple scenarios.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0009] 1. This invention generates control actions by inputting feature state vectors into a preset deep reinforcement learning algorithm model, resulting in a weighted incremental dual-channel control action that includes subsystem intervention weights and execution increments. This separates the subsystem intervention level from the specific physical action of the actuator, thereby achieving adaptive output of chassis system collaborative control actions in multiple scenarios. This effectively solves the problem of insufficient adaptability of chassis system collaborative control in multiple scenarios caused by the coupling of subsystem intervention level and actuator specific physical action in the prior art.

[0010] 2. This invention acquires key information about the vehicle and its surrounding environment, integrates the states, and generates a feature state vector corresponding to the current control cycle. This allows the vehicle dynamic data, the current state data of the actuators, and the scene identifier to be used as inputs to the deep reinforcement learning algorithm model. This achieves a unified expression of the vehicle's motion state, the current state of the actuators, and the scene context, solving the problem of insufficient scene perception capability caused by the single input information of control decisions in the prior art.

[0011] 3. This invention obtains the baseline execution data output by the brake controller, suspension controller and steering controller respectively, and performs collaborative calculation with the subsystem intervention weight and execution increment. This allows for dynamic correction of braking, suspension and steering execution commands while retaining the basic control capabilities of the underlying controller. This achieves collaborative control between the baseline execution layer and the intelligent scheduling layer, solving the problem of control conflicts that easily arise when each sub-controller executes independently in the prior art.

[0012] 4. This invention calculates a comprehensive reward by performing dynamic reward function calculation based on feature state vectors, final braking execution commands, final suspension execution commands, and final steering execution commands. After storing the experience tuples in the experience replay pool, it samples batch data to update the network parameters of the deep reinforcement learning algorithm model. This enables the control strategy to be continuously optimized around path tracking accuracy, driving stability, and ride comfort, thereby realizing closed-loop training and stable convergence of the collaborative control strategy of the vehicle intelligent chassis system in multiple scenarios. Attached Figure Description

[0013] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0014] Figure 1 This is the overall flowchart of the present invention;

[0015] Figure 2 This is a flowchart of the weighted incremental dual-channel control action generation and final execution instruction calculation of the present invention;

[0016] Figure 3 This is a flowchart of the dynamic reward function calculation and network parameter update process of the present invention;

[0017] Figure 4 This is a schematic diagram of the SAC algorithm of the present invention;

[0018] Figure 5 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0019] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0020] This method includes the following steps: acquiring key information about the vehicle and its surrounding environment, integrating the states, and generating a state space vector corresponding to the current control cycle; inputting the state space vector into a preset deep reinforcement learning algorithm model to generate control actions, obtaining a weighted incremental dual-channel control action corresponding to the current control cycle, which includes subsystem intervention weights and execution increments; acquiring the baseline execution data output by the brake controller, suspension controller, and steering controller, and co-calculating them with the subsystem intervention weights and execution increments to obtain the final braking execution command, suspension final execution command, and steering final execution command; calculating a dynamic reward function based on the state space vector, the final braking execution command, the final suspension execution command, and the final steering execution command to obtain a comprehensive reward; forming an experience tuple from the state space vector, the weighted incremental dual-channel control action, the comprehensive reward, and the state space vector corresponding to the next control cycle, and storing the experience tuple in an experience replay pool; sampling batch data from the experience replay pool, and updating the network parameters of the deep reinforcement learning algorithm model based on this batch data to obtain a control strategy for collaborative control of intelligent chassis systems in multiple scenarios, where the network parameters include policy network parameters, value network parameters, and target network parameters.

[0021] In this embodiment, as Figure 1 As shown, Figure 1This is the overall flowchart of the present invention, which illustrates the overall data processing process of the intelligent chassis cooperative control method for vehicles. The system acquires key information about the vehicle and its surrounding environment, integrates the states to generate feature state vectors, inputs these feature state vectors into a preset deep reinforcement learning algorithm model, and generates weighted incremental dual-channel control actions. Then, combining the baseline execution data of the brake controller, suspension controller, and steering controller, the system collaboratively calculates the final braking command, the final suspension command, and the final steering command. Subsequently, a dynamic reward function is calculated, the experience tuples are stored in the experience replay pool, and the network parameters are updated by sampling batch data to obtain a control strategy for the cooperative control of the intelligent chassis system in multiple scenarios.

[0022] Key information about the vehicle and its surrounding environment is represented by a dataset characterizing the vehicle's current motion state, the current state of its actuators, and the driving scenario in which the vehicle is located. This dataset can be acquired in real time through sensors or simulation interfaces. For example, vehicle dynamics data can be obtained through CarSim vehicle dynamics simulation software, actuator current state data can be obtained through Simulink controller simulation software, and scenario identifiers can be obtained through a standardized test scenario library or scenario configuration results.

[0023] The state space vector represents a data vector formed by arranging the vehicle dynamics data, actuator current state data, and scene identifier within the current control cycle in a preset order. This state space vector serves as input data for the deep reinforcement learning algorithm model, enabling the model to simultaneously obtain the vehicle motion state, actuator state, and scene context.

[0024] The deep reinforcement learning algorithm model, namely the SAC deep reinforcement learning algorithm, consists of a policy network, a value network, and a target network. The policy network is used to generate the probability distribution of control actions based on the state space vector; the value network is used to estimate the value corresponding to the state space vector and the control action; and the target network is used to slowly synchronize network parameters during training, improving the stability of the training process.

[0025] The weighted incremental dual-channel control action represents a control action composed of subsystem intervention weights and execution increments. The subsystem intervention weights are used to express the degree of intervention of the steering, braking, and suspension subsystems, and also to express the trust ratio of each subsystem controller. The execution increments are used to add correction amounts to the controller output, thereby giving the specific physical action quantities corresponding to the steering wheel angle, braking torque, and suspension force.

[0026] The baseline execution data represents the basic control quantities output by the brake controller, suspension controller, and steering controller, respectively. The brake controller corresponds to the basic braking torque, the suspension controller corresponds to the basic suspension force, and the steering controller corresponds to the optimal front wheel steering angle.

[0027] The comprehensive reward represents the reward value calculated by the dynamic reward function based on the vehicle state and the final executed command within the current control cycle. It is used to evaluate the overall performance of the current control action in terms of path tracking accuracy, driving stability, ride comfort, and control smoothness.

[0028] An experience tuple represents a data record consisting of the state space vector corresponding to the current control cycle, the weighted incremental dual-channel control action, the comprehensive reward, and the state space vector corresponding to the next control cycle. Experience tuples are stored in the experience replay pool for random sampling during subsequent network parameter updates. The mathematical formula for an experience tuple is:

[0029] ;

[0030] In the formula, This represents the empirical tuple corresponding to the current control cycle. This represents the state space vector corresponding to the current control cycle. This indicates the weight increment dual-channel control action corresponding to the current control cycle. This indicates the overall reward corresponding to the current control period. This represents the state space vector corresponding to the next control cycle.

[0031] The state space vector determines the input basis for the deep reinforcement learning algorithm model to generate control actions. The weighted incremental dual-channel control actions affect the final execution commands of braking, suspension, and steering. The final execution commands and vehicle state jointly influence the comprehensive reward, which is then used as training feedback data to participate in subsequent network parameter updates. Therefore, the experience tuples form a closed loop of training data from state input, action output, reward feedback to the next state.

[0032] The experience replay pool represents a data cache area used to store multiple experience tuples. Deep reinforcement learning algorithm models randomly sample batches of data from the experience replay pool to reduce the correlation between data in consecutive control periods and improve the stability of network parameter updates.

[0033] By acquiring key information about the vehicle and its surrounding environment and integrating it to generate a state space vector, the deep reinforcement learning algorithm model can simultaneously receive vehicle motion state, actuator current state, and scene context data, thus providing a unified data input foundation for collaborative control of the vehicle's intelligent chassis in multiple scenarios. By generating weighted incremental dual-channel control actions, the subsystem intervention weights are used to express the trust ratio of each subsystem controller, while the execution increment is used to superimpose corrections on the controller output. This separates subsystem scheduling and physical action correction, reducing the coupling problem caused by a single control quantity simultaneously expressing scene judgment and execution intensity. By co-calculating the baseline execution data with the subsystem intervention weights and execution increments, the deep reinforcement learning algorithm model can be introduced to correct complex operating conditions while retaining the basic control capabilities of the brake controller, suspension controller, and steering controller. By storing experience tuples in an experience replay pool and updating network parameters based on batch data, the control strategy can be continuously optimized in multi-control cycle feedback data, improving the overall adaptability, training stability, and usability of the intelligent vehicle chassis's collaborative control in multiple scenarios.

[0034] Furthermore, generating the state space vector corresponding to the current control cycle specifically includes: acquiring the scene identifier, vehicle dynamic data, and current state data of each actuator in the current state, and integrating the data to obtain vehicle state information; generating a state space vector based on the vehicle state information, the state space vector including longitudinal velocity, lateral velocity, center of gravity sideslip angle, yaw rate, lateral acceleration, vertical acceleration, roll angle, pitch angle, lateral position deviation, heading angle deviation, steering wheel angle, braking torque, suspension force, and scene identifier.

[0035] In this embodiment, vehicle dynamic data refers to data used to characterize the vehicle's motion state and body attitude, including longitudinal velocity, lateral velocity, sideslip angle, yaw rate, lateral acceleration, vertical acceleration, roll angle, pitch angle, lateral position deviation, and yaw angle deviation. Vehicle dynamic data can be acquired through vehicle sensors or output through the simulation interface of the CarSim vehicle dynamics simulation software.

[0036] The actuator current state data represents the state data of the steering actuator, braking actuator, and suspension actuator within the current control cycle, including steering wheel angle, braking torque, and suspension force. The actuator current state data can be output from the controller model or obtained through the vehicle actuator feedback interface.

[0037] A scenario identifier represents identification data used to characterize the current driving condition of the vehicle. The scenario identifier ensures that the state space vector includes not only the vehicle's physical state but also the operating condition information corresponding to the current control cycle.

[0038] Vehicle state information refers to the set of state data obtained by integrating scene identifiers, vehicle dynamic data, and actuator current state data. This data is used to further generate a state space vector, which is specifically:

[0039] ;

[0040] In the formula, Indicates the longitudinal speed of the vehicle, β represents the lateral velocity, ω represents the sideslip angle, and ω represents the yaw rate. Indicates lateral acceleration, φ represents vertical acceleration, θ represents roll angle, θ represents pitch angle, e represents lateral position deviation, and Δψ represents heading angle deviation. Indicates the steering wheel angle, Indicates braking torque, Indicates suspension force. Represents the scene identifier.

[0041] Among them, longitudinal velocity lateral velocity Reflecting the vehicle's speed and sideslip tendency; the sideslip angle β and yaw rate ω are core indicators of handling stability; lateral acceleration... Vertical acceleration These respectively reflect lateral stability and ride comfort; roll angle φ and pitch angle θ reflect vehicle attitude; lateral position deviation e and yaw angle deviation Δψ reflect path tracking accuracy; steering wheel angle... Braking torque Suspension force The current state of each actuator; scene identifier. Providing scene context enables the strategy to have scene awareness. Longitudinal velocity, lateral velocity, center of gravity sideslip angle, yaw rate, lateral acceleration, vertical acceleration, roll angle, and pitch angle collectively reflect the vehicle's motion state and body attitude; lateral position deviation and heading angle deviation collectively reflect path tracking deviation; steering wheel angle, braking torque, and suspension force collectively reflect the actuator's current state; scene identifiers reflect the scene context. These parameters collectively influence the weighted incremental dual-channel control actions output by the strategy network.

[0042] Action space It adopts a dual-channel structure of "weight + increment", with a total of 6 dimensions:

[0043] ;

[0044] ;

[0045] In the formula, Indicates the intervention weight of the steering subsystem. Indicates the intervention weight of the braking subsystem. Indicates the intervention weight of the suspension subsystem. For the increment of steering wheel angle, For the increase in braking torque, This is the increase in suspension force.

[0046] Taking the braking system as an example, In the underlying controller, this is reflected in the SAC's trust weight over the traditional controller's output:

[0047] ;

[0048] in The underlying load distribution controller calculates the basic braking torque using MPC optimization. This represents the final execution command of the braking system, i.e., the final braking torque output. It is derived from the base braking torque output by the underlying braking controller. Weighting via braking subsystem After weighting, the increase in braking torque output by the SAC is added. get. The closer it is to 1, the more trust is placed in the MPC controller, and the SAC only makes small corrections through Δ; The closer the value is to 0, the more the SAC determines that the current scenario exceeds the applicable range of the MPC's preset model and actively takes over control, primarily by directly issuing braking torque commands via Δ. This soft-switching mechanism replaces the hard mode switching in traditional methods, avoiding control oscillations. SAC stands for Soft Actor-Critic, representing the SAC deep reinforcement learning algorithm model, and MPC stands for Model Predictive Control.

[0049] By integrating vehicle dynamics data, actuator current state data, and scene identifiers into a feature state vector, the deep reinforcement learning algorithm model can simultaneously obtain vehicle motion state, vehicle attitude, path tracking deviation, actuator current state, and scene context information from the same data input. Longitudinal and lateral velocities reflect vehicle speed and sideslip tendency; center of gravity sideslip angle and yaw rate reflect handling stability; lateral and vertical acceleration reflect lateral stability and ride comfort, respectively; roll and pitch angles reflect vehicle attitude; lateral position deviation and heading angle deviation reflect path tracking accuracy; steering wheel angle, braking torque, and suspension force reflect the current state of chassis actuators; and scene identifiers provide scene context. This unified integration of multi-dimensional data avoids the deep reinforcement learning algorithm model making control decisions based solely on a single vehicle state, thereby improving the scene adaptability and data completeness of subsequent weighted incremental dual-channel control actions.

[0050] Furthermore, the vehicle dynamic data and the current state data corresponding to each actuator are obtained as follows: vehicle dynamic data is obtained based on a preset vehicle dynamics model, and the current state data of the actuators is obtained based on a preset controller model; data interaction between the vehicle dynamics model, the controller model, and the deep reinforcement learning algorithm model is achieved through real-time communication; clock synchronization is performed using a first preset step size, and an incrementing sequence number and a sending timestamp are added to each frame of interactive data; the interactive data is verified based on the incrementing sequence number and the sending timestamp, and if packet loss or timeout data is detected, timeout retransmission is triggered; otherwise, timeout retransmission is not triggered; the verified vehicle dynamic data and actuator current state data are used as the input data corresponding to the current control cycle.

[0051] In this embodiment, acquiring vehicle dynamic data specifically includes: using template calling and parameter calibration methods, calling a standardized vehicle template based on preset vehicle model parameters; performing fine calibration on the vehicle mass, wheelbase, tire parameters, suspension stiffness, and shock absorber damping coefficient to obtain a high-fidelity vehicle dynamics model; matching the high-fidelity vehicle dynamics model with a standardized test scenario library to obtain the current simulation scenario; and performing simulation calculations on the vehicle's motion state under the current simulation scenario to obtain vehicle dynamic data; the standardized test scenario library includes low-speed cruise scenario, high-speed cruise scenario, high-speed double lane change obstacle avoidance scenario, emergency braking scenario, low-adhesion road surface scenario, and composite scenario.

[0052] The vehicle dynamics model is specifically the CarSim vehicle dynamics model, and the controller model is specifically the Simulink controller model, which is used to output the current state data of the actuator and to interact with the vehicle dynamics model and the deep reinforcement learning algorithm model.

[0053] The real-time communication method specifically refers to the way in which the vehicle dynamics model, controller model, and deep reinforcement learning algorithm model interact with each other in real time according to the control cycle. Specifically, it is achieved through the UDP communication protocol to realize real-time data interaction between the CarSim vehicle dynamics model, the Simulink controller model, and the Python SAC algorithm; CarSim and Simulink conduct bidirectional data interaction through the S-Function interface, and Simulink and Python transmit state, action, and reward data in real time through UDP Socket.

[0054] This invention acquires vehicle dynamic data through a pre-set vehicle dynamics model and actuator current state data through a pre-set controller model, ensuring that the data source of the state space vector corresponds to the model output in the co-simulation platform. Real-time communication enables data interaction between the vehicle dynamics model, controller model, and deep reinforcement learning algorithm model, allowing vehicle state, control actions, and reward data to be transmitted within a unified control cycle. Using a first pre-set step size for clock synchronization reduces the risk of data misalignment caused by clock inconsistencies during multi-software co-simulation. By appending an incrementing sequence number and a sending timestamp to each frame of interactive data and triggering a timeout retransmission request when the receiver detects discontinuous sequence numbers, the integrity and timing accuracy of data transmission are improved, providing a stable data input foundation for subsequent state space vector generation, weighted incremental dual-channel control action generation, comprehensive reward calculation, and network parameter updates.

[0055] Furthermore, the weighted incremental dual-channel control action corresponding to the current control cycle is obtained, specifically including: inputting the state space vector into the policy network and outputting the probability distribution of the control action; generating a six-dimensional control action based on the probability distribution of the control action; dividing the six-dimensional control action into subsystem intervention weights and execution increments to obtain weighted incremental dual-channel control actions; the subsystem intervention weights include steering subsystem intervention weights, braking subsystem intervention weights, and suspension subsystem intervention weights, which are used to express the intervention degree of the steering, braking, and suspension subsystems, respectively; the execution increments include steering wheel angle increments, braking torque increments, and suspension force increments, which are used to give the specific physical action quantities corresponding to steering wheel angle, braking torque, and suspension force, respectively; the subsystem intervention weights are used to express the trust ratio of each subsystem controller, and the execution increments are used to superimpose correction amounts on the controller output.

[0056] In this embodiment, the policy network is specifically an Actor network, also known as a policy network. The policy network outputs a probability distribution of control actions. These control actions follow a normal distribution, with the mean representing the central tendency of the optimal control policy and the variance indicating the degree of randomness in the control exploration. The probability distribution of control actions represents the action distribution output by the policy network based on the state space vector corresponding to the current control cycle. By outputting this probability distribution, the policy network generates the control actions in the action space. A six-dimensional control action represents a data set in the action space that simultaneously includes the intervention weights of three subsystems and three execution increments.

[0057] The weighted incremental dual-channel control action represents a "weight + increment" dual-channel structure. It includes two channels: subsystem intervention weight and execution increment. The subsystem intervention weight expresses the degree of intervention of each subsystem, while the execution increment provides the specific physical action quantity of each actuator. The subsystem intervention weight includes the intervention weight of the steering subsystem, the intervention weight of the braking subsystem, and the intervention weight of the suspension subsystem. All three values ​​range from zero to one, used to express the intervention degree of the steering, braking, and suspension subsystems, and to control the trust ratio of each subsystem controller. The execution increment includes steering wheel angle increment, braking torque increment, and suspension force increment. These three are used to provide the specific physical action quantities corresponding to the steering wheel angle, braking torque, and suspension force, respectively, and are used to add correction amounts to the controller output.

[0058] By generating six-dimensional weighted incremental dual-channel control actions based on feature state vectors through a policy network, the intervention levels of the three subsystems—steering, braking, and suspension—and the specific corrections to steering wheel angle, braking torque, and suspension force can be output simultaneously. By dividing the control actions into subsystem intervention weights and execution increments, the subsystem intervention weights specifically express the intervention level of each subsystem and the degree of trust in the underlying controller, while the execution increments specifically express the specific physical action quantities. This avoids a single weight simultaneously undertaking the tasks of scene judgment and action intensity encoding.

[0059] Furthermore, the final execution commands for braking, suspension, and steering are obtained, specifically including: acquiring the base braking torque output by the brake controller, and calculating the braking subsystem intervention weight, braking torque increment, and base braking torque to obtain the final braking execution command; acquiring the base suspension force output by the suspension controller, and calculating the suspension subsystem intervention weight, suspension force increment, and base suspension force to obtain the final suspension execution command; acquiring the optimal front wheel steering angle output by the steering controller, and calculating the steering subsystem intervention weight, steering wheel angle increment, and optimal front wheel steering angle to obtain the final steering execution command; the subsystem intervention weight is used to control the trust ratio of each subsystem controller, and the execution increment is used to add a correction amount based on the controller output.

[0060] In this embodiment, as Figure 2 As shown, Figure 2 This is a flowchart illustrating the generation and final execution command calculation process of the weighted incremental dual-channel control action in this invention. The flowchart illustrates the process of generating the weighted incremental dual-channel control action and calculating the final execution command. After the feature state vector is input to the policy network, the policy network outputs the probability distribution of the control action and generates a six-dimensional control action. The six-dimensional control action is divided into subsystem intervention weights and execution increments, thus obtaining the weighted incremental dual-channel control action. The system acquires the basic braking torque, basic suspension force, and optimal front wheel angle, and, in conjunction with the corresponding subsystem intervention weights and execution increments, calculates the final braking command, the final suspension command, and the final steering command, forming the intelligent chassis system collaborative control output.

[0061] The base braking torque represents the reference braking control quantity output by the brake controller. The base braking torque can be the braking torque calculated by the underlying load distribution controller according to the model predictive control optimization, or it can be expressed as the optimal base braking torque in the MPC brake controller section.

[0062] The base suspension force represents the reference suspension control quantity output by the suspension controller. The suspension controller can be a hybrid ceiling-floor controller, which outputs a base suspension force that balances comfort and ground contact.

[0063] The optimal front wheel angle represents the reference steering control quantity output by the steering controller. The steering controller can be a linear quadratic regulator, which outputs the optimal front wheel angle based on a linear model.

[0064] The final braking execution command represents the braking control output obtained by co-calculating the basic braking torque, the intervention weight of the braking subsystem, and the braking torque increment.

[0065] The final suspension command represents the suspension control output obtained by co-calculating the basic suspension force, the intervention weight of the suspension subsystem, and the suspension force increment.

[0066] The final steering command represents the steering control output obtained by co-calculating the optimal front wheel angle, the intervention weight of the steering subsystem, and the steering wheel angle increment.

[0067] The basic braking torque is obtained as follows: The brake controller is constructed using the model predictive control method, with the vehicle longitudinal dynamics model as the prediction model, the longitudinal slip ratio and longitudinal velocity as the state variables, and the four-wheel braking torque as the control variables. The constraint optimization problem is solved in a rolling manner in each control cycle to obtain the four-wheel braking torque distribution that satisfies the tire adhesion limit constraint, the braking torque change rate constraint, and the total braking torque demand constraint, and this distribution is used as the basic braking torque.

[0068] ;

[0069] Subject to:

[0070] Tire adhesion limit constraints:

[0071] ;

[0072] Braking torque change rate constraint:

[0073] ;

[0074] Total braking torque:

[0075] ;

[0076] In the formula, i represents the wheel index, and i = 1, 2, 3, 4 correspond to the four wheels: front left, front right, rear left, and rear right, respectively. Indicates the first Braking torque of each wheel These correspond to the braking torques of the four wheels: left front, right front, left rear, and right rear. The MPC calculates the optimal distribution of the four-wheel braking torque in each control cycle, which is the base braking torque. Indicates the first The maximum permissible braking torque of each wheel under current operating conditions. The value represents the prediction time domain length, the number of control steps predicted forward by MPC (Multi-Step Control Calculation), used for rolling optimization of the control variable over multiple future steps, and k represents the time step index within the prediction time domain. Corresponding to each discrete moment from the current moment to the predicted time domain endpoint, This represents the predicted longitudinal velocity of the vehicle at step k. The objective function represents the longitudinal speed reference value, i.e., the target vehicle speed. The first term minimizes the deviation between the vehicle speed and the reference value to ensure speed tracking capability during braking. Indicates the first The first step of prediction The slip ratio of each wheel Indicates the optimal slip ratio. This represents the current road surface adhesion coefficient. This represents the vertical load on the i-th wheel. Indicates the number of control cycles within a single control period. The change in braking torque of each wheel Indicates the maximum permissible rate of change of braking torque. This indicates the total braking torque demand issued by the upper-level controller.

[0077] The optimal slip ratio is given by the following values: The slip ratio range is the range within which the tire can provide the maximum coefficient of adhesion. By minimizing the deviation between the actual slip ratio and the optimal slip ratio, the tire is ensured to always work in the high-adhesion range, improving braking efficiency and stability. During braking, load transfer causes changes in the vertical load of the front and rear wheels, which in turn changes the maximum braking torque limit of each wheel. The maximum allowable rate of change is used to limit the sudden change in braking torque, avoiding actuator response failure and vehicle impact, while improving braking smoothness. The total braking torque requirement issued by the upper controller is used to ensure that the sum of the braking torques of the four wheels meets the braking deceleration requirements of the entire vehicle.

[0078] MPC performs rolling optimization in each control cycle, outputting a four-wheel braking torque distribution that satisfies all safety constraints.

[0079] In the underlying controller, this is reflected in the SAC's trust weight over the traditional controller's output, which is the final braking execution command formula, specifically:

[0080] ;

[0081] In the formula, This indicates the final execution command for braking. Indicates the intervention weight of the braking subsystem. This represents the basic braking torque calculated by the underlying load distribution controller based on model predictive control optimization. This indicates the increment of braking torque.

[0082] The closer the braking subsystem intervention weight is to one, the more trust is placed in the model predictive controller. The SAC deep reinforcement learning algorithm model only makes small corrections through the braking torque increment. The closer the braking subsystem intervention weight is to zero, the more the SAC deep reinforcement learning algorithm model judges that the current scene exceeds the applicable range of the model predictive controller's preset model. It mainly issues braking torque commands directly through the braking torque increment, which is used to superimpose corrections on the basis of the basic braking torque.

[0083] When the brake controller specifically adopts a model predictive controller, the base braking torque output by the underlying brake controller is specifically the base braking torque output by the model predictive controller. In this case, the above general expression can be specified as:

[0084] ;

[0085] In the formula, This represents the base braking torque output by the model predictive controller.

[0086] SAC and Skyhook Controller Synergy: The skyhook controller outputs the base suspension force Fskyhook, which balances comfort and ground contact. The SAC... and Intelligent correction is performed, i.e., the final suspension command formula is:

[0087] ;

[0088] in, This indicates the final command executed by the suspension. Indicates the intervention weight of the suspension subsystem. This indicates the basic suspension force output by the ceiling controller. This indicates the increase in suspension force.

[0089] Under normal operating conditions, the closer the intervention weight of the suspension subsystem is to one, the more trust is placed in the decoupling strategy of the roof controller; when the intervention weight of the suspension subsystem decreases, the SAC deep reinforcement learning algorithm model provides stronger suspension correction force through suspension force increment.

[0090] The collaboration between SAC and LQR controllers: The LQR controller outputs the optimal front wheel steering angle δ_LQR based on a linear model, and SAC then... and Perform intelligent correction, i.e., switch to the final execution instruction formula:

[0091] ;

[0092] in, This indicates a jump to the final instruction to be executed. Indicates the intervention weight of the steering subsystem. This represents the optimal front wheel steering angle output by the linear quadratic regulator. This indicates the increment of the steering wheel angle.

[0093] Under normal operating conditions, the closer the intervention weight of the steering subsystem is to one, the more trust there is in the linear optimal solution of the linear quadratic regulator; when the intervention weight of the steering subsystem decreases, the SAC deep reinforcement learning algorithm model corrects the steering by adding the steering wheel angle increment.

[0094] By co-calculating the baseline execution data output by the brake controller, suspension controller, and steering controller with the subsystem intervention weights and execution increments, the basic control capabilities of the underlying controllers can be combined with the dynamic correction capabilities of the deep reinforcement learning algorithm model. The braking subsystem intervention weights, working in conjunction with the braking torque increments, allow the system to retain more of the model's predictive controller output under normal operating conditions, while improving braking intervention capabilities under extreme conditions by reducing the trust ratio and increasing the braking torque increment. The suspension subsystem intervention weights, working in conjunction with the suspension force increments, enable the system to maintain a basic balance between comfort and ground contact while providing stronger attitude control in scenarios with severe roll or pitch. The steering subsystem intervention weights, working in conjunction with the steering wheel angle increments, allow the system to utilize the path-following capability of the linear quadratic regulator under normal operating conditions, and compensate for model deviations through steering wheel angle increments under nonlinear conditions such as emergency obstacle avoidance. This step combines the "baseline execution layer" and the "intelligent scheduling layer" in the overall solution, improving the adaptability and reliability of the vehicle's intelligent chassis system's collaborative control.

[0095] Furthermore, a comprehensive reward is obtained, specifically including: calculating path tracking accuracy based on lateral position deviation and heading angle deviation in the state space vector; calculating driving stability based on centroid sideslip angle and yaw rate in the state space vector; calculating ride comfort based on vertical acceleration and pitch angle in the state space vector; obtaining real-time risk indicators and dynamically adjusting the weights based on real-time risk indicators, vehicle speed, and scene identifiers to obtain dynamic weights corresponding to path tracking accuracy, driving stability, and ride comfort, respectively. Real-time risk indicators include centroid sideslip ratio, yaw rate ratio, and reciprocal of collision time; and comprehensively calculating the comprehensive reward based on path tracking accuracy, driving stability, ride comfort, dynamic weights, and actuator command differences between adjacent control cycles.

[0096] In this embodiment, as Figure 3 As shown, Figure 3 This is a flowchart illustrating the dynamic reward function calculation and network parameter update process of the present invention. The system calculates path tracking accuracy, driving stability, and ride comfort respectively, and dynamically adjusts the weights based on vehicle speed, scene identifier, and real-time risk indicators to obtain a comprehensive reward. Subsequently, the experience tuples are stored in the experience replay pool, and batch data is randomly sampled from the experience replay pool to update the value network parameters, policy network parameters, and target network parameters in sequence. The system determines whether the round termination condition is triggered. If it is triggered, the current training round is stopped and the next round of training is executed. If it is not triggered, training continues based on the feature state vector corresponding to the next control cycle.

[0097] Path tracking accuracy is a performance metric used to evaluate whether a vehicle follows a target path. It consists of lateral position deviation and heading angle deviation, and is used to penalize lateral and heading deviations.

[0098] Driving stability refers to the performance indicators used to evaluate the handling stability of a vehicle. Driving stability is composed of the sideslip angle and yaw rate, and is used to prevent vehicle instability.

[0099] Ride comfort is a performance indicator used to evaluate a vehicle's vertical vibration and body pitch. Ride comfort is composed of vertical acceleration and pitch angle, and is used to penalize vertical vibration acceleration and body pitch angle.

[0100] Real-time risk metrics represent data used for risk analysis and dynamic weight adjustments.

[0101] Dynamic weights represent the evaluation weights for path tracking accuracy, driving stability, and ride comfort, respectively. These dynamic weights are adjusted based on vehicle speed, scenario identifiers, and real-time risk indicators, allowing the overall reward to employ different evaluation emphases in different scenarios.

[0102] The actuator command difference between adjacent control cycles represents the difference between the increment vectors of the current control cycle and the previous control cycle. The first-order difference of the actuator commands is constrained by the control smoothing reward to suppress large jumps in actions between adjacent control cycles.

[0103] A dynamic reward function is constructed that forms a closed loop of "vehicle + environmental perception - parameter quantification - optimal decision-making". Core performance indicators such as ride comfort, driving stability and path tracking accuracy are quantified into specific parameters, and the weight allocation is automatically adjusted according to different scenario characteristics, so that SAC can autonomously balance multiple objectives during the learning process.

[0104] Basic Reward Construction: Establishing a foundation based on path tracking accuracy Driving stability Ride comfort The core of the three-dimensional evaluation basis is as follows:

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] In the formula, Indicates the basic reward. The weights represent the path tracking accuracy. This represents the weight corresponding to driving stability. This indicates the weight corresponding to ride comfort. Indicates path tracking accuracy. Indicates driving stability. Indicates riding comfort. Indicates lateral positional deviation. Indicates the deviation of the heading angle. This represents the heading deviation weighting coefficient. Indicates the centroid sideslip angle. Indicates yaw rate. Indicates vertical acceleration. Indicates pitch angle, This represents the pitch weighting coefficient.

[0110] The higher the index values ​​corresponding to path tracking accuracy, driving stability, and ride comfort, the lower the base reward; the higher the corresponding dynamic weight, the stronger the impact of the index on the base reward. A larger lateral position deviation corresponds to a higher path tracking accuracy index, indicating a more severe deviation from the vehicle's path; a larger heading angle deviation corresponds to a higher path tracking accuracy index; the larger the heading deviation weight coefficient, the stronger the impact of the heading angle deviation on the path tracking accuracy index. A larger center-of-gravity sideslip angle corresponds to a higher driving stability index, indicating a more pronounced degree of vehicle sideslip; a larger yaw rate corresponds to a higher driving stability index, indicating a more pronounced change in vehicle yaw and a higher risk of vehicle instability. A larger vertical acceleration corresponds to a higher ride comfort index, indicating more pronounced vertical vibration; a larger pitch angle corresponds to a higher ride comfort index; the larger the pitch weight coefficient, the stronger the impact of the pitch angle on the ride comfort index.

[0111] Controlling smooth rewards:

[0112] ;

[0113] ;

[0114] In the formula, This represents the control smoothing reward corresponding to the current control cycle. This represents the control smoothing weighting coefficient. This represents the increment vector at the current moment. This represents the increment vector from the previous time step. Indicates the increment of steering wheel angle. Indicates the increment of braking torque. This indicates the increase in suspension force.

[0115] The greater the difference between the current increment vector and the previous increment vector, the smaller the control smoothing reward; the larger the control smoothing weight coefficient, the stronger the penalty for the overall reward when actions jump between adjacent control cycles.

[0116] A comprehensive reward and punishment system is in place.

[0117] Dynamic instability penalty: if or The vehicle was deemed to be on the verge of going out of control, and a severe penalty was imposed. And terminate the round.

[0118] Road boundary deviation penalty: If a vehicle leaves the drivable area (crosses a solid lane line or leaves the road), a penalty will be imposed. And terminate the round.

[0119] Collision penalty: If a collision occurs with an obstacle, a penalty is imposed. And terminate the round.

[0120] Mission completion reward: A large one-time positive reward will be given if the target point is successfully reached or a specific scenario is safely completed (such as returning to the lane after obstacle avoidance). It provides sparse but clear positive guidance signals.

[0121] Total reward function:

[0122] ;

[0123] In the formula, This represents the total reward corresponding to the current control period. This represents the dynamic weight corresponding to the path tracking accuracy in the current control cycle. This represents the dynamic weight corresponding to the driving stability during the current control cycle. This represents the dynamic weight corresponding to ride comfort during the current control cycle. Indicates path tracking accuracy; Indicates driving stability. Indicates riding comfort. This represents the control smoothing weighting coefficient. This represents the increment vector for the current control cycle. This represents the increment vector from the previous control cycle.

[0124] If the termination condition (dynamic instability, path deviation, collision) is triggered, the total reward is replaced with... If the task is successfully completed, the total reward will be replaced with... .

[0125] The higher the path tracking accuracy, driving stability, and ride comfort indicators, the lower the total reward; the higher the corresponding dynamic weight, the stronger the impact of the indicator on the total reward; the greater the difference in actuator increment between adjacent control cycles, the lower the total reward.

[0126] Lateral position deviation, heading angle deviation, centroid sideslip angle, yaw rate, vertical acceleration, and pitch angle all originate from the state space vector.

[0127] Path tracking accuracy is calculated from lateral position deviation and heading angle deviation; driving stability is calculated from center of gravity sideslip angle and yaw rate; ride comfort is calculated from vertical acceleration and pitch angle.

[0128] The incremental vector in the control smoothing reward consists of the steering wheel angle increment, braking torque increment, and suspension force increment of the current control cycle, all of which come from the weighted incremental dual-channel control actions output by the policy network.

[0129] By constructing a comprehensive reward system encompassing path tracking accuracy, driving stability, and ride comfort, the deep reinforcement learning algorithm model can simultaneously consider path deviation, vehicle instability risk, and ride quality. Introducing dynamic weights allows the evaluation focus of these three performance objectives to adjust according to changes in vehicle speed, scene identifiers, and real-time risk indicators across different scenarios. By controlling the difference in actuator commands between adjacent control cycles through smooth reward constraints, abrupt changes in steering wheel angle increments, braking torque increments, and suspension force increments can be reduced, ensuring smooth steering wheel rotation, gentle braking, and smooth suspension operation. This provides effective reward feedback to the experience tuples, enabling the deep reinforcement learning algorithm model to autonomously balance the three core objectives of path tracking accuracy, driving stability, and ride comfort during training.

[0130] Furthermore, the real-time risk indicators are obtained as follows: the ratio of the sideslip angle in the state space vector to the preset maximum sideslip angle is calculated to obtain the sideslip angle ratio; the ratio of the yaw rate in the state space vector to the preset maximum yaw rate is calculated to obtain the yaw rate ratio; the reciprocal of the collision time between the vehicle and the nearest obstacle is processed to obtain the reciprocal of the collision time; the sideslip angle ratio, the yaw rate ratio, and the reciprocal of the collision time are jointly marked as the real-time risk indicators.

[0131] In this embodiment, the centroid sideslip ratio represents the ratio between the centroid sideslip angle and the preset maximum centroid sideslip angle, and is used as one of the normalized real-time risk indicators.

[0132] The yaw rate ratio represents the ratio between the yaw rate and the preset maximum yaw rate.

[0133] The collision time countdown can be obtained by counting back the time of collision between the vehicle and the nearest obstacle.

[0134] Real-time risk indicators represent normalized risk input data composed of the sideslip ratio, yaw rate ratio, and reciprocal of the collision time. The lightweight fuzzy inference system takes the sideslip ratio, yaw rate ratio, and reciprocal of the collision time with the nearest obstacle as input and outputs a comprehensive risk coefficient. Real-time risk indicators:

[0135] ;

[0136] ;

[0137] ;

[0138] ;

[0139] In the formula, This indicates a real-time risk indicator. Indicates the ratio of the centroid's sideslip angle. Indicates the centroid sideslip angle. This indicates the preset maximum centroid sideslip angle. Indicates the yaw rate ratio. Indicates yaw rate. This indicates the preset maximum yaw rate. Indicates the countdown to the collision time. This indicates the time between the vehicle's collision with the nearest obstacle.

[0140] By constructing real-time risk indicators using the sideslip ratio, yaw rate ratio, and reciprocal of the collision time, dynamic weight adjustments can simultaneously consider both vehicle instability risk and collision risk with obstacles. The sideslip ratio reflects vehicle sideslip risk, the yaw rate ratio reflects vehicle yaw risk, and the reciprocal of the collision time reflects external collision risk related to the nearest obstacle. By jointly labeling these three data points as real-time risk indicators, a normalized input basis can be provided for subsequent comprehensive risk coefficient calculations. This allows the dynamic reward function to increase the evaluation weights corresponding to driving stability or path tracking accuracy in high-risk scenarios, thereby enhancing the overall responsiveness of the vehicle's intelligent chassis cooperative control strategy to extreme conditions.

[0141] Furthermore, the dynamic weights corresponding to path tracking accuracy, driving stability, and ride comfort are obtained, specifically including: comparing the vehicle speed with a preset safe speed threshold; when the vehicle speed exceeds the safe speed threshold, calculating a speed-based weight adjustment factor based on longitudinal speed, the safe speed threshold, and the gain coefficient; identifying the scenario type corresponding to the current control cycle based on the scenario identifier, and obtaining scenario-based weight switching data according to the scenario type; performing risk analysis based on the center of gravity sideslip ratio, yaw rate ratio, and the reciprocal of the collision time to obtain a comprehensive risk coefficient; and comprehensively calculating the speed weight adjustment factor, the scenario-based weight switching data, and the comprehensive risk coefficient to obtain the dynamic weights corresponding to path tracking accuracy, driving stability, and ride comfort, respectively.

[0142] In this embodiment, dynamic weights represent the evaluation weights corresponding to path tracking accuracy, driving stability, and ride comfort, respectively. These dynamic weights are not fixed but are adjusted based on vehicle speed, scene identifiers, and real-time risk indicators, allowing the deep reinforcement learning algorithm model to adopt different evaluation priorities in different scenarios.

[0143] The speed-based weighting adjustment factor represents the stability weight enhancement data generated based on the comparison between longitudinal speed and the safe speed threshold. Its function is to automatically and linearly increase the dynamic weight corresponding to driving stability when the vehicle speed exceeds the safe speed threshold.

[0144] When the longitudinal speed does not exceed the safe speed threshold, the stability weight is not additionally enhanced due to the speed factor; when the longitudinal speed exceeds the safe speed threshold, the stability weight increases linearly with the speed.

[0145] The weight switching data based on scene recognition represents the data used to adjust the weights of path tracking accuracy, driving stability, and ride comfort after identifying the scene type corresponding to the current control cycle based on the scene identifier.

[0146] The comprehensive risk coefficient represents the risk amplification factor obtained after risk analysis based on real-time risk indicators. The lightweight fuzzy inference system takes normalized real-time risk indicators as input and outputs the comprehensive risk coefficient, which ranges from one to the maximum risk coefficient.

[0147] Risk-based fuzzy logic adjustment refers to the process of taking the centroid sideslip ratio, yaw rate ratio, and reciprocal of collision time as inputs, outputting a comprehensive risk coefficient through a lightweight fuzzy inference system, and adjusting the dynamic weights accordingly.

[0148] Comprehensive calculation of dynamic weights:

[0149] ;

[0150] ;

[0151] In the formula, Indicates the current control cycle number Dynamic weights corresponding to each performance metric Indicates the first The basic weights corresponding to each performance metric This represents the weight adjustment factor obtained based on the state space vector. This represents the weighting adjustment factor based on vehicle speed. Indicates the first Each performance metric corresponds to a weight adjustment factor for scene recognition. Indicates the first Risk weight adjustment factor for each indicator.

[0152] Weight adjustment factor based on scene recognition The specific method for obtaining this information is as follows: based on the scenario identifier of the current control cycle, the weight adjustment coefficient corresponding to the performance index is directly obtained from the preset scenario and weight mapping rule table, which belongs to the discrete working condition switching factor.

[0153] The calculation formula is:

[0154] ;

[0155] ;

[0156] In the formula, i represents the performance index. Corresponding path tracking accuracy, Corresponding to driving stability, Corresponding to ride comfort, ScenarioID represents a scenario identifier, a discrete enumeration value corresponding to different driving scenarios such as high-speed cruising, emergency obstacle avoidance, low-traction road surfaces, low-speed cruising, and combined extreme conditions. The mapping function between scenarios and weights is essentially a pre-defined lookup table rule. The adjustment factors for the three types of indicators differ in different scenarios. Typical value examples are as follows:

[0157] For example, in emergency obstacle avoidance scenarios: the path tracking factor is significantly increased (e.g., 1.5~2.0), the stability factor is increased, and the comfort factor is significantly reduced (e.g., 0.1~0.3), allowing for a sacrifice in ride quality in exchange for a higher success rate in obstacle avoidance; in high-speed cruising scenarios: the stability factor is >1, and the tracking and comfort factors are ≈1, prioritizing high-speed driving stability; in normal smooth cruising scenarios: the comfort factor is >1, and the other two factors are ≈1, focusing on the ride experience.

[0158] Risk-based weighting factor The specific calculation steps are as follows: Input three types of normalized real-time risk indicators, and calculate the comprehensive risk coefficient through a lightweight fuzzy inference system; based on the comprehensive risk coefficient, differentiate the weights of the three performance indicators to obtain the risk adjustment factor corresponding to each indicator. The specific calculation formula is as follows:

[0159] Step 1: Calculate the overall risk coefficient:

[0160] ;

[0161] Where Risk∈[1,Rmax], and Rmax is the maximum risk amplification factor, which is usually taken as 3~5 in engineering.

[0162] In the formula, The sideslip ratio represents the center of gravity, a normalized index characterizing the vehicle's own sideslip instability risk. The yaw rate ratio is a normalized index that characterizes the risk of yaw instability in a vehicle. It represents the reciprocal of the collision time and is a normalized index characterizing the collision risk from the external environment. .

[0163] In the formula, The time to collision (TOT) represents the remaining time before the vehicle, in its current motion, collides with the nearest obstacle (obtained from environmental perception or simulation scene data). This represents a lightweight fuzzy inference system. Its pre-defined fuzzy logic module includes a fuzzification interface, a rule base, an inference engine, and a defuzzification interface. Following the pre-defined rule that "the higher the risk, the larger the output coefficient," it transforms the precise values ​​of the three inputs into a single, comprehensive risk quantification value. This achieves the fusion of multi-dimensional risks, integrating dispersed lateral deviation, yaw, and collision risks into a unified comprehensive risk coefficient. This represents the comprehensive risk coefficient, a quantitative value that integrates the vehicle's own instability risk and the external collision risk. It is the final representation of the risk level under the current operating conditions. Value range: Where 1 corresponds to a risk-free normal state. This corresponds to the highest risk level. It serves as an intermediate variable connecting the underlying risk indicators with the upper-level weight adjustments; the higher the risk, the stronger the amplification effect on the stability weights. This represents the maximum risk amplification factor, the upper limit of the overall risk coefficient, which is a preset constant. In engineering practice, a maximum amplification factor of 3 to 5 is typically used to limit the weights, preventing the weights from being infinitely amplified under high risk, which could lead to control imbalance.

[0164] Step 2: Calculate the risk adjustment factor for each indicator:

[0165]

[0166]

[0167] ;

[0168] It can also be abbreviated as:

[0169] ;

[0170] ;

[0171] In the formula, Indicates the first The risk weight adjustment factor for each indicator is the amplification / reduction coefficient of the risk level on the weight of the i-th performance indicator. The dynamic weight of the indicator is obtained by multiplying it by the basic weight.

[0172] A risk weight adjustment factor >1 indicates that the weights are amplified, while a risk weight adjustment factor <1 indicates that the weights are suppressed. Value range: Stability factor: Comfort factors: The unified comprehensive risk coefficient is transformed into differentiated weight adjustment ranges for three types of indicators, realizing the safety-first logic of "when the risk increases, the stability weight increases and the comfort weight decreases". This represents the risk gain coefficient of the path tracking weight. It is a calibration parameter that controls the increase in path tracking weight as risk increases, and is a preset constant. It is typically set to [0,1], and generally less than 1. It adjusts the priority of the path tracking index under high-risk conditions: ensuring path tracking requirements in obstacle avoidance scenarios without overriding the core priority of stability. This indicates the current control cycle time index. It also indicates that the adjustment factor is a time-varying parameter that updates in real-time with each control cycle. This reflects the real-time nature of the weight adjustment; the factor value is recalculated based on the current risk status each control cycle. Logical explanation: The higher the risk, the greater the amplification of the driving stability weight, the greater the reduction of the ride comfort weight, and the slight increase in the path tracking weight, achieving the control logic of "prioritizing safety under extreme conditions."

[0173] Speed-based stability enhancement: When the vehicle speed exceeds the safety threshold, the stability weight is automatically and linearly increased to ensure safe high-speed driving.

[0174] ;

[0175] In the formula This represents the speed-based weighting adjustment factor corresponding to driving stability. For safe vehicle speed threshold, This is the gain coefficient.

[0176] When the vehicle speed Exceed At that time, the dynamic weights corresponding to the driving stability in the current control cycle It increases linearly with vehicle speed.

[0177] When the longitudinal speed is less than or equal to the safe speed threshold, the non-negative difference is zero, and the speed-based stability enhancement factor is one. When the longitudinal speed is greater than the safe speed threshold, the greater the longitudinal speed, the greater the non-negative difference, and the greater the speed-based stability enhancement factor. The larger the gain coefficient, the greater the enhancement of the dynamic weight of driving stability after the longitudinal speed exceeds the safe speed threshold.

[0178] Scene-based weight switching: Instantly adjusts the weights of each target based on the ScenarioID. In emergency obstacle avoidance scenarios, the tracking accuracy weight is significantly improved, while the requirements for comfort indicators are proactively relaxed, allowing the SAC to sacrifice ride comfort in critical moments to safely complete obstacle avoidance. The ScenarioID is a discrete number label used to identify the current driving condition of the vehicle, belonging to the discrete input quantity of the condition category. Simply put, it assigns a numerical code to each typical driving scenario, allowing the algorithm to quickly identify "what condition the vehicle is currently driving under." Its function: It is one of the three main input bases for dynamic weight adjustment (the other two being vehicle speed and real-time risk indicators), used to generate "scene-based weight switching data": different ScenarioIDs correspond to a set of preset weight adjustment rules, which can instantaneously change the evaluation weights of the three targets: path tracking accuracy, driving stability, and ride comfort; ensuring that the control priority of the reinforcement learning strategy matches the core needs of the current scenario, avoiding the problem of a single weight being unable to adapt to multiple scenarios. It is a discrete enumeration quantity. In engineering, it is usually represented by an integer number corresponding to different operating conditions. For example: ScenarioID value / corresponding driving scenario / weight adjustment logic: 1 / Low-speed cruise scenario / Comfort weight is relatively high, stability and tracking weights are the baseline values; 2 / High-speed cruise scenario / Stability weight is significantly increased, tracking weight is the baseline, and comfort weight is slightly reduced; 3 / Emergency obstacle avoidance scenario / Path tracking accuracy weight is greatly increased, stability weight is increased, and comfort weight is significantly reduced; 4 / Emergency braking scenario / Stability weight is the highest, tracking weight is the second highest, and comfort weight is the lowest; 5 / Low-adhesion road surface scenario / Stability weight is significantly increased, slip rate is strictly limited, and comfort weight is reduced; 6 / Composite extreme operating condition / Stability weight is maximized, tracking weight is the second highest, and comfort weight is close to 0.

[0179] Risk-based fuzzy logic adjustment: Design a lightweight fuzzy inference system. Inputs include normalized real-time risk indicators—side slip ratio, yaw rate ratio, and the reciprocal of the collision time with the nearest obstacle. The system outputs a comprehensive risk coefficient. Stability weights under extreme conditions By amplifying the risk factor to 3-5 times the normal value, the comfort weight is increased. The corresponding value is reduced to below 0.1, enabling dynamic switching of evaluation criteria based on risk assessment results.

[0180] By comparing longitudinal velocity with a safe speed threshold and linearly increasing the dynamic weight corresponding to driving stability when the longitudinal velocity exceeds the safe speed threshold, the evaluation criteria for high-speed driving can focus more on vehicle stability, thereby reducing the risk of vehicle instability at high speeds. By switching weights based on scene identifiers, deep reinforcement learning algorithms can improve path tracking accuracy weights in emergency obstacle avoidance scenarios, while allowing for a slight reduction in ride comfort requirements when safety risks are high, thus aligning the evaluation criteria with the current driving conditions. Through risk analysis based on the center-of-gravity sideslip ratio, yaw rate ratio, and the reciprocal of the collision time, the dynamic weights can simultaneously consider both vehicle attitude risk and external collision risk.

[0181] Furthermore, a control strategy for collaborative control of intelligent chassis systems in multiple scenarios is obtained, specifically including: randomly sampling batch data from the experience replay pool, the batch data including multiple experience tuples; updating the value network parameters based on the batch data and outputting the value estimation result through the value network; updating the strategy network parameters based on the value estimation result and the probability distribution of control actions; performing soft updates on the target network parameters based on the strategy network parameters and the value network parameters to obtain the updated target network parameters; determining whether the current training round triggers the round termination condition based on the state space vector, the final braking command, the final suspension command, and the final steering command; if the round termination condition is triggered, stopping the current training round and re-acquiring the vehicle dynamic data and actuator current data corresponding to the next training round. The system uses state data and scene identifiers to execute the next round of training. If the round termination condition is not triggered, it continues to generate control actions, calculate the dynamic reward function, store experience tuples in the experience replay pool, and sample batch data from the experience replay pool to update the network parameters of the deep reinforcement learning algorithm model based on the batch data. This process is repeated until the deep reinforcement learning algorithm model converges, resulting in a control strategy for collaborative control of intelligent chassis systems in multiple scenarios. Round termination conditions include dynamic instability, road boundary deviation, collision, and task completion.

[0182] In this embodiment, as Figure 4 As shown, Figure 4 This is a schematic diagram of the SAC algorithm of the present invention, which illustrates the parameter update process of the policy network, value network and target network in the SAC algorithm.

[0183] Batch data refers to multiple empirical tuples randomly sampled from the empirical replay pool. Each control cycle randomly samples batch data from the empirical replay pool, with a batch size of up to 256, for use in updating network parameters.

[0184] The value network adopts a dual value network structure and takes the minimum value of the two value network outputs as the value estimate to suppress the overestimation of value.

[0185] The value estimation result represents the value estimation result of the value network based on the state and action outputs. During training, the minimum value of the two value network outputs is taken as the value estimate.

[0186] The policy network outputs the probability distribution of control actions and updates its parameters by maximizing the weighted sum of expected reward and policy entropy.

[0187] The target network refers to the network corresponding to the value network. The target network uses a soft update method to slowly synchronize parameters to ensure the stability of the training process.

[0188] Policy entropy is a measure of the uncertainty of the distribution of output actions of the policy network. The SAC maximum entropy objective function encourages the policy to continue exploring and avoids premature convergence to a local optimum control policy.

[0189] The round termination condition indicates the trigger condition for stopping the current training round, including dynamic instability, road boundary deviation, collision, and task completion.

[0190] Empirical tuples: ;

[0191] In the formula, This represents the state space vector corresponding to the current control cycle. This indicates the control action corresponding to the current control cycle. This indicates the overall reward corresponding to the current control period. This represents the state space vector corresponding to the next control cycle.

[0192] The state space vector serves as the input to the policy network for generating control actions; the control actions influence the final executed instructions; the comprehensive reward serves as training feedback; and the state space vector corresponding to the next control cycle is used to construct the next state. All of the above data together constitute the training samples required for updating the network parameters.

[0193] The optimization objective of the SAC strategy is to maximize the weighted sum of expected return and policy entropy.

[0194] ;

[0195] In the formula, Indicates the strategy optimization objective. Indicates the current state. Indicates the current action. This represents the state-action distribution corresponding to the strategy. This indicates the reward corresponding to the current state and the current action. Represents the entropy regularization coefficient. This represents the policy entropy of the policy in the current state.

[0196] The maximum entropy policy optimization objective function of the SAC algorithm is the core formula of the SAC algorithm in deep reinforcement learning. Unlike ordinary reinforcement learning, which only pursues "maximizing cumulative reward", SAC adds an additional policy entropy term in addition to the reward term. The core idea is to enable the policy to obtain the highest possible control reward while maintaining sufficient randomness (exploration) to avoid premature convergence to local optima and improve the policy's generalization ability in complex multi-condition chassis control scenarios. It should be noted that J(π) represents the policy optimization objective / the total value objective that the entire SAC algorithm aims to maximize. The larger the value, the better the policy π performs in terms of both "obtaining high control reward" and "maintaining exploration randomness". The mathematical expectation operator / represents the average value of the terms within parentheses under the state and action distribution generated by policy π. It can be understood as: the comprehensive value obtainable in each control cycle under an average meaning. Represents the state and action distribution of the strategy / When the vehicle executes strategy π, the state and actions The probability distribution of occurrence is determined by both the strategy and environmental dynamics. The state space vector at time t represents the state of the current control cycle, and includes all input information such as longitudinal velocity, centroid sideslip angle, yaw rate, path deviation, actuator state, and scene identifier. The action at time t represents the weight increment of the dual-channel control action output in the current control cycle, which includes the 3-dimensional subsystem intervention weight and the 3-dimensional execution increment, for a total of 6-dimensional continuous actions. Instant reward function / in state Next action The resulting instantaneous comprehensive reward corresponds to the dynamic reward function in the patent, which integrates path tracking accuracy, driving stability, ride comfort, and control smoothness. The larger the value, the more the strategy tends to be exploratory (with stronger randomness in actions). The smaller the α value, the more the strategy tends to pursue higher rewards (more certain actions). In the early stages of training, a larger α value encourages extensive exploration of the action space; in the later stages of training, α gradually decays, allowing the strategy to gradually converge to the optimal deterministic control behavior.

[0197] Entropy regularization coefficient automatically balances exploration and exploitation: in the early stages of training. Larger space, encouraging extensive exploration of the action space, later on As the behavior gradually diminishes, the strategy converges to the optimal deterministic behavior.

[0198] ① Policy Network (Actor): Probability distribution of output control actions Mean The central tendency and variance represent the optimal control strategy. The degree of randomness in the exploration is controlled. The network adopts a multi-layer fully connected structure, with hidden layer dimensions... The activation function is ReLU. The actor network updates its parameters by maximizing the weighted sum of the expected reward and the policy entropy.

[0199] ;

[0200] In the formula, The policy network loss function is represented as follows. This represents the minimum value estimate in the output of the dual-value network. Represents the entropy regularization coefficient. Indicates the state Down Output Action The probability of.

[0201] ② Value Network (Critic): Employing a dual Network Structure and the corresponding target network. During training, the minimum value of the two outputs is taken. As a value estimate, it effectively suppresses the bias of overestimating Q-values. The Critic network updates by minimizing the temporal difference error:

[0202] ;

[0203] In the formula, Represents the value network loss function. This represents the value estimation result corresponding to the current state and the current action. Indicates the current reward. Indicates the discount factor. This represents the value estimation results corresponding to the next state and the next action.

[0204] The target network uses a soft update method to slowly synchronize parameters:

[0205] ;

[0206] In the formula, Indicates the target network parameters. Indicates the current network parameters. This represents the soft update coefficient.

[0207] Among them, the soft update coefficient This ensures that the target network parameters change slowly, thus improving training stability.

[0208] By storing experience tuples in an experience replay pool and randomly sampling batch data from the pool to update network parameters, the correlation between data in continuous control cycles can be reduced, providing stable training samples for the deep reinforcement learning algorithm model. By outputting value estimation results through the value network and using the minimum output of the dual value network as the value estimation result, excessive bias in value estimation can be suppressed, improving the stability of value assessment. Updating policy network parameters based on value estimation results and the probability distribution of control actions guides the policy network to generate better, more exploratory, weighted incremental dual-channel control actions. Slowly synchronizing parameters through soft updates of the target network avoids training instability caused by drastic changes in target network parameters. By stopping the current training round and entering the next training round when the round termination condition is triggered, the training process forms clear feedback boundaries around dynamic instability, road boundary deviation, collision, and task completion. This step achieves closed-loop training from experience storage, batch sampling, value network update, policy network update to soft updates of the target network in the overall scheme, enabling the deep reinforcement learning algorithm model to obtain control strategies for the collaborative control of vehicle intelligent chassis systems through multi-scenario iterative learning.

[0209] like Figure 5 As shown, Figure 5 The system structure diagram of this invention, a vehicle intelligent chassis cooperative control system based on deep reinforcement learning, includes the following modules: a state acquisition module, used to acquire key information about the vehicle and its surrounding environment, and perform state integration to generate a state space vector corresponding to the current control cycle; an initial action generation module, used to input the state space vector into a preset deep reinforcement learning algorithm model to generate control actions, obtaining a weighted incremental dual-channel control action corresponding to the current control cycle; and a cooperative calculation module, used to acquire the baseline execution data output by the brake controller, suspension controller, and steering controller, and perform cooperative calculation with the subsystem intervention weights and execution increments to obtain the final braking execution command. The system comprises: a suspension final execution command and a steering final execution command; a comprehensive reward analysis module, used to calculate a dynamic reward function based on the state space vector, braking final execution command, suspension final execution command, and steering final execution command to obtain a comprehensive reward; an experience tuple storage module, used to assemble the state space vector, weighted incremental dual-channel control action, comprehensive reward, and the state space vector corresponding to the next control cycle into an experience tuple, and store the experience tuple in an experience playback pool; and a collaborative strategy generation module, used to sample batch data from the experience playback pool and update the network parameters of the deep reinforcement learning algorithm model based on the batch data to obtain a control strategy for collaborative control of the intelligent chassis system in multiple scenarios.

[0210] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A vehicle intelligent chassis cooperative control method based on deep reinforcement learning, characterized in that: Includes the following steps: Acquire key information about the vehicle and its surrounding environment, integrate the states, and generate the state space vector corresponding to the current control cycle; The state space vector is input into a preset deep reinforcement learning algorithm model to generate control actions, thereby obtaining the weight increment dual-channel control action corresponding to the current control cycle. The weight increment dual-channel control action includes subsystem intervention weights and execution increments. The baseline execution data output by the brake controller, suspension controller and steering controller are obtained respectively, and they are used in conjunction with the subsystem intervention weight and execution increment to obtain the final execution command of braking, suspension and steering. A dynamic reward function is calculated based on the state space vector, the final braking command, the final suspension command, and the final steering command to obtain the comprehensive reward. The state space vector, the weighted incremental dual-channel control action, the comprehensive reward, and the state space vector corresponding to the next control cycle are combined into an experience tuple, and the experience tuple is stored in the experience replay pool. Batch data is sampled from the experience replay pool, and the network parameters of the deep reinforcement learning algorithm model are updated based on the batch data to obtain a control strategy for collaborative control of intelligent chassis systems in multiple scenarios. The network parameters include policy network parameters, value network parameters, and target network parameters.

2. The vehicle intelligent chassis cooperative control method based on deep reinforcement learning according to claim 1, characterized in that: The generation of the state space vector corresponding to the current control cycle specifically includes: The system acquires the scene identifier, vehicle dynamic data, and current state data of each actuator in the current state, and integrates the data to obtain vehicle state information. A state space vector is generated based on vehicle state information. The state space vector includes longitudinal velocity, lateral velocity, center of gravity sideslip angle, yaw rate, lateral acceleration, vertical acceleration, roll angle, pitch angle, lateral position deviation, heading angle deviation, steering wheel angle, braking torque, suspension force, and scene identifier.

3. The vehicle intelligent chassis cooperative control method based on deep reinforcement learning according to claim 2, characterized in that: The specific method for obtaining the vehicle dynamic data and the current status data corresponding to each actuator is as follows: Vehicle dynamic data is obtained based on a preset vehicle dynamics model, and actuator current state data is obtained based on a preset controller model. Real-time communication enables data interaction between the vehicle dynamics model, the controller model, and the deep reinforcement learning algorithm model. Clock synchronization is performed using a first preset step size, and an incrementing sequence number and a sending timestamp are appended to each frame of interactive data; The interaction data is validated based on the incrementing sequence number and the sending timestamp. If lost or timed-out data is detected, a timeout retransmission is triggered; otherwise, a timeout retransmission is not triggered. The vehicle dynamic data and actuator current status data after verification are used as the input data for the current control cycle.

4. The vehicle intelligent chassis cooperative control method based on deep reinforcement learning according to claim 1, characterized in that: The specific steps for obtaining the weight increment dual-channel control action corresponding to the current control cycle include: The state space vector is input into the policy network, and the probability distribution of the control action is output. Six-dimensional control actions are generated based on the probability distribution of control actions; The six-dimensional control action is divided into subsystem intervention weights and execution increments, resulting in weight increment dual-channel control action; The subsystem intervention weights include steering subsystem intervention weights, braking subsystem intervention weights, and suspension subsystem intervention weights, which are used to express the degree of intervention of the steering, braking, and suspension subsystems, respectively. The execution increments include steering wheel angle increments, braking torque increments, and suspension force increments, which are used to give the specific physical action quantities corresponding to steering wheel angle, braking torque, and suspension force, respectively. The subsystem intervention weight is used to express the trust ratio of each subsystem controller, and the execution increment is used to add a correction amount to the controller output.

5. The vehicle intelligent chassis cooperative control method based on deep reinforcement learning according to claim 4, characterized in that: The process of obtaining the final braking command, the final suspension command, and the final steering command specifically includes: Obtain the basic braking torque output by the brake controller, and calculate the intervention weight of the braking subsystem, the braking torque increment, and the basic braking torque to obtain the final braking execution command; Obtain the basic suspension force output by the suspension controller, and calculate the intervention weight of the suspension subsystem, the suspension force increment, and the basic suspension force to obtain the final suspension execution command; Obtain the optimal front wheel angle output by the steering controller, and calculate the intervention weight of the steering subsystem, the steering wheel angle increment, and the optimal front wheel angle to obtain the final steering execution command; The subsystem intervention weight is used to control the trust ratio of each subsystem controller, and the execution increment is used to add a correction amount to the controller output.

6. The vehicle intelligent chassis cooperative control method based on deep reinforcement learning according to claim 2, characterized in that: The comprehensive reward received specifically includes: The path tracking accuracy is calculated based on the lateral position deviation and heading angle deviation in the state space vector. The driving stability is calculated based on the centroid sideslip angle and yaw rate in the state space vector. Ride comfort is calculated based on the vertical acceleration and pitch angle in the state space vector, and the ride comfort is obtained. Real-time risk indicators are acquired, and dynamic weight adjustments are made based on real-time risk indicators, vehicle speed, and scene identifiers to obtain dynamic weights corresponding to path tracking accuracy, driving stability, and ride comfort, respectively. The real-time risk indicators include the center of gravity sideslip ratio, yaw rate ratio, and reciprocal of collision time. The comprehensive reward is calculated based on a combination of factors including path tracking accuracy, driving stability, ride comfort, dynamic weights, and the difference in actuator commands between adjacent control cycles.

7. The vehicle intelligent chassis cooperative control method based on deep reinforcement learning according to claim 6, characterized in that: The specific method for obtaining the real-time risk indicators is as follows: The ratio of the centroid sideslip angle in the state space vector to the preset maximum centroid sideslip angle is calculated to obtain the centroid sideslip angle ratio. The yaw rate ratio is obtained by calculating the ratio of the yaw rate in the state space vector to the preset maximum yaw rate. The reciprocal of the collision time is obtained by taking the inverse of the collision time between the vehicle and the nearest obstacle. The center of mass sideslip ratio, yaw rate ratio, and reciprocal of collision time are jointly labeled as real-time risk indicators.

8. The vehicle intelligent chassis cooperative control method based on deep reinforcement learning according to claim 6, characterized in that: The obtained dynamic weights for path tracking accuracy, driving stability, and ride comfort specifically include: The vehicle speed is compared with a preset safe speed threshold. When the vehicle speed exceeds the safe speed threshold, a weighting adjustment factor based on the vehicle speed is calculated based on the longitudinal speed, the safe speed threshold, and the gain coefficient. The scenario type corresponding to the current control cycle is identified based on the scenario identifier, and the weight switching data based on the scenario identification is obtained according to the scenario type. Risk analysis was conducted based on the center of mass sideslip ratio, yaw rate ratio, and the reciprocal of the collision time to obtain a comprehensive risk coefficient. By comprehensively calculating the weight adjustment factor of vehicle speed, the weight switching data of scene recognition, and the comprehensive risk coefficient, dynamic weights corresponding to path tracking accuracy, driving stability, and ride comfort are obtained.

9. The vehicle intelligent chassis cooperative control method based on deep reinforcement learning according to claim 1, characterized in that: The control strategy obtained for collaborative control of intelligent chassis systems in multiple scenarios specifically includes: Randomly sample batch data from the experience replay pool, the batch data comprising multiple experience tuples; The value network parameters are updated based on batch data, and the value estimation results are output through the value network. The policy network parameters are updated based on the value estimation results and the probability distribution of control actions. The target network parameters are softly updated based on the policy network parameters and the value network parameters to obtain the updated target network parameters. Based on the state space vector, the final execution command of braking, the final execution command of suspension, and the final execution command of steering, determine whether the current training round triggers the round termination condition; If the round termination condition is triggered, the current training round is stopped, and the vehicle dynamic data, actuator current state data and scene identifier corresponding to the next training round are reacquired, thereby executing the training for the next round. If the round termination condition is not triggered, the control action generation, dynamic reward function calculation, and experience tuple storage into the experience replay pool will continue based on the state space vector corresponding to the next control cycle. Batch data will be sampled from the experience replay pool, and the network parameters of the deep reinforcement learning algorithm model will be updated based on the batch data. Repeatedly execute the following steps: generating control actions, calculating dynamic reward functions, storing experience tuples in the experience replay pool, sampling batch data from the experience replay pool, and updating the network parameters of the deep reinforcement learning algorithm model based on the batch data, until the deep reinforcement learning algorithm model converges, to obtain a control strategy for collaborative control of intelligent chassis systems in multiple scenarios. The round termination conditions include dynamic instability, road boundary deviation, collision, and mission completion.

10. A system applying the vehicle intelligent chassis cooperative control method based on deep reinforcement learning as described in any one of claims 1-9, characterized in that: Includes the following modules: The state acquisition module is used to acquire key information about the vehicle and its surrounding environment, integrate the states, and generate the state space vector corresponding to the current control cycle. The initial action generation module is used to input the state space vector into a preset deep reinforcement learning algorithm model to generate control actions, thereby obtaining the weight increment dual-channel control action corresponding to the current control cycle. The collaborative calculation module is used to obtain the baseline execution data output by the brake controller, suspension controller and steering controller respectively, and to perform collaborative calculation with the subsystem intervention weight and execution increment to obtain the final execution command of braking, suspension and steering. The comprehensive reward analysis module is used to calculate the comprehensive reward by performing dynamic reward function based on the state space vector, the final execution command of braking, the final execution command of suspension, and the final execution command of steering. The experience tuple storage module is used to assemble the state space vector, the weighted incremental dual-channel control action, the comprehensive reward, and the state space vector corresponding to the next control cycle into an experience tuple, and store the experience tuple into the experience replay pool. The collaborative strategy generation module is used to sample batch data from the experience replay pool and update the network parameters of the deep reinforcement learning algorithm model based on the batch data to obtain a control strategy for collaborative control of intelligent chassis systems in multiple scenarios.

Images