A vehicle stability coordination control method based on reinforcement learning

Abstract

The present invention discloses a vehicle stability coordinated control method based on reinforcement learning, comprising: constructing a reinforcement learning network model, including: a strategy network, a value network, a target strategy network and a target value network; using vehicle driving state parameters as input variables of the strategy network, and using allocation coefficients of an active front wheel steering device and a yaw torque controller as output variables of the strategy network; using vehicle driving state parameters and their corresponding allocation coefficients of an active front wheel steering device and a yaw torque controller as input variables of a value network, and the value network outputs an estimated value of a current strategy; optimizing multiple reinforcement learning network models respectively with the maximum estimated value as an optimization target, and selecting an optimized network model with the maximum cumulative return as the optimal network model; during vehicle driving, obtaining the allocation coefficients of an active front wheel steering device and a yaw torque controller through the optimal network model, and coordinately controlling an additional front wheel steering angle and an additional yaw torque of the vehicle.

Description

Technical Field

[0001] The invention belongs to the technical field of electric vehicle stability control, and in particular relates to a vehicle stability coordination control method based on reinforcement learning. Background Art

[0002] Four-wheel independent drive electric vehicles can achieve vehicle stability through active front wheel steering and yaw moment control, but the two use the same actuator, which means that they will interfere with each other during the execution process. Therefore, how to combine active front wheel steering with yaw moment control to improve the maneuverability and stability of the vehicle during the steering process, thereby ensuring the safety of the vehicle, is an urgent problem to be solved. Summary of the invention

[0003] The purpose of the present invention is to provide a vehicle stability coordinated control method based on reinforcement learning, which can solve the coordinated control problem of the additional front wheel angle and yaw moment of an electric vehicle under the premise of ensuring safety.

[0004] The technical solution provided by the present invention is:

[0005] A vehicle stability coordination control method based on reinforcement learning, comprising:

[0006] Construct a strong chemical network model, which includes: strategy network, value network, target strategy network and target value network;

[0007] The vehicle driving state parameters are used as input variables of the strategy network, and the allocation coefficients of the active front wheel steering gear and the yaw moment controller are used as output variables of the strategy network; the vehicle driving state parameters and the corresponding allocation coefficients of the active front wheel steering gear and the yaw moment controller are used as input variables of the value network, and the value network outputs the estimated value of the current strategy;

[0008] The target strategy network is used to select the output variable of the strategy network, and the target value network is used to calculate the output variable of the value network;

[0009] The vehicle driving state parameters include: vehicle longitudinal speed, front wheel turning angle, center of mass sideslip angle, yaw rate, the difference between yaw rate and nominal yaw rate, and road adhesion coefficient;

[0010] The calculation formula of the estimated value is:

[0011] r t (s t ,a t )=A1r β +A2r w +A3r Δw +A4r m ;

[0012] In the formula, r β is the center of mass sideslip angle report, r w is the angular velocity return, r Δw is the yaw rate error report, r m The center of mass sideslip angle and yaw rate limit report; A1, A2, A3, A4 are r β 、r w 、r Δw 、r m The weight value of

[0013] Initializing a set of policy network parameters and a set of value network parameters to obtain multiple reinforcement learning network models; optimizing the multiple reinforcement learning network models respectively with the maximum estimated value as the optimization goal to obtain multiple optimized network models;

[0014] Select the optimized network model with the largest cumulative return as the optimal network model;

[0015] During vehicle driving, vehicle state parameters are detected in real time as input variables of the optimal network model, and the distribution coefficients of the active front wheel steering device and the yaw moment controller are obtained through the optimal network model; and the additional front wheel steering angle and additional yaw moment of the vehicle are coordinated and controlled according to the distribution coefficients.

[0016] Preferably, the mass center side slip angle report r β , the angular velocity of the pendulum is reported as r w and the yaw rate error is reported as r Δw The value rules are:

[0017] When the sideslip angle β∈(β min ,β max ), r β =1, otherwise r β =0;

[0018] When the yaw angular velocity ω∈(ω min ,ω max ), r w =1, otherwise r w =0;

[0019] When the difference between the yaw rate and the nominal yaw rate Δω∈(Δω min ,Δω max ), r Δw =1, otherwise r Δw =0;

[0020] When the sideslip angle and yaw rate both exceed their maximum values, r m =0, otherwise r m =1;

[0021] Among them, β min ,β max are the minimum and maximum values ​​of the sideslip angle of the center of mass respectively; ω min ,ω max are the minimum and maximum values ​​of the yaw angular velocity respectively; Δω min ,Δω max are the minimum and maximum values ​​of the difference between the yaw rate and the nominal yaw rate, respectively.

[0022] Preferably, according to the current front wheel turning angle δ f and road adhesion coefficient μ to determine β min ,β max ,ω min ,ω max , Δω min ,Δω max The value of:

[0023] When μ≤0.2 or v>v max or δ f >δ fmax hour:

[0024] β min =-β limit1 ,β max =β limit1

[0025] ω min =-Ψω limit ,ω max =Ψω limit

[0026] Δω min =-Δω limit1 ,Δω max =Δω limit1

[0027] Among them, β limit1 is the first limit value of the center of mass sideslip angle; Δω limit1 is the first limit value of the yaw rate deviation; ω limit is the first limit value of the yaw rate; Ψ is the yaw rate gain value; β limit1 =0.05rad, Δω limit1 =0.02rad / s;δ fmax is the front wheel turning angle limit threshold, which is set according to the vehicle conditions; v max is the speed limit threshold, and its size is set according to the vehicle conditions;

[0028] When μ≤0.2, v>v max or v>v max ,δ f>δ fmax or μ≤0.2, δ f >δ fmax hour:

[0029] β min =-β limit2 ,β max =β limit2

[0030] ω min =-Ψω limit ,ω max =Ψω limit

[0031] Δω min =-Δω limit2 ,Δω max =Δω limit2

[0032] Among them, β limit2 is the second limit value of the sideslip angle at the center of mass; Δω limit2 is the second limit value of the yaw rate deviation; β limit2 =0.03rad, Δω limit2 =0.01rad / s;

[0033] When μ≤0.2, v>v max ,δ f >δ fmax hour:

[0034] β min =-β limit3 ,β max =β limit3

[0035] ω min =-Ψω limit ,ω max =Ψω limit

[0036] Δω min =-Δω limit3 ,Δω max =Δω limit3

[0037] Among them, β limit3 is the third limit value of the sideslip angle at the center of mass; Δω limit3 is the third limit value of the yaw rate deviation; β limit3 =0.01rad, Δω limit3 =0.005rad / s;

[0038] Other cases:

[0039] Among them, β limit is the initial limit value of the sideslip angle at the center of mass; Δω limit is the initial limit value of the yaw rate deviation; β limit =0.2rad, Δω limit =0.005rad / s.

[0040] Preferably, ω limit Determined by the following formula:

[0041]

[0042] Among them, G is the calculation coefficient between the front wheel angle and the desired yaw rate; v is the vehicle speed, and g is the acceleration due to gravity.

[0043] Preferably, Ψ is determined by the following formula:

[0044]

[0045] in,

[0046] Where k is the yaw rate gain coefficient; c1 is the yaw rate gain coefficient correlation coefficient, and m is the vehicle mass.

[0047] Preferably, the vehicle stability coordination control method based on reinforcement learning further includes:

[0048] With the minimum tire utilization as the goal, according to the additional yaw moment M z * And the driver's demand torque T, control the output torque of the four wheels:

[0049]

[0050]

[0051]

[0052] Among them, T rl , T rr , T fl , T fr They are the driving torques of the left rear wheel, right rear wheel, left front wheel, and right front wheel respectively; ΔM z * is the actual additional yaw moment; μ rl , μ rr , μ fl , μ fr are the road adhesion coefficients of the left rear wheel, right rear wheel, left front wheel, and right front wheel respectively; B is the wheel track; r is the wheel rolling radius; F z_rl 、F z_rr 、Fz-fl 、F z-fr They are the wheel loads of the left rear wheel, right rear wheel, left front wheel and right front wheel respectively.

[0053] Preferably, in the process of optimizing the reinforcement learning network model, the front wheel steering angle output by the driver model is used as an input parameter.

[0054] Preferably, the nominal yaw rate is output via a two-degree-of-freedom reference model.

[0055] The beneficial effects of the present invention are:

[0056] The vehicle stability coordinated control method based on reinforcement learning provided by the present invention can combine active front wheel steering with yaw moment control, thereby improving the maneuverability and stability of the vehicle during the steering process, thereby ensuring the safety of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a diagram of the vehicle stability coordination control strategy based on reinforcement learning described in the present invention.

[0058] Figure 2 This is a flow chart of the vehicle stability coordination control based on reinforcement learning described in the present invention. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0060] like Figure 1-2 As shown, the present invention provides a vehicle stability coordination control method based on reinforcement learning, which mainly includes two processes. The first process is to design a reinforcement learning coordination controller and complete the training of the reinforcement learning controller. The second process is to migrate the trained controller to the whole vehicle controller, determine the control relationship between the active front wheel steering angle and the additional yaw moment, and coordinate the control of the vehicle's active front wheel steering angle and the additional yaw moment.

[0061] The reinforcement learning coordination controller includes a driver model, a two-degree-of-freedom reference model, a stability coordination controller, a torque distribution module and a vehicle model. The driver model can generate torque requirements and steering wheel angle signals; the two-degree-of-freedom reference model can generate nominal yaw rate and nominal center-of-mass sideslip angle; the stability coordination controller includes an active front wheel steering gear, a yaw torque controller and a reinforcement learning coordination control model; the stability coordination controller is used to coordinate the control of the active front wheel angle and yaw torque to generate appropriate additional front wheel angle and yaw torque, wherein the stability coordination controller is coordinated by the reinforcement learning coordination control model; the torque distribution module can optimize the torque distribution among the four wheels, with the minimum wheel utilization as the optimization target, while meeting the driver's torque and yaw torque requirements.

[0062] The reinforcement learning controller training design process is as follows:

[0063] 1. Design a reinforcement learning coordination controller to solve the environment, agent, action and reward.

[0064] The reinforcement learning agent actions are as follows:

[0065] a t ={κ},κ∈[0,1];

[0066] Where κ is the allocation coefficient of the active front wheel steering and the yaw moment controller.

[0067] The state of the reinforcement learning agent is as follows:

[0068] s t ={β,∫β,Δω,∫Δω,ω,v x ,μ,δ f}

[0069] Where β is the sideslip angle at the center of mass, ∫β is the integral value of the sideslip angle at the center of mass, Δω is the difference between the actual yaw rate and the nominal yaw rate, ∫Δω is the integral of the difference between the actual yaw rate and the nominal yaw rate, v x is the longitudinal velocity, μ is the road adhesion coefficient, δ f is the front wheel turning angle.

[0070] The rewards (estimated value) of the reinforcement learning agent are as follows:

[0071] r t (s t ,a t )=A1r β +A2r w +A3r Δw +A4r m

[0072] In this embodiment, the specific rule is set as follows: when the center of mass sideslip angle β∈(β min ,β max ), the center of mass sideslip angle report r β =1, otherwise r β =0; when the yaw angular velocity ω∈(ω min ,ω max ), the yaw rate report rw=1, otherwise rw=0; when the difference between the actual yaw rate and the nominal yaw rate Δω∈(Δω min ,Δω max ), yaw rate error report r Δw =1, otherwise r Δw = 0; when the sideslip angle and yaw rate exceed their maximum values, the sideslip angle and yaw rate limit report r m =0, otherwise r m = 1. A1, A2, A3, A4 are r β 、r w 、r Δw 、r m The weight value of .

[0073] The training return is defined as the cumulative return, which is calculated as follows:

[0074] U t =r t +λ·r t+1 +λ 2 ·r t+2 +…+λ k ·r t+k =r t +λ·U t+1

[0075] Where λ is the discount factor, 0<λ<1; r t 、r t+1 、r t+2 ...are the returns corresponding to the tth, t+1, t+2, ... training cycles respectively.

[0076] Determine β based on the current front wheel steering angle and road adhesion coefficient min ,β max ,ω min ,ω max , Δω min ,Δω max When the front wheel turning angle is large and the road adhesion coefficient is low, β should be reduced. min ,β max ,ω min ,ω max , Δω min ,Δω maxAs a further embodiment of the present invention, the correlation coefficient is determined as follows:

[0077] When μ≤0.2 or v>v max or δ f >δ fmax hour:

[0078]

[0079] Among them, β limit1 =0.05, Δω limit1 =0.02;

[0080] When μ≤0.2 and v>v max , or v>v max And δ f >δ fmax , or μ≤0.2 and δ f >δ fmax hour:

[0081]

[0082] Among them, β limit2 =0.03, Δω limit2 =0.01;

[0083] When μ≤0.2, v>v max And δ f >δ fmax ,

[0084]

[0085] Among them, β limit3 =0.01, Δω limit3 =0.005;

[0086] Other cases:

[0087]

[0088] Among them, β limit =0.2, Δω limit =0.005;

[0089]

[0090]

[0091]

[0092] Where c1 is the correlation coefficient of the yaw rate gain coefficient, v is the vehicle speed, K f , K ris the total cornering stiffness of the front and rear wheels, L is the wheelbase, a and b are the distances from the center of mass to the front and rear axles respectively, and m is the vehicle mass.

[0093] 2. Construct a deep reinforcement learning coordinated control network model, including policy network, value network, target policy network, and target value network.

[0094] 2.1. The policy network includes the first input layer, the first hidden layer, the second hidden layer, and the first output layer. The input is the system state s t , a total of 8 input neurons, the output is a t ; Initialize the strategy network parameter set θ0;

[0095] 2.2. The value network includes the second input layer, the third hidden layer, the fourth hidden layer, and the second output layer. The input includes the system state s t 、Action a t , a total of nine input neurons, the output is the estimated value (reward) r t ; Initialize the value network parameter set w0;

[0096] 2.3. The target policy network has the same structure as the policy network and is responsible for replaying the next state s sampled in the pool based on the experience t Choose the best next strategy a t ; Let the target network parameter θ'0 = θ0, where θ'0 is the initial target strategy network parameter and θ0 is the initial strategy network parameter;

[0097] 2.4. The target value network has the same structure as the value network and is responsible for calculating the target r t value; let the target network parameter w'0 = w0, where w'0 is the initial target value network parameter and w0 is the initial value network parameter;

[0098] 3. Train the deep reinforcement learning network model;

[0099] 3.1. Initialize the vehicle's tth state parameter s t ;

[0100] 3.2. Improving the exploratory nature of the agent’s strategy by adding noise to the policy network

[0101] a t =μ(s t ,θ t )+ε t ;

[0102] Among them, ε t is the motion noise, s t is the vehicle state at time t, θ t is the strategic network parameter at time t;

[0103] 3.2. Estimated value r based on value network output t ;

[0104] 3.3. According to different vehicle status t The value obtained t And the vehicle state s at the next moment t+1 , store the data set (s t ,a t ,r t ,s t+1 );

[0105] 3.4. Calculate the time difference target value, that is

[0106] y t =r t +λ·q(s t+1 ,μ(s t+1 ,θ t ), w t )

[0107] Among them, q(s t+1 ,μ(s t+1 ,θ t ), w t ) is the value function at the next moment, λ is the discount coefficient, μ(s t ,θ t ) is the policy network, q(s t ,a t ,w t ) is the value network;

[0108] 3.5. Define the minimization loss function

[0109]

[0110] Computing Gradients

[0111] ▽ ω L(ω)=(q t -y t )▽ ω q(s t ,a t ,w t )

[0112] Update the value network parameter w by gradient descent

[0113] w t+1 ←αω t -(1-α)▽ ω L(w t );

[0114] w′ t+1 =w t+1

[0115] Among them, α is the value network learning rate, w t is the value network parameter at time t, w t+1 is the value network parameter at time t+1, w t ' +1 is the target value network parameter at time t+1;

[0116] 3.6. Update θ by gradient ascent

[0117] θ t+1 ←τθ t +(1-τ)▽ θ μ(s t ,θ t )·▽ a q(s t ,a t ,ω t )

[0118] θ' t+1 =θ t+1

[0119] τ is the policy network learning rate, θ t is the policy network parameter at time t, θ t+1 is the policy network parameter at time t+1, θ' t+1 is the target strategy network parameter at time t+1;

[0120] 3.7. During a single training process, when the vehicle yaw rate deviation and center of mass sideslip angle deviation are both within the threshold, the model stops the current training. When the predetermined number of training times is reached, the optimal strategy network model and value network model are obtained based on the cumulative return:

[0121] Δω<Δω m ,Δβ<Δβ m

[0122] Among them, Δω m is the yaw rate training stop threshold, Δβ m Threshold for stopping training for the sideslip angle.

[0123] 4. Migrate the trained reinforcement learning strategy network model to the vehicle controller.

[0124] As a further solution of the present invention: the vehicle stability coordination control is designed as follows:

[0125] 1. When receiving the driver's steering signal, the two-degree-of-freedom model generates the nominal yaw rate and nominal center-of-mass sideslip angle and sends them to the vehicle stability coordination controller.

[0126] The differential motion equation of the two-degree-of-freedom model is described as follows:

[0127]

[0128] When the vehicle is in a steady state, the yaw rate and the sideslip angle of the center of mass are both constant, that is, Then the desired yaw rate and sideslip angle can be calculated as:

[0129]

[0130]

[0131] Among them, δ f is the front wheel turning angle, v x is the vehicle speed, L is the wheelbase, m is the vehicle mass, a is the distance from the vehicle center of mass to the front axle, b is the distance from the vehicle center of mass to the rear axle, and K is the stability factor K f ,K r I is the front and rear axle lateral stiffness; z is the vehicle's moment of inertia around the z-axis.

[0132] 2. The vehicle stability coordination controller will adjust the additional front wheel steering angle Δδ according to the current state of the vehicle and the output of the active front wheel steering gear. f And the additional yaw moment M output by the yaw moment controller z , and obtain the optimal distribution coefficient κ of active front wheel steering and yaw moment control.

[0133] 3. Calculation of the nominal additional front wheel steering angle Δδ based on the active front wheel steering gear f , and outputs the additional front wheel steering angle Δδ according to the distribution coefficient κ f * , whose sizes are as follows:

[0134] Δδ f * =κ×Δδ f

[0135] 4. Calculate the additional yaw moment M according to the yaw moment controller z , and outputs the additional yaw moment M according to the distribution coefficient κ z * , whose sizes are as follows:

[0136] M z * =(1-κ)×M z

[0137] 5. The front wheel is adjusted according to the additional front wheel turning angle Δδ f * and the driver input steering angle δ driver , output its actual front wheel steering angle δfact , whose sizes are as follows:

[0138] δ fact =δ driver +Δδ f *

[0139] 6. The torque distribution module receives the additional yaw moment M z * After the driver's required torque T is calculated, the actual output torque of the four wheels is calculated:

[0140] The objective function takes the minimum tire utilization as the optimization target.

[0141]

[0142] Among them, F x-ij is the longitudinal driving force of each wheel hub motor, F x-ij is the wheel lateral force, μ ij is the road adhesion coefficient of each wheel, F z-ij is the vertical load on each wheel.

[0143] The wheel hub motor can only control the longitudinal force, which can be simplified to

[0144]

[0145] in:

[0146]

[0147] Where, T ij is the driving torque of the wheel hub motor of each wheel, and r is the rolling radius of the wheel. The final objective function is:

[0148]

[0149] available

[0150] T fl +T fr +T rl +T rr =T

[0151]

[0152] Wherein, B is the wheel track.

[0153] The formula for the front axle driving torque can be obtained:

[0154]

[0155]

[0156] Substituting the above formula into the final objective function, we can get

[0157]

[0158] Find T from the above formula fl and T fr Partial Derivative

[0159]

[0160]

[0161] Let the above formulas be equal to zero, then the minimum value of the objective function can be obtained as follows:

[0162]

[0163]

[0164] Substituting the above formula back into the front axle driving torque formula, we can get T fl and T fr :

[0165]

[0166]

[0167] 7. Send the calculated torque of each wheel to the execution system.

[0168] The present invention determines the coordinated control relationship between the active front wheel steering device and the yaw torque controller through a reinforcement learning network model, which makes up for the shortcomings of traditional control schemes in the coordination between the two and enables the controller to select a better control strategy according to the vehicle state.

[0169] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A vehicle stability coordination control method based on reinforcement learning, characterized in that: include: Constructing a reinforcement learning network model, wherein the network model includes: a strategy network, a value network, a target strategy network, and a target value network; The vehicle driving state parameters are used as input variables of the strategy network, and the allocation coefficients of the active front wheel steering gear and the yaw moment controller are used as output variables of the strategy network; the vehicle driving state parameters and the corresponding allocation coefficients of the active front wheel steering gear and the yaw moment controller are used as input variables of the value network, and the value network outputs the estimated value of the current strategy; The target strategy network is used to select the output variable of the strategy network, and the target value network is used to calculate the output variable of the value network; The vehicle driving state parameters include: vehicle longitudinal speed, front wheel turning angle, center of mass sideslip angle, yaw rate, the difference between yaw rate and nominal yaw rate, and road adhesion coefficient; The calculation formula of the estimated value is: r t (s t ,a t )=A1r β +A2r w +A3r Δw +A4r m ; In the formula, r β is the center of mass sideslip angle report, r w is the angular velocity return, r Δw is the yaw rate error report, r m The center of mass sideslip angle and yaw rate limit report; A1, A2, A3, A4 are r β 、r w 、r Δw 、r m The weight value of Initializing a set of policy network parameters and a set of value network parameters to obtain multiple reinforcement learning network models; optimizing the multiple reinforcement learning network models respectively with the maximum estimated value as the optimization goal to obtain multiple optimized network models; Select the optimized network model with the largest cumulative return as the optimal network model; During vehicle driving, vehicle state parameters are detected in real time as input variables of the optimal network model, and the distribution coefficients of the active front wheel steering device and the yaw moment controller are obtained through the optimal network model; and the additional front wheel steering angle and additional yaw moment of the vehicle are coordinated and controlled according to the distribution coefficients.

2. The vehicle stability coordination control method based on reinforcement learning according to claim 1 is characterized in that: Center of mass slip angle report r β , the angular velocity of the pendulum is reported as r w and the yaw rate error report r Δw The value rules are: When the center of mass sideslip angle β∈(β min ,β max ), r β =1, otherwise r β =0; When the yaw angular velocity ω∈(ω min ,ω max ), r w =1, otherwise r w =0; When the difference between the yaw rate and the nominal yaw rate Δω∈(Δω min ,Δω max ), r Δ w=1,otherwise r Δw =0; When the sideslip angle and yaw rate both exceed their maximum values, r m =0, otherwise r m =1; Among them, β min ,β max are the minimum and maximum values ​​of the sideslip angle of the center of mass respectively; ω min ,ω max are the minimum and maximum yaw angular velocity respectively; Δω min ,Δω max are the minimum and maximum values ​​of the difference between the yaw rate and the nominal yaw rate, respectively.

3. The vehicle stability coordination control method based on reinforcement learning according to claim 2 is characterized in that: According to the current front wheel steering angle δ f and road adhesion coefficient μ to determine β min ,β max ,ω min ,ω max , Δω min ,Δω max The value of: When μ≤0.2 or v>v max or δ f >δ fmax hour: β min ＝-β limit1 ,β max ＝β limit1 oh min =-Pso limit ,oh max =Pso limit Give min =-See limit1 ,I see max =Here limit1 Among them, β limit1 is the first limit value of the sideslip angle at the center of mass; Δω limit1 is the first limit value of the yaw rate deviation; ω limit is the first limit value of the yaw rate; Ψ is the yaw rate gain value; β limit1 =0.05, Δω limit1 =0.02; δ fmax is the front wheel turning angle limit threshold; v max is the speed limit threshold; When μ≤0.2, v>v max or v>v max ,δ f >δ fmax or μ≤0.2, δ f >δ fmax hour: β min ＝-β limit2 ,β max ＝β limit2 oh min =-Pso limit ,oh max =Pso limit Give min =-See limit2 ,I see max =Here limit2 Among them, β limit2 is the second limit value of the mass center sideslip angle; Δω limit2 is the second limit value of the yaw rate deviation; β limit2 =0.03, Δω limit2 =0.01; When μ≤0.2, v>v max , δ f >δ fmax hour: β min ＝-β limit3 ,β max ＝β limit3 oh min =-Pso limit ,oh max =Pso limit Give min =-See limit3 ,I see max =Here limit3 Among them, β limit3 is the third limit value of the sideslip angle at the center of mass; Δω limit3 is the third limit value of the yaw rate deviation; β limit3 =0.01, Δω limit3 =0.005; Other cases: Among them, β limit is the initial limit value of the sideslip angle at the center of mass; Δω limit is the initial limit value of the yaw rate deviation; β limit =0.2, Δω limit =0.

005.

4. The vehicle stability coordination control method based on reinforcement learning according to claim 3 is characterized in that: ω limit Determined by the following formula: Among them, G is the calculation coefficient between the front wheel angle and the desired yaw rate; v is the vehicle speed, and g is the acceleration due to gravity.

5. The vehicle stability coordination control method based on reinforcement learning according to claim 3 or 4, characterized in that: Ψ is determined by the following formula: in, Where k is the yaw rate gain coefficient; c1 is the yaw rate gain coefficient correlation coefficient, and m is the vehicle mass.

6. The vehicle stability coordination control method based on reinforcement learning according to claim 1, characterized in that: Also includes: With the minimum tire utilization as the goal, according to the additional yaw moment M z * And the driver's demand torque T, control the output torque of the four wheels: Among them, T rl , T rr , T fl , T fr They are the driving torques of the left rear wheel, right rear wheel, left front wheel, and right front wheel respectively; ΔM z * is the actual additional yaw moment; μ rl , μ rr , μ fl , μ fr are the road adhesion coefficients of the left rear wheel, right rear wheel, left front wheel, and right front wheel respectively; B is the wheel track; r is the wheel rolling radius; F z_rl 、F z_rr 、F z_fl 、F z_fr They are the wheel loads of the left rear wheel, right rear wheel, left front wheel and right front wheel respectively.

7. The vehicle stability coordination control method based on reinforcement learning according to claim 1 is characterized in that: In the process of optimizing the reinforcement learning network model, the front wheel steering angle output by the driver model is used as the input parameter.

8. The vehicle stability coordination control method based on reinforcement learning according to claim 7 is characterized in that: The nominal yaw rate is output by a two-degree-of-freedom reference model.

Images