Automobile stability coordination control method based on reinforcement learning

Abstract

The invention discloses an automobile stability coordination control method based on reinforcement learning, and belongs to the technical field of electric automobile control, the method comprises the following steps: using a linear two-degree-of-freedom model to obtain an expected yaw velocity and an expected side slip angle; additional yawing moment and a steering angle of a driving rear wheel are obtained through sliding mode control; acquiring the state type of the current vehicle; constructing an intelligent agent and a reward function of reinforcement learning, and continuously training the intelligent agent until the optimal cooperative control coefficients of the additional yaw moment and the steering angle of the active rear wheel are output, so as to realize the cooperative control of the electric vehicle; the reward function includes a steady state reward, a yaw velocity error and side slip angle error reward, a critical steady state reward, and an anti-jitter reward. According to the invention, the steering and yawing moment of the driving rear wheel can be coordinated and controlled, so that the stability and safety of the vehicle are ensured.

Description

Technical Field

[0001] The invention belongs to the technical field of electric vehicle control, and in particular relates to a vehicle stability coordination control method based on reinforcement learning. Background Art

[0002] The development of new energy vehicles is gradually accelerating, and they have unique advantages such as no pollution, zero emission, simple structure and easy control. Distributed drive installs the motor directly inside the wheel hub or wheel edge of each wheel, directly drives the wheel through the motor, and the torque of each wheel is independently controllable, giving full play to the advantages of the motor's fast response speed and precise torque adjustment, providing more degrees of freedom for vehicle dynamics control.

[0003] Distributed drive electric vehicles can control the output torque of the four drive motors to meet the longitudinal movement of the vehicle, and can also achieve lateral additional yaw torque and active rear wheel steering by changing the torque of some drive motors. When the vehicle is driving at low speed, active rear wheel steering can provide a turning angle in the opposite direction of the front wheel turning angle, reduce the turning radius of the vehicle, and increase the flexibility of the vehicle; when driving at high speed, active rear wheel steering can provide a turning angle in the same direction as the front wheel turning angle to avoid the vehicle from skidding. However, under extreme working conditions, the tire force tends to be saturated. At this time, the tire cannot provide enough lateral force to maintain the stability of the vehicle, and the active rear wheel steering system cannot be used to improve the handling stability of the vehicle. At this time, the longitudinal force of the tire can be changed to form a yaw torque to control the vehicle, which can achieve the purpose of maintaining the stability of the vehicle; but as a complex nonlinear system, how to combine active rear wheel steering with yaw torque control to improve the maneuverability and stability of the vehicle during the steering process, thereby ensuring the safety of the vehicle is an urgent problem to be solved. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a vehicle stability coordination control method based on reinforcement learning, which can coordinate the control of active rear wheel steering and yaw torque to ensure the stability and safety of the vehicle.

[0005] The present invention provides the following technical solutions:

[0006] A vehicle stability coordinated control method based on reinforcement learning is provided, comprising:

[0007] According to the vehicle's structural parameters, vehicle speed and front wheel angle, a linear two-degree-of-freedom model is used to obtain the expected yaw rate ω d and the desired sideslip angle β d ;

[0008] According to the expected yaw rate ω d , actual yaw rate, expected sideslip angle β dThe actual center of mass sideslip angle is controlled by sliding mode to obtain the additional yaw moment ΔM and the active rear wheel steering angle δ r ;

[0009] Based on the established vehicle state parameter cluster diagram under the set working condition and the state parameters of the current vehicle, the state type of the current vehicle is obtained; the state type is one of a stable state, a critical stable state and an unstable state;

[0010] Construct a reinforcement learning agent and reward function, and continuously train the agent until it outputs the additional yaw torque ΔM and the active rear wheel steering angle δ r The respective optimal cooperative control coefficients are used to achieve coordinated control of the electric vehicle; the reward function includes a stable state reward, a yaw angular velocity error and a center of mass sideslip angle error reward, a critical stable state reward and an anti-shake reward.

[0011] Optionally, the structural parameters, vehicle speed and front wheel steering angle of the vehicle are used to obtain the expected yaw angular velocity ω using a linear two-degree-of-freedom model. d and the desired sideslip angle β d middle:

[0012] The linear two-degree-of-freedom model is in the form of:

[0013]

[0014] Among them, k1 is the stiffness of the front axle of the vehicle, k2 is the stiffness of the rear axle of the vehicle, β is the side slip angle of the vehicle center of mass, and v x is the longitudinal speed, a is the distance from the vehicle center of mass to the front axle, b is the distance from the vehicle center of mass to the rear axle, ω exp is the expected yaw rate output by the linear two-degree-of-freedom model, δ is the standard front wheel turning angle, m is the vehicle mass, and a y is the lateral acceleration of the vehicle, I z is the vehicle's moment of inertia, ω exp The derivative of

[0015] The desired yaw rate ω output by the linear two-degree-of-freedom model is exp for:

[0016]

[0017] Among them, K is the vehicle stability factor;

[0018] The desired yaw rate ω d and the desired sideslip angle β d for:

[0019]

[0020] Among them, μ is the adhesion coefficient, g is the acceleration of gravity, β d is the expected sideslip angle of the center of mass, sgn(δ) is the judgment function marking the sign of the front wheel turning angle, when the front wheel turning angle is positive, sgn(δ) is 1, when the front wheel turning angle is negative, sgn(δ) is -1.

[0021] Optionally, the standard front wheel turning angle δ is the average of the wheel turning angles of the two front wheels of the vehicle; the wheel turning angles of the two front wheels of the vehicle and the longitudinal vehicle speed v x The vehicle's rotational inertia I is obtained by a sensor; z Based on the vehicle's expected torque T d Obtain the desired torque T of the vehicle d for:

[0022]

[0023] Where θ is the accelerator pedal opening, T imax is the peak torque of the i-th wheel hub motor.

[0024] Optionally, the desired yaw rate ω d , actual yaw rate, expected sideslip angle β d The actual center of mass sideslip angle is controlled by sliding mode to obtain the additional yaw moment ΔM and the active rear wheel steering angle δ r , specifically including:

[0025] According to the expected yaw rate ω d and the actual yaw angular velocity, design the sliding surface of the yaw moment controller, and obtain the additional yaw moment ΔM;

[0026]

[0027] According to the expected sideslip angle β d and the actual center of mass sideslip angle, design the sliding surface of the active rear wheel controller, and obtain the active rear wheel steering angle δ r ;

[0028]

[0029] Where Iz is the moment of inertia of the vehicle, c ω is the relative weight coefficient between the error and the error change rate, a is the distance from the vehicle center of mass to the front axle, b is the distance from the vehicle center of mass to the rear axle, C f is the stiffness of the front wheel of the vehicle, C r The stiffness of the rear wheels of the vehicle, v x is the longitudinal speed, and e ω and e βThe derivative of ω is the control variable deviation of yaw rate, e β is the control variable deviation of the sideslip angle of the center of mass, K0 is the approach rate constant, s ω is the sliding mode shear function.

[0030] Optionally, in the acquisition of the state type of the current vehicle based on the established cluster diagram of vehicle state parameters under set working conditions and the state parameters of the current vehicle, there are a total of 5 cluster centers in the cluster diagram of vehicle state parameters under set working conditions, among which the middle cluster is a stable state, the clusters located on the left and right edges of the middle cluster are unstable states, and the cluster located between the stable state and the unstable state is a critical stable state;

[0031] The vehicle state parameters include: lateral velocity, lateral acceleration, yaw rate, center of mass sideslip angle and tire sideslip angle.

[0032] Optionally, the intelligent agent and reward function of reinforcement learning are constructed, and the intelligent agent is continuously trained until the additional yaw moment ΔM and the active rear wheel steering angle δ are output. r The optimal coordinated control coefficients of each include:

[0033] Define the state s of the agent t for:

[0034] s t ={ω,Δω,β,Δβ,a y}

[0035] Where Δω is the difference between the yaw rate and the actual yaw rate, Δβ is the difference between the desired center of mass sideslip angle and the actual center of mass sideslip angle, ω is the actual yaw rate, β is the actual center of mass sideslip angle, and a y is the lateral acceleration;

[0036] Define action a t is the coordinated relationship between the rear wheel steering angle and the additional yaw moment, expressed as follows:

[0037] a t ={κ,1-κ},κ∈[0,1]

[0038] Where κ is the active rear wheel steering angle δ r The coordinated control coefficient of , 1-κ is the coordinated control coefficient of the additional yaw moment ΔM;

[0039] Define the reward function r t for:

[0040] r t =a1r s +a2r e +a3r cs +a4rc

[0041] Among them, r s is the steady state reward, r e is the yaw rate error and center of mass sideslip angle error reward, r cs is the critical stable state reward, r c For anti-shake reward, a1, a2, a3 and a4 are r s 、r e 、r cs and r c The reward factor.

[0042] Optionally, the steady-state reward r s It includes the current state reward of the vehicle and the predicted state reward of the vehicle in the future. The specific formula is:

[0043] r s =r s1 +r s2 +r s3 +r s4

[0044]

[0045] Among them, r s1 is the reward for the car’s current state, r s2 、r s3 and r s4 are the state rewards of the car at the first, second and third moments of the prediction respectively; Dis1 is the Euclidean distance from the vehicle state parameter to the cluster center; t1, t2, t3 and t4 are r s1 、r s2 、r s3 and r s4 The adjustment coefficient; q is the category of the cluster to which the current vehicle belongs; q = 1 means the current vehicle is in a stable state, q = 2 or q = 4 means the current vehicle is in a critically stable state, and q = 3 and q = 5 both mean the current vehicle is in an unstable state;

[0046] r s2 、r s3 and r s4 The corresponding clustering categories are obtained based on the predicted vehicle state parameters and the clustering diagram of the vehicle state parameters under the set working conditions.

[0047] The predicted vehicle state parameters are: based on the vehicle state parameters at the current moment and the additional yaw moment ΔM and active rear wheel steering angle δ of the control input r , according to the linear two-degree-of-freedom model, predict r s2 、r s3 and r s4The vehicle status parameters at the corresponding moment.

[0048] Optionally, the yaw rate error and center of mass sideslip angle error reward r e for:

[0049] r e =-a ω Δω 2 -a β Δβ 2 +W

[0050] Where W is a constant, a ω and a β are the adjustment coefficients of Δω and Δβ, respectively;

[0051] The critical stable state reward r cs for:

[0052]

[0053] Wherein, D1 is the dot product of the vector formed by the vehicle's current state parameters and the parameters of the cluster center to which the vehicle belongs and the vector formed from the second cluster center to the first cluster center; is the length of the vector from the second cluster center to the first cluster center; D2 is the dot product of the vector formed by the vehicle's current state parameters and the cluster center parameters to which the vehicle belongs and the vector formed by the fourth cluster center to the first cluster center, is the length of the vector from the fourth cluster center to the first cluster center; n is the adjustment coefficient of the dot product, k p is the adjustment coefficient of the critical stable state reward;

[0054] The anti-shake reward r c for:

[0055] r c =-Δδ r 2 -ΔM z 2

[0056] Among them, Δδ r is the difference between the current active rear wheel steering angle and the previous moment; ΔM z The difference between the additional yaw moment at the current moment and the previous moment.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] The present invention applies reinforcement learning algorithm to vehicle stability coordinated control, which can achieve accurate tracking of the ideal yaw angular velocity, give full play to the respective advantages of active rear-wheel steering and direct yaw moment control, improve the vehicle stability control margin, greatly improve the utilization rate of tires, and ensure the stability, comfort and control moderation of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a control principle diagram of the automobile stability coordinated control method based on reinforcement learning of the present invention;

[0060] Figure 2 It is a K-means clustering diagram given by the present invention;

[0061] Figure 3 It is a reinforcement learning training result diagram of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0063] It should be noted that the term "comprises" and any variations of the term in the specification and claims of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or apparatus.

[0064] like Figure 1 As shown, a vehicle stability coordination control method based on reinforcement learning is provided, comprising the following steps:

[0065] S1: Obtain the expected yaw rate ω based on the vehicle's structural parameters, vehicle speed and front wheel steering angle using a linear two-degree-of-freedom model d and the desired sideslip angle β d .

[0066] The structural parameters of the vehicle include the stiffness of the front axle of the vehicle, the stiffness of the rear axle of the vehicle, the distance from the center of mass of the vehicle to the front axle, the distance from the center of mass of the vehicle to the rear axle, and the vehicle mass.

[0067] Specifically, the desired vehicle torque is obtained according to the accelerator pedal opening, and the desired yaw rate and the desired sideslip angle of the center of mass are calculated using a linear two-degree-of-freedom model according to the actual vehicle speed and the front wheel turning angle; the accelerator pedal opening is obtained by a sensor.

[0068] The desired torque of the vehicle is T d for:

[0069]

[0070] Where θ is the accelerator pedal opening, T imax is the peak torque of the i-th wheel hub motor.

[0071] The linear two-degree-of-freedom model is in the form of:

[0072]

[0073] Where k1 is the stiffness of the front axle of the vehicle, k2 is the stiffness of the rear axle of the vehicle, β is the side slip angle of the vehicle center of mass, and v x is the longitudinal speed, a is the distance from the vehicle center of mass to the front axle, n is the distance from the vehicle center of mass to the rear axle, ω exp is the expected yaw rate output by the linear two-degree-of-freedom model, δ is the standard front wheel turning angle, m is the vehicle mass, and v y is the lateral speed, a y is the lateral acceleration of the vehicle, Iz is the moment of inertia of the vehicle, ω exp The derivative of .

[0074] The standard front wheel turning angle δ is the average of the wheel turning angles of the two front wheels of the vehicle; the wheel turning angle δ of the two front wheels of the vehicle f and longitudinal speed v x Obtained through sensors; vehicle moment of inertia I z Based on the vehicle's expected torque T d Get.

[0075] The expected yaw rate ω output by the linear two-degree-of-freedom model exp for:

[0076]

[0077] Among them, K is the vehicle stability factor.

[0078] Taking into account the influence of the adhesion coefficient μ, the lateral acceleration of the vehicle during driving Should meet When the vehicle is in a steady state, the sideslip angle β of the center of mass is small, and the lateral acceleration can be expressed as:

[0079]

[0080] Considering the actual driving capability of the electric wheel drive vehicle, a 5% stability margin is processed; and the expected center of mass slip angle of the electric wheel drive vehicle is 0, so the final expected yaw rate and expected center of mass slip angle can be obtained:

[0081]

[0082] Among them, μ is the adhesion coefficient, g is the acceleration of gravity, β d is the expected sideslip angle of the center of mass, sgn(δ) is the judgment function marking the sign of the front wheel turning angle. When the front wheel turning angle is positive, sgn(δ) is 1, and when the front wheel turning angle is negative, sgn(δ) is -1.

[0083] S2: According to the expected yaw rate ω d , actual yaw rate, expected sideslip angle β d The actual center of mass sideslip angle is controlled by sliding mode to obtain the additional yaw moment ΔM and the active rear wheel steering angle δ r .

[0084] Specifically, S2.1: using the expected yaw rate and the actual yaw rate value of the vehicle, design the sliding surface of the yaw moment controller, and then solve the additional yaw moment ΔM that ensures stability.

[0085] S2.2: According to the expected center of mass sideslip angle β d and the actual center of mass sideslip angle, design the sliding surface of the active rear wheel controller, and obtain the active rear wheel steering angle δ r .

[0086] Step S2.1 specifically includes the following steps:

[0087] The vehicle lateral swing dynamic equation with ΔM added is:

[0088]

[0089] Among them, C f is the stiffness of the front wheel of the vehicle, C r is the stiffness of the rear wheels of the vehicle.

[0090] The control amount of the additional yaw moment is determined according to the deviation between the sideslip angle of the center of mass and the yaw angular velocity. The control variable deviation of the sliding mode controller is:

[0091] e ω =(ω-ω d ), e β =(ω-ω β );

[0092] Where, e is the control variable deviation, ω is the actual yaw rate, and β is the actual center of mass sideslip angle;

[0093] The sliding mode switching function is defined as follows:

[0094]

[0095] In the formula, c ω is the relative weight coefficient between error and error change rate, cω >0.

[0096]

[0097] In the formula, s ω is the sliding mode shear function, is the sliding mode index approach rate;

[0098] The sliding mode index approach rate satisfies:

[0099]

[0100] In the formula, K0 is the approach rate constant, which indicates the rate at which the state point of the system approaches the sliding surface. In order to eliminate chattering, the saturation function sat(S) is used instead of the sign function, and the boundary layer thickness and the approach rate index are both positive numbers.

[0101] The saturation function satisfies:

[0102]

[0103] In the formula, k e is the boundary layer thickness, and k e =0.5.

[0104] Then there is

[0105]

[0106] The expected additional yaw moment is calculated by combining the sliding mode exponential approach rate formula:

[0107]

[0108] Among them, I z is the moment of inertia of the vehicle, c ω is the relative weight coefficient between the error and the error change rate, a is the distance from the vehicle center of mass to the front axle, b is the distance from the vehicle center of mass to the rear axle, C f is the stiffness of the front wheel of the vehicle, C r The stiffness of the rear wheels of the vehicle, v x is the longitudinal speed, and e ω and e β The derivative of ω is the control variable deviation of yaw rate, e β is the control variable deviation of the sideslip angle of the center of mass, K0 is the approach rate constant, s ω is the sliding mode shear function.

[0109] When calculating the active rear wheel steering angle δ rWhen adding the rear wheel turning angle δ r The vehicle lateral swing dynamic equation is:

[0110]

[0111] In the formula, δ f is the front wheel turning angle.

[0112] In calculating the active rear wheel steering angle δ r The same sliding mode method is used to calculate the additional yaw moment ΔM. Therefore, the active rear wheel steering angle δ r for:

[0113]

[0114] S3: Based on the established vehicle state parameter cluster diagram under the set working condition and the state parameters of the current vehicle, the state type of the current vehicle is obtained; the state type is one of a stable state, a critical stable state and an unstable state.

[0115] There are a total of 5 cluster centers in the cluster diagram of vehicle state parameters under the set working conditions, among which the middle cluster is a stable state, the clusters on the left and right edges of the middle cluster are unstable states, and the cluster between the stable state and the unstable state is a critical stable state.

[0116] The vehicle state parameters include: lateral velocity, lateral acceleration, yaw rate, center of mass sideslip angle and tire sideslip angle.

[0117] Specifically, the cluster diagram of vehicle state parameters under set working conditions is established, taking into account the vehicle input parameters and various state parameters characterizing the vehicle's lateral stability. A specific working condition is selected in CarSim, and simulation is performed within the vehicle speed range of 10-100 km / h. The obtained state parameter data is clustered offline and divided into 5 categories, of which the first category is stable state, the second and fourth categories are critical stable states, and the third and fifth categories are unstable states. Five cluster centers are obtained by clustering.

[0118] As an alternative, if Figure 2 As shown in the figure, the clustering diagram obtained when the simulation test working condition is the double lane change working condition (DLC), where the green and orange areas are the third and fifth categories of clusters, that is, unstable states; the yellow and purple areas are the second and fourth categories of clusters, respectively, representing critical stable states; the blue area is the first category, that is, stable state. When performing offline clustering, it is necessary to ensure that the working condition settings are the same as those for learning in step S4.

[0119] S4: Construct a reinforcement learning agent and reward function, and continuously train the agent until it outputs the additional yaw torque ΔM and the active rear wheel steering angle δ rThe respective optimal cooperative control coefficients are used to achieve coordinated control of the electric vehicle; the reward function includes a stable state reward, a yaw angular velocity error and a center of mass sideslip angle error reward, a critical stable state reward and an anti-shake reward.

[0120] The reinforcement learning algorithm of this application can be described as including the vehicle state s t , collaborative control action a t and instant rewards t Specifically, the agent makes a decision based on the current vehicle state s t , take an action a t To coordinate the rear wheel angle and add yaw torque, the vehicle is updated to a new state s t+1 According to the vehicle state s at the next moment t+1 , generating an immediate reward r t (s t , a t ). Through iterative training, the agent can obtain the optimal strategy network To get the maximum cumulative return. In summary, the Markov decision process can be expressed as follows: MDP = {S, A, P s , r, λ}, where S is the state space; A is the action space; P s is the state transfer function; r is the reward function; λ is the discount factor to ensure the convergence of the reward function.

[0121] Define the state s of the agent t for:

[0122] s t ={ω,Δω,β,Δβ,a y}

[0123] Where Δω=ω-ω d is the difference between the yaw rate and the actual yaw rate, Δβ=β-β d is the difference between the desired center of mass sideslip angle and the actual center of mass sideslip angle, ω is the actual yaw rate, β is the actual center of mass sideslip angle, a y is the lateral acceleration;

[0124] Define action a t is the coordinated relationship between the rear wheel steering angle and the additional yaw moment, expressed as follows:

[0125] a t ={κ,1-κ},κ∈[0,1]

[0126] Where κ is the active rear wheel steering angle δ r , 1-κ is the coordination coefficient of the additional yaw moment ΔM.

[0127] The corrected rear wheel steering angle and additional yaw moment are as follows:

[0128]

[0129] Reward refers to a value returned by the environment to the agent after the agent performs an action. The reward function serves as a pointer for deep reinforcement learning control optimization. The definition of reward directly affects the effect of reinforcement learning.

[0130] Define the reward function r t for:

[0131] r t =a1r s +a2r e +a3r cs +a4r c

[0132] Among them, r s is the steady state reward, r e is the yaw rate error and center of mass sideslip angle error reward, r cs is the critical stable state reward, r c For anti-shake reward, a1, a2, a3 and a4 are r s 、r e 、r cs and r c The reward factor.

[0133] In order to keep the car in a stable state, i.e. the first class of clustering, the stable state reward r s for:

[0134] r s =r s1 +r s2 +r s3 +r s4

[0135]

[0136]

[0137] Among them, r s1 is the reward for the car’s current state, r s2 、r s3 and r s4 are the state rewards of the car at the first, second and third moments of the prediction respectively; Dis1 is the Euclidean distance from the vehicle state parameter to the cluster center; t1, t2, t3 and t4 are r s1、 r sz 、r s3 and r s4The adjustment coefficient; q is the category of the cluster to which the current vehicle belongs; q=1 means the current vehicle is in a stable state, q=2 or q=4 means the current vehicle is in a critically stable state, and q=3 and q=5 both mean the current vehicle is in an unstable state.

[0138] Steady state reward r s This includes the current state reward of the vehicle and the predicted state reward of the vehicle in the future. Designing the predicted state reward of the vehicle in the future into the reward function can avoid the sudden change of the coordination coefficient caused by the sudden change of the vehicle state parameters at the next moment.

[0139] r s2 、r s3 and r s4 The corresponding clustering categories are obtained based on the predicted vehicle state parameters and the clustering diagram of the vehicle state parameters under the set working conditions;

[0140] The predicted vehicle state parameters are: based on the vehicle state parameters at the current moment and the additional yaw moment ΔM and active rear wheel steering angle δ of the control input r , according to the linear two-degree-of-freedom model, predict r s2 、r s3 and r s4 The vehicle status parameters at the corresponding moment.

[0141] Specifically, the linear two-degree-of-freedom model is transformed into the following form:

[0142]

[0143] Discretize the above form:

[0144]

[0145] Among them, β′ and ω′ are the derivative sideslip angle and yaw rate of the center of mass, respectively.

[0146] Through the discrete form of the linear two-degree-of-freedom model, the vehicle state parameters at the current moment, and the additional yaw moment ΔM and active rear wheel steering angle δ of the control input r , determine the predicted vehicle state parameters for several moments in the future.

[0147] The yaw rate error and the center of mass sideslip angle error are rewarded r e for:

[0148] r e =-a ω Δω 2 -a β Δβ 2 +W

[0149] Among them, aω and a β are the adjustment coefficients of Δω and Δβ respectively; W is a constant, and when Δω 2 When ≤0.01, W=0.5, otherwise W=0.

[0150] When the car is in a critical stable state, i.e., the second or fourth class of clustering, we hope that the car state will be improved from a critical stable state to a stable state. Therefore, we introduce a critical stable state reward r cs for:

[0151]

[0152] Wherein, D1 is the dot product of the vector formed by the vehicle's current state parameters and the parameters of the cluster center to which the vehicle belongs and the vector formed from the second cluster center to the first cluster center; is the length of the vector from the second cluster center to the first cluster center; D2 is the dot product of the vector formed by the vehicle's current state parameters and the cluster center parameters to which the vehicle belongs and the vector formed by the fourth cluster center to the first cluster center, is the length of the vector from the fourth cluster center to the first cluster center; n is the adjustment coefficient of the dot product, k p is the adjustment coefficient of the critical stable state reward;

[0153] In order to ensure ride comfort during the control process, the change in the rear wheel angle and additional yaw moment between the previous moment and the current moment should not be too large. The smaller the action value, the smaller the vehicle's shaking degree. c for:

[0154] r c =-Δδ r 2 -ΔM z 2

[0155] Among them, Δδ r is the difference between the current active rear wheel steering angle and the previous moment; ΔM z The difference between the additional yaw moment at the current moment and the previous moment.

[0156] As an alternative, if Figure 3 As shown in the figure, the curve of 100 trainings under the DLC condition with a vehicle speed of 90 km / h and an adhesion coefficient of 0.5. The light blue line represents the reward value obtained at each step of learning, and the dark blue line represents the average reward. Figure 3 It can be seen that the average reward of this application is stable and the agent remains stable during the training process.

[0157] The active rear wheel steering controller is based on the active rear wheel steering angle δ output in step S2. rand the active rear wheel steering angle δ output in step S4 r The optimal cooperative control coefficient is used to actively control the rear wheel steering of the vehicle; the direct yaw torque controller distributes the torque according to the additional yaw torque ΔM output in step S2 and the optimal cooperative control coefficient of the additional yaw torque ΔM output in step S4. The torque distribution method refers to the prior art, so as to obtain the driving or braking torque of the four wheels to control the four wheels separately.

[0158] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0159] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A vehicle stability coordination control method based on reinforcement learning, characterized in that: include: According to the vehicle's structural parameters, vehicle speed and front wheel angle, a linear two-degree-of-freedom model is used to obtain the expected yaw rate ω d and the desired sideslip angle β d ; According to the expected yaw rate ω d , actual yaw rate, expected sideslip angle β d The actual center of mass sideslip angle is controlled by sliding mode to obtain the additional yaw moment ΔM and the active rear wheel steering angle δ r ; Based on the established vehicle state parameter cluster diagram under the set working condition and the state parameters of the current vehicle, the state type of the current vehicle is obtained; the state type is one of a stable state, a critical stable state and an unstable state; Construct a reinforcement learning agent and reward function, and continuously train the agent until it outputs the additional yaw torque ΔM and the active rear wheel steering angle δ r The respective optimal cooperative control coefficients are used to achieve coordinated control of the electric vehicle; the reward function includes a stable state reward, a yaw angular velocity error and a center of mass sideslip angle error reward, a critical stable state reward and an anti-shake reward.

2. The vehicle stability coordinated control method based on reinforcement learning according to claim 1 is characterized in that: The vehicle's structural parameters, vehicle speed and front wheel steering angle are used to obtain the expected yaw rate ω using a linear two-degree-of-freedom model. d and the desired sideslip angle β d middle: The linear two-degree-of-freedom model is in the form of: Among them, k1 is the stiffness of the front axle of the vehicle, k2 is the stiffness of the rear axle of the vehicle, β is the side slip angle of the vehicle center of mass, and v x is the longitudinal speed, a is the distance from the vehicle center of mass to the front axle, b is the distance from the vehicle center of mass to the rear axle, ω exp is the expected yaw rate output by the linear two-degree-of-freedom model, δ is the standard front wheel turning angle, m is the vehicle mass, and a y is the lateral acceleration of the vehicle, i z is the vehicle's moment of inertia, ω exp The derivative of The desired yaw rate ω output by the linear two-degree-of-freedom model is exp for: Among them, K is the vehicle stability factor; The desired yaw rate ω d and the desired sideslip angle β d for: Among them, μ is the adhesion coefficient, g is the acceleration of gravity, β d is the expected sideslip angle of the center of mass, sgn(δ) is the judgment function marking the sign of the front wheel turning angle, when the front wheel turning angle is positive, sgn(δ) is 1, when the front wheel turning angle is negative, sgn(δ) is -1.

3. The vehicle stability coordinated control method based on reinforcement learning according to claim 2 is characterized in that: The standard front wheel turning angle δ is the average of the wheel turning angles of the two front wheels of the vehicle; the wheel turning angles of the two front wheels of the vehicle and the longitudinal vehicle speed v x The vehicle's rotational inertia I is obtained by a sensor; z Based on the vehicle's expected torque T d Obtain the desired torque T of the vehicle d for: Where θ is the accelerator pedal opening, T imax is the peak torque of the i-th wheel hub motor.

4. The vehicle stability coordination control method based on reinforcement learning according to claim 1 is characterized in that: According to the desired yaw rate ω d , actual yaw rate, expected sideslip angle β d The actual center of mass sideslip angle is controlled by sliding mode to obtain the additional yaw moment ΔM and the active rear wheel steering angle δ r , including: According to the expected yaw rate ω d and the actual yaw angular velocity, design the sliding surface of the yaw moment controller, and obtain the additional yaw moment ΔM; According to the expected sideslip angle β d and the actual sideslip angle of the center of mass, design the sliding surface of the active rear wheel controller, and obtain the active rear wheel steering angle δ r ; Among them, I z is the moment of inertia of the vehicle, c ω is the relative weight coefficient between the error and the error change rate, a is the distance from the vehicle center of mass to the front axle, b is the distance from the vehicle center of mass to the rear axle, C f is the stiffness of the front wheel of the vehicle, C r The stiffness of the rear wheels of the vehicle, v x is the longitudinal speed, and e ω and e β The derivative of ω is the control variable deviation of yaw rate, e β is the control variable deviation of the sideslip angle of the center of mass, K0 is the approach rate constant, s ω is the sliding mode shear function.

5. The vehicle stability coordinated control method based on reinforcement learning according to claim 1 is characterized in that: In the step of obtaining the state type of the current vehicle based on the established cluster diagram of vehicle state parameters under the set working condition and the state parameters of the current vehicle, There are 5 cluster centers in the cluster diagram of vehicle state parameters under the set working conditions, among which the middle cluster is in a stable state, the clusters on the left and right edges of the middle cluster are in an unstable state, and the cluster between the stable state and the unstable state is in a critical stable state; The vehicle state parameters include: lateral velocity, lateral acceleration, yaw rate, center of mass sideslip angle and tire sideslip angle.

6. The vehicle stability coordinated control method based on reinforcement learning according to claim 1 is characterized in that: The intelligent agent and reward function of reinforcement learning are constructed, and the intelligent agent is continuously trained until the additional yaw moment ΔM and the active rear wheel steering angle δ are output. r The optimal coordinated control coefficients of each include: Define the state s of the agent t for: s t ={ω,Δω,β,Δβ,a y } Where Δω is the difference between the yaw rate and the actual yaw rate, Δβ is the difference between the desired center of mass sideslip angle and the actual center of mass sideslip angle, ω is the actual yaw rate, β is the actual center of mass sideslip angle, and a y is the lateral acceleration; Define action a t is the coordinated relationship between the rear wheel steering angle and the additional yaw moment, expressed as follows: a t ={κ,1-k},κ∈[0,1] Where κ is the active rear wheel steering angle δ r The coordinated control coefficient is , 1-k is the coordinated control coefficient of the additional yaw moment ΔM; Define the reward function r t for: r t =a1r s +a2r e +a3r cs +a4r c Among them, r s is the steady state reward, r e is the yaw rate error and center of mass sideslip angle error reward, r cs is the critical stable state reward, r c For anti-shake reward, a1, a2, a3 and a4 are r s 、r e 、r cs and r c The reward factor.

7. The vehicle stability coordinated control method based on reinforcement learning according to claim 6 is characterized in that: The steady-state reward r s It includes the current state reward of the vehicle and the predicted state reward of the vehicle in the future. The specific formula is: r s ＝r s1 +r s2 +r s3 +r s4 Among them, r s1 is the reward for the car’s current state, r s2 、r s3 and r s4 are the state rewards of the car at the first, second and third moments of the prediction respectively; Dis1 is the Euclidean distance from the vehicle state parameter to the cluster center; t1, t2, t3 and t4 are r s1 、r s2 、r s3 and r s4 The adjustment coefficient; q is the category of the cluster to which the current vehicle belongs; q = 1 means the current vehicle is in a stable state, q = 2 or q = 4 means the current vehicle is in a critically stable state, and q = 3 and q = 5 both mean the current vehicle is in an unstable state; r s2 、r s3 and r s4 The corresponding clustering categories are obtained based on the predicted vehicle state parameters and the clustering diagram of the vehicle state parameters under the set working conditions; The predicted vehicle state parameters are: based on the vehicle state parameters at the current moment and the additional yaw moment ΔM and active rear wheel steering angle δ of the control input r , according to the linear two-degree-of-freedom model, predict r s2 、r s3 and r s4 The vehicle status parameters at the corresponding moment.

8. The vehicle stability coordinated control method based on reinforcement learning according to claim 6 is characterized in that: The yaw rate error and the center of mass sideslip angle error reward r e for: r e =-a ω Give 2 -a β Db 2 +W Where W is a constant, a ω and a β are the adjustment coefficients of Δω and Δβ, respectively; The critical stable state reward r cs for: Where D1 is the dot product of the vector formed by the vehicle's current state parameters and the parameters of the cluster center to which the vehicle belongs and the vector formed from the second cluster center to the first cluster center; i 1length is the length of the vector from the second cluster center to the first cluster center; D2 is the dot product of the vector formed by the vehicle's current state parameters and the cluster center parameters to which the vehicle belongs and the vector formed by the fourth cluster center to the first cluster center, i 2length is the length of the vector from the fourth cluster center to the first cluster center; n is the adjustment coefficient of the dot product, k p is the adjustment coefficient of the critical stable state reward; The anti-shake reward r c for: r c =-Dδ r 2 -DM z 2 Among them, Δδ r is the difference between the current active rear wheel steering angle and the previous moment; ΔM z The difference between the additional yaw moment at the current moment and the previous moment.

Images