AFS, ARS and DYC cooperative control method for distributed electric drive vehicle based on DDPG

Abstract

The application discloses an AFS, ARS and DYC cooperative control method of a distributed electric drive vehicle based on DDPG. The method utilizes a cooperative control method and a deep reinforcement learning method, designs and develops a vehicle stability control method based on a centroid side slip angle and a yaw angular velocity, takes driving signals provided by a driver and measured or estimated vehicle state information as input, and judges the stability of the vehicle. When the active safety system needs to intervene, a DDPG-AAD model trained offline by a strategy deep gradient algorithm is used to coordinately control AFS, ARS and DYC systems, precise and independent control of each wheel is realized, and thus the active safety driving stability of the vehicle is improved.

Description

Technical Field

[0001] This invention relates to active safety control of distributed electric drive vehicles, and more particularly, to a method for collaborative active safety control of AFS, ARS and DYC in distributed electric drive vehicles based on the DDPG algorithm. Background Technology

[0002] With the increasing use of distributed electric vehicles (DEVs), vehicle stability control and active safety have received growing attention. In active safety control, vehicle active safety systems adjust the vehicle's operating state to maintain a relatively safe condition as much as possible at all times. In vehicle stability control, the stability and reliability of the stability control system directly affect the vehicle's maneuverability, handling, and driver safety. Therefore, research on the control performance of vehicle stability control systems is of great significance for improving vehicle performance.

[0003] Active safety control systems in vehicles typically include at least three subsystems: Active Front Steering (AFS), Active Rear Steering (ARS), and Direct Yaw Control (DYC). AFS and ARS improve vehicle yaw dynamics by adjusting the input steering angles of the front and rear wheels, thereby enhancing vehicle stability. They offer advantages such as fast, direct, and efficient control response. However, when a vehicle is under extreme conditions, lateral forces tend to saturate, limiting the effectiveness of AFS and ARS. In such cases, DYC is needed to individually control the longitudinal forces of each wheel, improving vehicle stability and handling under extreme conditions. DYC demonstrates good performance in both the linear and nonlinear regions of vehicle dynamics.

[0004] However, excessive use of the DYC subsystem can lead to reduced vehicle maneuverability and should be limited. An effective way to limit the frequent use of the DYC subsystem while ensuring vehicle stability is to integrate AFS, ARS, and DYC control, while controlling each wheel individually.

[0005] Currently, vehicle stability control has attracted research and attention from scholars both domestically and internationally, and has achieved considerable results and progress. However, the following problems still exist: First, with the development of transportation technology, the traffic environment is becoming increasingly complex, and the uncertainties in environmental models are becoming more and more complex and variable. At present, most domestic and international research is based on rule-based and model-integrated control methods, which heavily rely on accurate models or experience. When there are many uncertainties in the external environment, the reliability and stability of the control are difficult to guarantee. Second, current domestic and international research on AFS, ARS, and DYC systems mainly focuses on the whole vehicle level, lacking precise control of the rotation angle and force of each wheel through system integration.

[0006] In order to improve the performance of integrated control, model-free reinforcement learning control methods have been used by more and more scholars in recent years. Through model-free reinforcement learning, relatively accurate control of each wheel can be achieved when the model is inaccurate or there is no model.

[0007] Therefore, designing a method to improve the reliability and stability of the control system under numerous uncertainties and to achieve precise control of each wheel has always been a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0008] This invention proposes a cooperative control method for AFS, ARS, and DYC of distributed electric drive vehicles based on DDPG. It utilizes cooperative regulation methods and deep reinforcement learning to design and develop a vehicle stability control method based on the center of gravity sideslip angle and yaw rate. Taking the driving signals provided by the driver and the measured or estimated vehicle state information as input, the stability of the vehicle is judged. When intervention is required, the policy deep gradient algorithm is used for offline learning to coordinate the control of the AFS, ARS, and DYC systems, achieving precise and independent control of each wheel, and ultimately realizing the integrated stability control of distributed electric drive vehicles.

[0009] One of the objectives of this invention is to address vehicle data that is difficult to measure directly by introducing a fading factor based on the Extended Kalman Filter (EKF) and designing a fading extended Kalman Filter observer to accurately estimate vehicle data.

[0010] The second objective of this invention is to design a vehicle stability judgment module based on data provided by sensors or observers. The vehicle stability judgment module is used to judge vehicle stability by combining the dual-line method and the yaw rate threshold method. When intervention is required, the data provided by the sensors or observers is transmitted to the coordination controller of the AFS, ARS and DYC systems.

[0011] The third objective of this invention is to design the DDPG-AAD model. For the coordinated control of AFS, ARS and DYC, a vehicle dynamics model with AFS, ARS and DYC system is established. Considering vehicle stability and stability margin, and combined with the longitudinal force requirements of MPC speed tracking, the driving force or braking force of each wheel and the steering angle are controlled by the model trained by DDPG (i.e. DDPG-AAD model).

[0012] Based on the current state of vehicle chassis research both domestically and internationally, this paper elucidates the research on coordinated control of ARS, AFS, and DYC systems in distributed electric drive vehicles. It introduces vehicle and wheel dynamics models and commonly used nonlinear tire models for distributed electric vehicles, as well as the theory and methods of coordinated control of vehicle subsystems. The AFS, ARS, and DYC coordinated control method for distributed electric drive vehicles based on DDPG in this invention includes the following steps:

[0013] Step 1: Establish a dynamic model of a typical vehicle. The sensor array collects driving signals and vehicle status signals, and transmits the vehicle signals to the fading EKF observer.

[0014] Step 2: Based on the dynamic model of a conventional vehicle, establish the differential equations of a four-wheel independent steering vehicle. The fading EKF observer receives driving signals and vehicle state signals on this basis and estimates the center of gravity sideslip angle and yaw rate.

[0015] Step 3: Calculate the desired yaw rate and center of mass sideslip angle based on the linear two-degree-of-freedom model;

[0016] Step 4: The vehicle stability judgment module judges the vehicle's stability state based on the measured value, nominal value, and expected value, while also considering the vehicle's yaw rate and center of gravity sideslip angle, by combining the yaw rate threshold method and the dual-line method.

[0017] Step 5: Establish vehicle state equations with AFS, ARS, and DYC systems to provide a basis for the construction of state space, action space, and reward function in subsequent deep reinforcement learning.

[0018] Considering that the AFS and ARS systems input different steering angles for each wheel, calculate the vehicle's longitudinal force requirements, tracking error, and vehicle stability margin;

[0019] Step 6: Determine the control objective of DDPG and construct the state space s t Action space a t Train the policy offline with a reward function r, specify the stopping condition, and output the optimal policy.

[0020] Compared with the prior art, the technical advantages of the method of the present invention are as follows:

[0021] ① By designing a fading extended Kalman yaw rate and center-of-gravity sideslip angle observer, accurate observation of vehicle data was achieved. Compared with existing methods, this not only improves the observation capability, but more importantly, it makes the system more adaptable and robust. When the system changes, the fading EKF can still obtain relatively accurate observation values;

[0022] ② Based on the combined determination of vehicle status using the dual-line method and the yaw rate threshold method, compared with existing methods, it simultaneously considers the influence of vehicle center of gravity sideslip angle and yaw rate on vehicle stability.

[0023] ③ Combining the dynamic model of distributed electric drive vehicles, the state space, action space, and reward function of the DDPG algorithm are designed, and the DDPG-AAD model is constructed to realize the coordinated control of AFS, ARS, and DYC systems. Compared with existing methods, this method has better generalization ability and model-free learning ability, and can handle complex environments better.

[0024] ④ Compared with other methods, the method of constructing the DDPG-AAD model has better control accuracy, better coordinated control effect of AFS, ARS and DYC systems, and is easier to ensure the stability of distributed electric drive vehicles.

[0025] ⑤ The entire system design achieves coherent and complete coverage in data observation, vehicle stability assessment, and vehicle control. This ensures the system's comprehensiveness and consistency in overall operation, providing a reliable solution for practical applications; Attached Figure Description

[0026] Figure 1 This is a control system block diagram of a distributed electric drive vehicle based on DDPG, which is a collaborative control method of AFS, ARS and DYC.

[0027] Figure 2 This is a diagram showing the wheel system distribution of a distributed electric drive vehicle.

[0028] Figure 3 This is a system block diagram of the fading EKF observer constructed according to the present invention.

[0029] Figure 4 This is a system block diagram for constructing the DDPG-AAD model according to the present invention.

[0030] Figure 5 This is a control block diagram of the AFS, ARS, and DYC systems using the method of this invention.

[0031] Figure 6 It is the yaw rate error curve obtained by applying the method of this invention to a serpentine steering maneuver test, with the speed gradually increasing from 40km / h to 100km / h.

[0032] Figure 7 It is the center of gravity sideslip angle error curve under serpentine steering maneuvering conditions, where the speed is gradually increased from 40km / h to 100km / h, using the method of this invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0034] Conversely, this invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the invention as defined in the claims. Furthermore, to provide a better understanding of the invention, certain specific details are described in detail below. However, those skilled in the art will fully understand the invention even without these detailed descriptions.

[0035] The technical concept of this invention is as follows: First, a dynamic model of a distributed electric drive vehicle system is established. In the upper-level controller, parameters such as centroid sideslip and yaw rate are estimated through a fading EKF observer. The desired centroid sideslip angle and yaw rate are determined through the dynamic model. Then, the vehicle state is jointly determined by the dual-line method and the yaw rate threshold method in the middle-level controller. When the AFS, ARS, and DYC systems need to intervene, the lower-level controller coordinates and controls the AFS, ARS, and DYC systems through the optimal strategy trained offline by the DDPG algorithm.

[0036] Constructing a dynamic model for distributed electric drive vehicles

[0037] Based on the "Simulation Study of Active Control of Vehicle Handling Stability" published in November 2019, authored by Liu Chunhui, Chapter 2, which discloses the vehicle coordinate system, tire coordinate system, three-degree-of-freedom dynamic model, and tire model in the establishment of the vehicle dynamics model, this invention constructs a dynamic model. and tire models It is obtained by improving the method of this invention.

[0038] The dynamic model designed in this invention Hehe Tire Model There are three coordinate systems involved, such as Figure 2 As shown. The geodetic coordinate system is denoted as XOY, which follows the right-hand rule. The vehicle coordinate system is denoted as xoy, with the x-axis along the longitudinal direction of the vehicle, the y-axis along the lateral direction of the vehicle, and the z-axis perpendicular to the xy plane, also following the right-hand rule. The tire coordinate system is denoted as x... T o T y T x T The axis is along the radial direction of the tire, y T It is along the tire axis direction, z T The axis is perpendicular to x. T y T The orientation of the plane satisfies the right-hand rule. For example... Figure 2The physical meanings of the letters in the three coordinate systems shown are as follows: m is the vehicle mass; g is the acceleration due to gravity; h is the distance from the center of mass to the roll axis, also known as the center of mass height; φ is the vehicle roll angle. s This represents the distance from the center of gravity to the axial direction of the left and right tires. f This is the distance from the center of mass to the front axle. r θ is the distance from the center of mass to the rear axle. β is the vehicle's heading angle. α is the rear wheel steering angle. δ is the front wheel steering angle. v is the vehicle's speed. x This represents the longitudinal speed of the vehicle. y F represents the vehicle's lateral speed. r represents the vehicle's yaw rate. xfl F represents the longitudinal force on the front left tire. yfl F represents the lateral force exerted by the front left tire. xfr F represents the longitudinal force on the front right tire. yfr F represents the lateral force exerted by the front right tire. xrl F represents the longitudinal force on the rear left tire. yrl F represents the lateral force exerted by the rear left tire. xrr F represents the longitudinal force on the rear right tire. yrr This refers to the lateral force on the rear right tire.

[0039] In autonomous and safe driving, Distributed Electric Vehicles (DEVs) typically require steering geometry models to describe the geometric relationship between the vehicle and the path during steering. However, traditional dynamic models lack crucial information such as vehicle speed and acceleration, limiting their applicability to low-speed and continuous road curvature conditions. Vehicle kinematic models, based on vehicle kinematic relationships, primarily reflect the relationship between vehicle position, speed, and acceleration, but they also do not address vehicle forces. When a vehicle is under extreme curvature handling conditions, its dynamics exhibit nonlinearity, strong coupling, and parameter uncertainty. Therefore, establishing a complete vehicle model capable of describing the nonlinear dynamic characteristics of DEVs is a critical technology for active safety control of DEVs.

[0040] To achieve motion control of the entire DEV vehicle under unstable conditions, the present invention designs a vehicle dynamics model. The description is as follows:

[0041]

[0042]

[0043]

[0044] This represents the total longitudinal forces on the vehicle.

[0045] This represents the total lateral forces acting on the vehicle.

[0046] This represents the total torque of the vehicle around the z-axis.

[0047] m represents the vehicle's mass.

[0048] v x This refers to the longitudinal speed of the vehicle.

[0049] This refers to the vehicle's longitudinal acceleration.

[0050] v y This refers to the vehicle's lateral speed.

[0051] This refers to the vehicle's lateral acceleration.

[0052] ω r Let yaw rate be the vehicle's angular velocity.

[0053] Let yaw acceleration be the acceleration of the vehicle.

[0054] δ is the steering angle of the front wheel tires.

[0055] I z Let be the yaw moment of inertia of the vehicle about the z-axis in the vehicle coordinate system xoy.

[0056] l f This is the distance from the front axle to the center of mass.

[0057] l r This is the distance from the rear axle to the center of mass.

[0058] t f This is the track width of the front axle.

[0059] t r This is the track width of the rear axle.

[0060] ΔM is the additional yaw torque.

[0061] The vehicle dynamics model designed in this invention Active safety control of vehicles is crucial. In the model... The design process fully considers longitudinal and lateral acceleration, as well as additional yaw torque, enabling the DEV's vehicle stability control system (ESP) to more accurately understand the vehicle's state and facilitate precise drive control of the entire vehicle.

[0062] Constructing a tire model for a distributed electric drive vehicle

[0063] In the tire system of a Distributed Electric Vehicle (DEV), the tire control of the entire DEV is accomplished through driving signals output from the steering wheel and vehicle status signals output from the vehicle sensor group.

[0064] To achieve precise control of the driving status of a DEV vehicle under unstable conditions and different road conditions, this invention is based on... The designed tire model Described as:

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073] F x =μDsin{Carctan[Bλ-E(Bλ-arctan(Bλ))]} (12)

[0074] F y =μDsin{Carctan[Bξ-E(Bξ-arctan(Bξ))]} (13)

[0075] This is the torque of the front left wheel.

[0076] This is the torque of the front right wheel.

[0077] This is the torque of the rear left wheel.

[0078] This is the torque of the rear right wheel.

[0079] F zfl The vertical force is the force on the front left wheel.

[0080] F zfr The vertical force is applied to the front right wheel.

[0081] F zrlThe vertical force is for the rear left wheel.

[0082] F zrr This is the vertical force on the rear right wheel.

[0083] F x This refers to the longitudinal force of the tire.

[0084] F y This refers to the lateral force of the tire.

[0085] I w This is the moment of inertia of the tire.

[0086] ω fl This is the rotational speed of the front left wheel.

[0087] This is the acceleration of the front left wheel.

[0088] ω fr This is the rotational speed of the front right wheel.

[0089] This is the acceleration of the front right wheel.

[0090] ω rl This refers to the rotational speed of the rear left wheel.

[0091] This is the acceleration of the rear left wheel.

[0092] ω rr This refers to the rotational speed of the rear right wheel.

[0093] This is the acceleration of the rear right wheel.

[0094] T qfl This is the driving torque of the front left wheel.

[0095] T sfl This is the braking torque of the front left wheel.

[0096] T qfr This is the driving torque of the front right wheel.

[0097] T sfr This is the braking torque of the front right wheel.

[0098] T qrl This is the driving torque of the rear left wheel.

[0099] T srl This is the braking torque of the rear left wheel.

[0100] T qrr This is the driving torque for the rear right wheel.

[0101] T srr This is the braking torque of the rear right wheel.

[0102] R is the wheel radius.

[0103] m represents the vehicle's mass.

[0104] g is the acceleration due to gravity, taken as 9.8 m / s². 2 .

[0105] This refers to the vehicle's longitudinal acceleration.

[0106] This refers to the vehicle's lateral acceleration.

[0107] μ is the road surface adhesion coefficient.

[0108] D is the peak factor in the tire magic formula.

[0109] C is the shape factor in the tire magic formula.

[0110] B is the stiffness factor in the tire magic formula.

[0111] λ is the tire slip ratio.

[0112] E is the curvature factor in the tire magic formula.

[0113] ξ is the tire slip angle.

[0114] The tire model constructed in this invention fully considers the complex interaction between the tire and the road surface, including important factors such as friction and lateral forces. Through the design of this precise tire model, the DEV's vehicle stability control system (ESP) can more accurately predict the vehicle's behavior under different road conditions. The accuracy of tire modeling directly affects the effectiveness of vehicle control; therefore, precise tire modeling is crucial in vehicle stability control and active safety systems.

[0115] Part 1: Constructing a Gradually Decreasing EKF Observer

[0116] To address the challenges of directly measuring vehicle data in DEVs (Dual Electric Vehicles) or the high cost of direct measurement equipment, this invention designs a fading EKF observer. This invention uses the fading EKF observer to accurately estimate vehicle data and outputs the optimal solution, which is the fading factor λ(k) at time k. λ(k) is calculated using the Extended Kalman Filter (EKF) method.

[0117] For extended Kalman filtering, refer to "Methods for Computing Manifolds of Nonlinear Systems and Nonlinear Signal Processing", published in August 2018, pp. 125-126, by Jia Meng.

[0118] In DEV, the fading EKF observer estimates the yaw rate and centroid sideslip angle of the DEC based on the acceleration signal.

[0119] The extended Kalman filter model [X(k+1), Z(k)] based on the DEV vehicle motion design is described in this invention as follows:

[0120] X(k)=[β EKF ω EKF v x v y ] T (14)

[0121] Z(k)=h[X(k),u(k)]+ε(k)=[v x v y ] T (15)

[0122] X(k+1)=g[X(k),u(k)]+w(k) (16)

[0123]

[0124]

[0125] p(k∣k-1)=λ(k)[Φ(k)×p(k-1)×Φ T [(k)+Q] (19)

[0126]

[0127] C(k)={c(k)[Φ(k)×p(k)×Φ(k) T +Q]c(k T}+R″ (21)

[0128] K(k)=p(k∣k-1)×c T (k)[c(k)×p(k∣k-1)×c T (k)+R] -1 (twenty two)

[0129]

[0130] p(k)=[I n -K(k)c(k)]×p(k∣k-1) (24)

[0131] X(k) represents the vehicle state at time k.

[0132] Z(k) is the measurement variable of the vehicle state at time k.

[0133] β EKFThe centroid sideslip angle is the result of filtering by the fading EKF observer.

[0134] ω EKF The yaw rate is the yaw rate after filtering by the fading EKF observer.

[0135] v x This represents the longitudinal speed of the vehicle.

[0136] v y This represents the lateral speed of the vehicle.

[0137] The superscript T represents the coordinate transpose symbol.

[0138] X(k+1) represents the vehicle state at time k+1.

[0139] This represents the vehicle state updated at time k+1.

[0140] h[X(k),u(k)] is the filter output equation under vehicle state X(k) and estimate u(k).

[0141] g[X(k),u(k)] is the filtered state equation under the vehicle state X(k) and the estimator u(k).

[0142] u(k) is the estimator at time k.

[0143] w(k) represents the filtering process noise, where the process noise covariance is assumed to be denoted as Q. In this invention, Q is set to be diag(1×10). -5 1×10 -5 1×10 -4 1×10 -4 ).

[0144] ε(k) represents the filtered measurement noise, where the measurement noise covariance is assumed to be denoted as R″. In this invention, R″ is set to = diag(1×10⁻⁶). -4 1×10 -4 ).

[0145] Φ(k) is the Jacobian matrix of g(X(k),u(k)) with respect to the state variables.

[0146] Φ(k|k-1) is the Jacobian matrix at time k predicted by time k-1.

[0147] p(k-1) is the covariance matrix at time k-1.

[0148] p(k|k-1) is the covariance matrix at time k predicted by time k-1.

[0149] c(k) is the Jacobian matrix of h(X(k),u(k)) with respect to the state variables.

[0150] C(k) is the estimated covariance matrix.

[0151] K(k) is the Kalman filter gain.

[0152] λ(k) is the fading factor at time k.

[0153] tr is the symbol for trace operation.

[0154] p(k) is a substitution symbol, and p(k) = [I n - K(k)c(k)] × p(k|k - 1), where I n is the identity matrix.

[0155] In the present invention, as shown in Figure 3 the steps to construct the fading EKF observer are as follows:

[0156] Step A, initialize the fading EKF observer; receive the driving signal and vehicle state signal collected by the sensor; execute Step B;

[0157] Step B, based on the vehicle dynamics model and according to the yaw rate ω r , the sideslip angle β of the center of mass EKF , the lateral velocity v y and the longitudinal velocity v x perform digital differential equation processing, and the differential result is used for vehicle motion control; execute Step C;

[0158] In the differential equation where, where, is the vehicle yaw angular acceleration, δ fl is the tire steering angle of the front left wheel, δ fr is the tire steering angle of the front right wheel, δ rl is the tire steering angle of the rear left wheel, δ rr is the tire steering angle of the rear right wheel, t f is the wheelbase of the front axle, I z is the yaw moment of inertia of the vehicle about the z - axis.

[0159] In the differential equation where, ω EKF is the yaw rate filtered by the fading EKF observer.

[0160] In the differential equation where,

[0161] In the differential equation where,

[0162] Step C involves discretizing the vehicle differential equations. A section of the Euler method is used to discretize the continuous vehicle state equations, resulting in the discrete-vehicle differential equations [ω]. r (k+1),β EKF (k+1),v x (k+1),v y [(k+1)], execute step D;

[0163] In the discrete-vehicle differential equation [ω r (k+1),β EKF (k+1),v x (k+1),v y In (k+1)], the ω r (k+1) represents the vehicle's yaw rate at time k+1, ω r Let k be the yaw rate of the vehicle at time k. Let Δt be the yaw acceleration of the vehicle, and Δt be the time step.

[0164] In the discrete-vehicle differential equation [ω r (k+1),β EKF (k+1),v x (k+1),v y In [(k+1)], the β EKF (k+1)=β EKF (k)+(β EKF ×Δt), β EKF (k+1) is the centroid sideslip angle at time k+1, β EKF (k) is the centroid sideslip angle at time k.

[0165] In the discrete-vehicle differential equation [ω r (k+1),β EKF (k+1),v x (k+1),v y In [k+1], the ν x (k+1)=ν x (k)+(v x +ν y ω r )Δt, v x (k+1) represents the longitudinal velocity of the vehicle at time k+1, v x (k) represents the longitudinal velocity of the vehicle at time k.

[0166] In the discrete-vehicle differential equation [ω r (k+1),β EKF (k+1),v x(k+1),v y In [k+1], the ν y (k+1)=ν y (k)+(v y -ν x ω r )Δt, v y (k+1) represents the lateral velocity of the vehicle at time k+1, v y (k) represents the lateral velocity of the vehicle at time k.

[0167] Step D: Use the vehicle turning angle information as the adjustment parameter for the fading EKF observer; execute step D.

[0168] Step E: Establish the state prediction equation and observation prediction equation of the fading EKF observer based on formulas (14), (15), and (16), and then execute step F;

[0169] The state variable is X(k), and the measurement variable is Z(k). The state equation g[X(k),u(k)] and the measurement output equation h[X(k),u(k)] of the DEV vehicle motion can be estimated through the state variable and the measurement variable.

[0170] Step F involves local linearization in the fading EKF observer based on Φ(k) and c(k), and around the estimated value. Linearize Φ(k) and c(k); execute step G;

[0171] Step G: Introduce the fading factor λ(k) to predict the vehicle motion covariance;

[0172] Among them, the covariance prediction matrix of the fading EKF observer after introducing the fading factor λ(k) is p(k|k-1). When λ(k)≥1, Q(k-1) is the covariance matrix of the noise, and step H is executed.

[0173] Step H: Calculate the Kalman filter gain K(k); then proceed to step I.

[0174] Step I: The vehicle state is updated according to formula (23), and the vehicle motion covariance is updated according to formula (24). If the updated state does not satisfy the optimal solution output of the fading EKF observer, the process returns to step D for the next round of adjustment until the optimal solution is output. This concludes the estimation of the fading EKF observer. The optimal solution is represented by the yaw rate ω. r And the centroid side slip angle β EKF .

[0175] Part Two: Constructing the DDPG-AAD Model

[0176] In this invention, the Deep Deterministic Policy Gradient (DDPG) algorithm from deep reinforcement learning is used to train the coordinated control of the AFS, ARS, and DYC subsystems. The trained coordinated controller of the AFS, ARS, and DYC subsystems, applied to vehicle control, is called the active cooperative safety control model, namely the DDPG-AAD model. When the vehicle stability assessment module determines that the vehicle needs intervention, the DDPG-AAD model is used to control the vehicle.

[0177] The DDPG algorithm is based on the paper "CONTINUOUS CONTROL WITH DEEPREINFORCEMENT LEARNING" published at the 2016 ICLR conference, authored by Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, and others.

[0178] The DDPG-AAD model of this invention is based on the sideslip angle and yaw rate of the center of gravity of the distributed electric drive vehicle to jointly complete the coordinated control of the AFS subsystem, ARS subsystem and DYC subsystem.

[0179] The DDPG-AAD model designed in this invention is described as follows:

[0180]

[0181]

[0182]

[0183]

[0184]

[0185] U=[Δδ fl ,Δδ fr ,α rl ,α rr ,F xfr ,F xfl ,F xrr ,F xrl ] T (30)

[0186]

[0187] F desl =F xfl +F xfr +F xrl +F xrr(32)

[0188] e β =b EKF -b d (33)

[0189] e ω =ω EKF -oh rd (34)

[0190] e=p1(β EKF -b d )+p2(ω EKF -oh rd ) (35)

[0191]

[0192]

[0193] r=r ad +r torque

[0194]

[0195]

[0196]

[0197] s tar_i =r i +γQ′(s i+1 ,ψ'(s i+1 |θ ψ ')i Q′ ) (40)

[0198] Y pre_i =Q(s i ,a i ∣θ Q ) (41)

[0199]

[0200]

[0201]

[0202] i Q' ←tth Q' +(1-τ)θ Q' (45)

[0203] i ψ' ←tth ψ +(1-τ)θ ψ'(46)

[0204] m represents the vehicle's mass.

[0205] ν x This represents the longitudinal speed of the vehicle.

[0206] This is the sideslip angle acceleration of the vehicle's center of gravity.

[0207] Let x be the yaw acceleration of the vehicle.

[0208] k f This is the equivalent lateral stiffness of the vehicle's front wheels.

[0209] k r This is the equivalent lateral stiffness of the vehicle's rear wheels.

[0210] l f This is the distance from the center of mass to the front axle.

[0211] l r This is the distance from the center of mass to the rear axle.

[0212] δ f The steering angle of the front wheels as controlled by the driver.

[0213] Δδ fl This is the adjustment angle for the front left wheel steering angle by the AFS system.

[0214] Δδ fr This is the adjustment angle for the front right wheel steering angle by the AFS system.

[0215] α rl This is the input steering angle for the rear left wheel in the ARS system.

[0216] α rr This is the input steering angle for the rear right wheel in the ARS system.

[0217] I z Let be the moment of inertia of the vehicle about the z-axis.

[0218] t f This refers to the front axle track.

[0219] t r This refers to the rear axle track.

[0220] It is the first derivative of the state variable with respect to time.

[0221] X is a state variable.

[0222] A is the state matrix.

[0223] B is the input matrix.

[0224] U is the input vector.

[0225] E is the interference matrix.

[0226] F xfr This is the longitudinal force of the front right wheel.

[0227] F xfl This is the longitudinal force of the front left wheel.

[0228] F xrr This is the longitudinal force of the rear right wheel.

[0229] F xrl This is the longitudinal force of the rear left wheel.

[0230] F desl The desired total longitudinal force.

[0231] e β This represents the centroid side deflection angle error.

[0232] β EKF The centroid sideslip angle obtained from the gradually diminishing EKF observation.

[0233] β d The desired centroid sideslip angle.

[0234] e ω This represents the yaw rate error.

[0235] The yaw rate obtained from EKF observations is gradually decreasing.

[0236] ω rd This represents the desired yaw rate.

[0237] e represents the weighted error of the centroid sideslip angle and yaw rate.

[0238] p1 is the weight of the centroid side slip angle error.

[0239] p2 is the weight of the yaw rate error.

[0240] η fl This refers to the load utilization rate of the front left wheel.

[0241] η fr This refers to the load utilization rate of the front right wheel.

[0242] η rl The load utilization rate of the rear left wheel.

[0243] η rr The load utilization rate of the rear right wheel.

[0244] μ is the road adhesion coefficient.

[0245] J1 is the primary objective.

[0246] J2 is the second objective.

[0247] r is the reward function.

[0248] r ad To track rewards.

[0249] r torque To provide a stable margin of reward.

[0250] H ad This is a marker for angular velocity tracking.

[0251] H β This is a marker for tracking the center of mass side slip angle.

[0252] H ω For yaw rate tracking representation

[0253] L ad This is a penalty flag for angular velocity.

[0254] L torque This is a penalty sign for vehicle stability margin.

[0255] δ fl This refers to the turning angle of the front left wheel.

[0256] δ fr This refers to the steering angle of the front right wheel.

[0257] δ rl The left rear wheel turns.

[0258] δ rr The rear right wheel turns.

[0259] ΔM is the additional yaw torque.

[0260] F desl The desired total longitudinal force.

[0261] s i This is the current state.

[0262] a i For the current state s i The action to be performed.

[0263] s tar_i For the current state s i The target state is as follows.

[0264] r i For the current state s i The reward obtained by performing the following action.

[0265] γ is a discount factor used to measure the importance of future rewards.

[0266] Q′ is the policy network, Q′(s) i+1 ,ψ'(s i+1 |θ ψ' )θ Q′ ψ'(s) represents the estimated cumulative reward under the target state, given by the Critic network. i+1 |θ ψ' ) represents the action output by the Actor network.

[0267] ψ'(s i+1 |θ μ′ ) represents the target network parameters θ of the Actor. ψ' Next state s i+1 The generated action.

[0268] θ represents the parameters of the neural network.

[0269] θ ψ' These are the target network parameters for the Actor.

[0270] θ ψ These are the current network parameters for the Actor.

[0271] θ Q′ These are the target network parameters for Critic.

[0272] θ Q These are the current network parameters for Critic.

[0273] Y pre_i This is the predicted Q value.

[0274] Q is the action value function.

[0275] J(θ Q ) is the root mean square error loss function.

[0276] n is the number of randomly selected samples.

[0277] y tar (i) represents the target Q value, which is calculated using the Bellman equation, and indicates the current state s. i Next, execute action a i The theoretically optimal Q value is then determined.

[0278] J(θ ψ ) is the loss function.

[0279] Q(s i ,a i |θ ψ ) is the parameter θ ψ For state s i Selected action a i The Q value.

[0280] α ψ The learning rate is a hyperparameter. The larger the learning rate, the greater the impact of each update on the parameters, and vice versa.

[0281] For the loss function J(θ) ψ Regarding the parameter θ ψ The gradient represents the direction in which the loss function increases most rapidly with the current parameters. By updating the parameters in the opposite direction of the gradient, we attempt to reduce the loss function.

[0282] The gradient of the action value function corresponding to the current state, i.e., the current state s = s j The following action Ψ'(s=s) is taken. j When ), the gradient of the action value function for the action.

[0283] Let θ be the gradient of the action function corresponding to the current state of the Actor network, i.e., the current network parameters of the Actor are θ. Ψ The current state s = s j Below, the gradient of the action function.

[0284] τ is the soft update coefficient of the DDPG-AAD model.

[0285] The DDPG-AAD model designed in this invention estimates the continuous actions of the vehicle under instability based on the state spaces of the AFS subsystem, ARS subsystem, and DYC subsystem, thereby completing the coordinated control of the AFS subsystem, ARS subsystem, and DYC subsystem.

[0286] See Figure 4 As shown, the steps for constructing the DDPG-AAD model in this invention are as follows:

[0287] S1, initialize the DDPG-AAD model; establish the vehicle differential equations of AFS, ARS and DYC in coordination according to formula (25);

[0288] Based on the vehicle's yaw moment requirement, the vehicle's additional yaw moment is expressed as: Then formula (25) is modified into the state-space equation formula (26); execute step S2;

[0289] S2, Data processing within the DDPG-AAD model: After receiving data from the sensor group and observer, and combining formulas (32) to (36), the longitudinal force constraint of the desired speed, the tracking error of the centroid sideslip angle and yaw rate, and the vehicle stability margin are calculated, and step S3 is executed.

[0290] S3, set the control objective of the DDPG-AAD model, which is formula (37), and execute step S4;

[0291] S4, Design the reward function based on parameters S2 and S3, which is formula (38), and execute step S5;

[0292] The reward function r is determined by the tracking reward r ad With stable margin reward r torque It consists of two parts, corresponding to targets J1 and J2 respectively. In equation (38), H ad The angular velocity tracking indicator means that if equation (35) is less than 0.1 rad / s, the tracking error between the yaw angular velocity and the sideslip angle of the center of mass is small, and a reward is given; H β The centroid sideslip angle tracking indicator is defined as follows: if the centroid sideslip angle tracking error is less than 0.1 rad / s according to equation (33), a reward is given. ω For yaw rate tracking, the meaning is that if equation (34) is less than 0.1 rad / s, the yaw rate tracking error is small, and a reward is given; H torque The angular velocity penalty indicator means that if the value of equation (35) is greater than 1 rad / s, the error is too large and a penalty is imposed; L torque The vehicle stability margin penalty sign indicates the following meaning: Insufficient vehicle stability margin will result in additional penalties; H ad H torque L torque All of these are qualitative reward objectives, while the remainder are quantitative reward objectives that have been optimized to extreme values.

[0293] S5, based on the DDPG algorithm and combined with e β ,e ω Given the vehicle state and 'e', ​​calculate the state space of the DDPG reinforcement learning agent that satisfies EDV, denoted as s. t =(e β ,e ω ,v x ,v y ), proceed to step S6;

[0294] S6: According to the state equation Given the input, calculate the action space of the DDPG reinforcement learning agent that satisfies EDV, denoted as a. t =(Δδ fl ,Δδ fr ,δ rr ,F xfl ,F xfr ,F xrl ,F xrrMeanwhile, since the longitudinal speed is subject to the desired longitudinal force and vehicle turning angle constraints, it must satisfy formula (39) and execute step S7.

[0295] S7, Initialize the network parameters and training parameters in deep reinforcement learning, and then execute step S8;

[0296] S8, clear the experience replay pool M, and proceed to step S9;

[0297] S9. Determine whether the number of iterations N exceeds the set maximum number of iterations W. In this invention, the maximum number of iterations W is set to 50000. If N>M, then training terminates and step S19 is executed; otherwise, step S10 is executed.

[0298] S10, Initialize the state s of the agent, and execute step S11;

[0299] S11, Determine the current state of the agent. t Is it a terminated state? If it is, proceed to step S9; otherwise, proceed to step S12 and continue training.

[0300] S12, the current Actor network π(s) based on the DDPG-AAD model. t Generate the state s at p future time steps by generating m groups of p actions forward from θ. t+i '、Action a t+i Rewards r t+i (1≤i≤p), proceed to step S13;

[0301] S13, calculate the total reward of group m, select the action a with the highest total reward to execute, transition the state to state s', obtain reward r, and execute step S14;

[0302] S14, store {s,a,r,s'} into the experience replay pool M, and execute step S15;

[0303] S15, randomly select n samples from M, calculate the target value and the actual value according to formula (40) and formula (41), and execute step S16;

[0304] S16, update the current Critic network Q and the current Actor network ψ of the DDPG-AAD model, and execute formulas (40-43) during the update process; the policy gradient update formula is (44), and execute S17;

[0305] S17, determine whether the number of iterations is an integer multiple of the set update frequency C. If not, proceed to step S11; otherwise, proceed to step S18.

[0306] S18, Update the target network parameters of the DDPG-AAD model according to formulas (45) and (46), and execute S19;

[0307] S19, Output the optimal control strategy π of the EDV system from the Actor network of the DDPG-AAD model. * (a*,θ)=a*.

[0308] π * (a*,θ) is the optimal strategy under parameter θ. It receives parameter θ and outputs the corresponding optimal action a*.

[0309] a* represents the optimal action.

[0310] The DDPG-AAD model designed in this invention establishes vehicle state equations based on the AFS, ARS, and DYC subsystems, providing a basis for constructing the vehicle state space, vehicle action space, and reward function under instability conditions in subsequent deep reinforcement learning. Considering different steering angles input to each wheel by the AFS and ARS subsystems, the longitudinal force requirement, tracking error, and vehicle stability margin are calculated. The control objective of DDPG is determined in the trained DDPG-AAD model, stopping conditions are specified, and the optimal strategy is output.

[0311] Part Three: Active Safety Control of Distributed Electric Drive Vehicles Based on the Decreasing EKF Observer and the DDPG-AAD Model

[0312] See Figure 1 , Figure 4 , Figure 5 As shown, during the stable driving process of a Distributed Electric Vehicle (DEV) system, if instability occurs in the vehicle stability control process based on the sideslip angle and yaw rate, the present invention employs the fading EKF observer and DDPG-AAD model for active safety control of the DEV, and will execute the following steps:

[0313] Step 1: Vehicle initialization;

[0314] Initialize the dynamic model and the module for calculating the desired yaw rate and center of mass sideslip angle;

[0315] Initialize the Active Front Steering (AFS) subsystem, Active Rear Steering (ARS) subsystem, and Direct Yaw Control (DYC) subsystem;

[0316] The vehicle performs actions based on driving signals provided by the driver and vehicle status signals from the vehicle sensor array.

[0317] Step 2: The EKF observer is gradually reduced to process the yaw rate and the sideslip angle of the center of mass.

[0318] The fading EKF observer receives driving signals, vehicle state signals, and dynamic model parameters on one hand; and on the other hand, it is based on the whole vehicle dynamic model. And based on the yaw rate ω r , centroid side slip angle β EKF Lateral velocity v y and longitudinal velocity v x The process involves: digital differential processing → discretizing the continuous vehicle state using a one-section Euler method; thirdly, adjusting the vehicle's turning angle information as a parameter for the fading EKF observer; fourthly, establishing the state prediction equation and observation prediction equation for the fading EKF observer; and fifthly, performing local linearization in the fading EKF observer based on Φ(k) and c(k), and focusing on the estimated value. The first aspect linearizes Φ(k) and c(k); the sixth aspect completes the current vehicle state update under the fading factor λ(k) and Kalman filter gain K(k).

[0319] Step 3: Establish the linear two-degree-of-freedom spatial state equations, and combine them with the calculation of the desired yaw rate and the sideslip angle of the center of mass;

[0320] Establish the space equation as follows Where, k f and k r These are the equivalent lateral stiffnesses of the front and rear wheels, respectively.

[0321] Yaw rate is affected not only by vehicle speed but also by road surface adhesion conditions; the desired sideslip angle is minimal when driving on low-adhesion surfaces. Therefore, the formulas for calculating the desired yaw rate and the desired sideslip angle are as follows:

[0322] Step 4: In the vehicle stability judgment module, determine whether the current vehicle state is under stability control or not.

[0323] The vehicle stability judgment module receives the optimal solution output by the fading EKF observer on one hand, and the expected yaw rate and centroid sideslip angle output by the expected yaw rate and centroid sideslip angle calculation module on the other hand. Thirdly, it judges the vehicle stability state based on the yaw rate threshold method and the bilinear method. When the vehicle state exceeds the stability boundary of the bilinear phase diagram or the yaw rate threshold, the AFS, ARS and DYC subsystems are activated and output an instability start signal.

[0324] In this invention, the stability boundary of the double-line phase diagram is |B1β|. EFK +B2β|≤ξ, where B1 is the first road surface adhesion coefficient, B2 is the second road surface adhesion coefficient, and βEFK ξ is the centroid sideslip angle after being filtered by the fading EKF observer, β is the vehicle heading angle, and ξ is the tire sideslip angle.

[0325] Based on ω rd β ref ω and β are used to determine the vehicle's stability using a combination of the yaw rate threshold method and the bilinear method. The bilinear method determines the influence of the center-of-gravity sideslip angle on the vehicle; the stability boundary of the bilinear phase diagram can be expressed as... Among them, B1 and B2 are related to the road surface adhesion coefficient, as shown in Table 1.

[0326] Table 1. Parameters of the Stability Boundary Equations

[0327] 0.8≤μ<1 0.685 8.798 0.6 ≤ μ < 0.8 0.426 6.856 0.4 ≤ μ < 0.6 0.384 5.263 0.2 ≤ μ < 0.4 0.296 3.824 μ < 0.2 0.208 2.226

[0328] In this invention, the upper and lower threshold values ​​of the yaw rate are: ω h ω is the upper threshold of the yaw rate. l ω is the lower threshold of the yaw rate. rd For the desired yaw rate, Δω h The upper offset is Δω l This is the lower offset.

[0329] Step 5: Process the vehicle motion space based on the DDPG-AAD model;

[0330] The DDPG-AAD model receives vehicle state signals from the vehicle sensor array and instability start signals from the vehicle stability assessment module. Thirdly, it establishes the four-wheel steering vehicle differential equations for the AFS, ARS, and DYC subsystems, and then discretizes them. Fourthly, to better consider speed tracking, it uses MPC to track the desired speed instead of PID, outputting the longitudinal force F required to achieve the desired speed. desl =F xfl +F xfr +F xrl +F xrr The fifth aspect involves calculating the tracking error e of the centroid sideslip angle and yaw rate. β e ω e; The wheel load utilization rate is used to characterize the vehicle stability margin. The smaller the wheel load, the greater the vehicle stability margin. The sixth aspect is to set the cooperative control objective of EDV and the DDPG reward function. The seventh aspect is to output the vehicle motion space parameters and apply them to the whole vehicle to complete the coordinated control AFS, ARS and DYC subsystems under instability.

[0331] Determine the control objective of DDPG and construct the state space s t Action space a tTrain the policy offline with a reward function r, specify the stopping condition, and output the optimal policy.

[0332] The system state of the agent is determined as s t s t =(e β ,e ω ,v x ,v y The actions of the intelligent agent must also satisfy formula (39).

[0333] The action space of the intelligent agent is a t a t =(Δδ fl ,Δδ fr ,δ rr ,F xfl ,F xfr ,F xrl ,F xrr ).

[0334] Initialize neural network parameters, state, and training parameters; clear the experience replay pool M.

[0335] Based on the Actor network generation, the optimal action is selected for execution and {s,a,r,s'} is stored in M. Q and ψ are then updated. The update process is as follows:

[0336] The policy gradient is updated according to formula (44), and then the target network parameters are updated in combination with the set update frequency C.

[0337] Determine whether the state is a terminated state after each transition. If it is a terminated state, proceed to the next learning cycle; otherwise, continue the action.

[0338] The training iteration count N is not less than the set maximum iteration count W, then the process ends, and the optimal control strategy is output from the Actor network.

[0339] Example

[0340] See Figure 6 and Figure 7 As shown, this is a serpentine steering maneuver test with a road adhesion coefficient of 0.8 and an initial longitudinal speed of 40 km / h. Some important vehicle parameters are as follows:

[0341]

[0342]

[0343] In addition, to demonstrate the algorithm's superiority and increase the testing difficulty, the speed was gradually increased from 40 km / h to 100 km / h, with an acceleration of 2 m / s².2 , yaw rate error and centroid side slip angle error.

[0344] exist Figure 6 and Figure 7 In the case of speeds below 70 km / h, the errors in yaw rate and sideslip angle can mostly be kept within 10°. ―5 The error was on the order of magnitude, with very few cases showing yaw rate and center of mass sideslip error reaching 10. ―2 The error is on the order of magnitude, but the maximum is no more than 0.02959. Although the error increases when the speed exceeds 70 km / h, the vast majority of errors can still be controlled within 10. ―5 ~10 ―4 The error was on the order of magnitude, with very few cases showing yaw rate and center of mass sideslip error reaching 10. ―1 The error was on the order of magnitude and the maximum yaw rate error did not exceed 0.18255, and the maximum centroid side slip angle error did not exceed 0.12872.

Claims

1. A cooperative control method for AFS, ARS, and DYC of a distributed electric drive vehicle based on DDPG, comprising the following steps: Step 1: Establish a dynamic model of a typical vehicle. The sensor array collects driving signals and vehicle status signals, and transmits the vehicle signals to the fading EKF observer. Step 2: Based on the dynamic model of a conventional vehicle, establish the differential equations of a four-wheel independent steering vehicle. The fading EKF observer receives driving signals and vehicle state signals on this basis and estimates the center of gravity sideslip angle and yaw rate. Step 3: Calculate the desired yaw rate and center of mass sideslip angle based on the linear two-degree-of-freedom model; Step 4: The vehicle stability judgment module judges the vehicle's stability state based on the measured value, nominal value, and expected value, while also considering the vehicle's yaw rate and center of gravity sideslip angle, by combining the yaw rate threshold method and the dual-line method. Step 5: Establish vehicle state equations with AFS, ARS, and DYC systems to provide a basis for the subsequent construction of state space, action space, and reward function for deep reinforcement learning; Considering that the AFS and ARS systems input different steering angles for each wheel, calculate the vehicle's longitudinal force requirements, tracking error, and vehicle stability margin; Step 6: Determine the control objective of the DDPG-AAD model and construct the state space s t Action space a t Train the policy offline with a reward function r, specify the stopping condition, and output the optimal policy. Its features are: The fading EKF observer estimates the yaw rate and sideslip angle of the DEC based on the acceleration signal; the parameterized description of the fading EKF observer [X(k+1), Z(k)] is as follows: X(k)=[β EKF ω EKF v x v y ] T (14) Z(k)=h[X(k),u(k)]+ε(k)=[v x v y ] T (15) X(k+1)=g[X(k),u(k)]+w(k) (16) p(k∣k-1)=λ(k)[Φ(k)×p(k-1)×Φ T (k)+Q] (19) C(k)={c(k)[Φ(k)×p(k)×Φ(k) T +Q]c(k) T }+R″ (21) K(k)=p(k∣k-1)×c T (k)[c(k)×p(k∣k-1)×c T (k)+R] -1 (22) p(k)=[I n -K(k)c(k)]×p(k∣k-1) (24) $X(k)$ is the vehicle state at time $k$; $Z(k)$ is the measurement variable of the vehicle state at time $k$; $\beta$ EKF is the sideslip angle of the center of mass filtered by the fading EKF observer; $\omega$ EKF is the yaw rate filtered by the fading EKF observer; $v$ x is the longitudinal speed of the vehicle; $v$ y is the lateral speed of the vehicle; the superscript $T$ represents the coordinate transpose symbol; $X(k + 1)$ is the vehicle state at time $k + 1$; is the updated vehicle state at time $k + 1$; $h[X(k), u(k)]$ is the filtering output equation under the vehicle state $X(k)$ and the estimator $u(k)$; $g[X(k), u(k)]$ is the filtering state equation under the vehicle state $X(k)$ and the estimator $u(k)$; $u(k)$ is the estimator at time $k$; $w(k)$ is the filtering process noise; $\varepsilon(k)$ is the filtering measurement noise; $\Phi(k)$ is the Jacobian matrix of $g(X(k), u(k))$ with respect to the state variables; $\Phi(k|k - 1)$ is the Jacobian matrix at time $k$ predicted through time $k - 1$; $p(k - 1)$ is the covariance matrix at time $k - 1$; $p(k|k - 1)$ is the covariance matrix at time $k$ predicted through time $k - 1$; $c(k)$ is the Jacobian matrix of $h(X(k), u(k))$ with respect to the state variables; $C(k)$ is the estimated covariance matrix; $K(k)$ is the Kalman filter gain; $\lambda(k)$ is the fading factor at time $k$; $tr$ is the trace operation symbol; $p(k)$ is a substitute symbol, and $p(k)=[I$ n $-K(k)c(k)]\times p(k|k - 1)$ where $I$ n is the identity matrix; The parameterized description of the DDPG-AAD model is as follows: U=[Δδ fl ,Dd fr ,a rl ,a rr ,F xfr ,F xfl ,F xrr ,F xrl ] T (30) F desl =F xfl +F xfr +F xrl +F xrr (32) e β ＝β EKF -β d (33) e ω =ω EKF -oh rd (34) e=p1(β EKF -b d )+p2(ω EKF -oh rd ) (35) s tar_i =r i +γQ′(s i+1 ,ψ'(s i+1 |θ ψ' )i Q′ ) (40) AND pre_i =Q(s i ,to i ∣θ Q ) (41) i Q' ←tth Q' +(1-τ)θ Q' (45) i ψ' ←tth ψ +(1-τ)θ ψ' (46) m represents the vehicle mass; ν x The longitudinal speed of the vehicle; This is the sideslip angle acceleration at the vehicle's center of gravity. k is the yaw acceleration of the vehicle. f k is the equivalent lateral stiffness of the vehicle's front wheels. r The equivalent lateral stiffness of the vehicle's rear wheels; f The distance from the center of mass to the front axle; l r δ is the distance from the center of mass to the rear axle; f The steering angle of the front wheels controlled by the driver; Δδ fl The adjustment angle for the front left wheel steering angle by the AFS system; Δδ fr The adjustment angle for the front right wheel steering angle by the AFS system; α rl α is the input steering angle of the ARS system for the rear left wheel. rr The input steering angle for the rear right wheel in the ARS system; I z Let t be the moment of inertia of the vehicle about the z-axis; f The front axle track width; t r This refers to the rear axle track. Let X be the first derivative of the state variable with respect to time; X be the state variable; A be the state matrix; B be the input matrix; U be the input vector; E be the disturbance matrix; F be the disturbance matrix. xfr The longitudinal force of the front right wheel; F xfl The longitudinal force of the front left wheel; F xrr The longitudinal force of the rear right wheel; F xrl The longitudinal force of the rear left wheel; F desl e represents the desired total longitudinal force. β For the centroid sideslip angle error; β EKF The centroid sideslip angle obtained from the gradually diminishing EKF observations; β d e is the desired centroid sideslip angle; ω This refers to the yaw rate error. To gradually eliminate the yaw rate obtained from EKF observations; ω rd η is the desired yaw rate; e is the weighted error of the sideslip angle and yaw rate; p1 is the weight of the sideslip angle error; p2 is the weight of the yaw rate error; η is the weight of the yaw rate error. fl η represents the load utilization rate of the front left wheel. fr η represents the load utilization rate of the front right wheel. rl η represents the load utilization rate of the rear left wheel. rr The load utilization rate of the rear right wheel; μ is the road adhesion coefficient; J1 is the first objective; J2 is the second objective; r is the reward function; r ad To track rewards; r torque For stable margin rewards; H ad H is the indicator for angular velocity tracking. β For tracking the center of mass sideslip angle; H ω L represents the yaw rate tracking. ad For angular velocity penalty; L torque Vehicle stability margin penalty sign; δ fl The steering angle of the front left wheel; δ fr The steering angle of the front right wheel; δ rl The rear left wheel steering angle; δ rr The rear right wheel steering angle is ΔM; the additional yaw torque is F. desl The desired total longitudinal force; s i The current state; a i For the current state s i The action to be performed; s tar_i For the current state s i The target state is as follows; r i For the current state s i The reward obtained by performing the action; γ is the discount factor used to measure the importance of future rewards; Q′ is the policy network, Q′(s i+1 ,ψ'(s i+1 |θ ψ' )θ Q′ ψ'(s) represents the estimated cumulative reward under the target state, given by the Critic network. i+1 |θ ψ ') represents the action output by the Actor network; ψ'(s) i+1 |θ μ′ ) represents the target network parameters θ of the Actor. ψ' Next state s i+1 The generated action; θ is the parameter of the neural network; θ ψ' For the Actor target network parameters; θ ψ θ represents the current network parameters of the Actor. Q′ θ represents the target network parameters for Critic. Q Y represents the current network parameters of Critic; pre_i J(θ) represents the predicted Q value; Q is the action value function; J(θ) is the predicted Q value. Q ) represents the root mean square error loss function; n is the number of randomly selected samples; y tar (i) represents the target Q value, which is calculated using the Bellman equation, and indicates the current state s. i Next, execute action a i The theoretical optimal Q value; J(θψ) is the loss function; Q(s) i ,a i |θ ψ ) is the parameter θ ψ For state s i Selected action a i Q value; α ψ The learning rate is a hyperparameter. The larger the learning rate, the greater the impact of each update on the parameters, and vice versa. For the loss function J(θ) ψ Regarding the parameter θ ψ The gradient represents the direction in which the loss function increases the fastest with the current parameters; by updating the parameters in the opposite direction of the gradient, we attempt to reduce the loss function. The gradient of the action value function corresponding to the current state, i.e., the current state s = s j The following action Ψ'(s=s) is taken. j When ), the gradient of the action value function with respect to the action; Let θ be the gradient of the action function corresponding to the current state of the Actor network, i.e., the current network parameters of the Actor are θ. Ψ The current state s = s j Below, the gradient of the action function; τ is the soft update coefficient of the DDPG-AAD model.

2. The AFS, ARS, and DYC cooperative control method for distributed electric drive vehicles based on DDPG according to claim 1, characterized in that: The description of the vehicle dynamics model is as follows: This represents the sum of the longitudinal forces on the vehicle. This represents the total lateral forces acting on the vehicle. The sum of the torques on the vehicle about the z-axis; m is the total mass of the vehicle; v x The longitudinal speed of the vehicle; v is the longitudinal acceleration of the vehicle. y The lateral speed of the vehicle; ω is the lateral acceleration of the vehicle. r The vehicle's yaw rate; This refers to the vehicle's yaw acceleration. δ is the steering angle of the front tires; I z Let l be the yaw moment of inertia of the vehicle about the z-axis in the vehicle coordinate system xoy; f The distance from the front axle to the center of gravity; l r t is the distance from the rear axle to the center of mass. f The track width of the front axle; t r The rear axle track width; ΔM is the additional yaw torque; the aforementioned 3. The AFS, ARS, and DYC cooperative control method for distributed electric drive vehicles based on DDPG according to claim 1, characterized in that: based on The designed tire model Described as: F x =μDsin{Carctan[Bλ-E(Bλ-arctan(Bλ))]} (12) F y =μDsin{Carctan[Bξ-E(Bξ-arctan(Bξ))]} (13) The torque is for the front left wheel; The torque is for the front right wheel; The torque of the rear left wheel; The torque of the rear right wheel; F zfl The vertical force F is the force on the front left wheel. zfr The vertical force F is the force on the front right wheel. zrl The vertical force F is the force on the rear left wheel. zrr F is the vertical force on the rear right wheel. x For the longitudinal force of the tire; F y For the lateral force of the tire; I w ω is the moment of inertia of the tire. fl This refers to the rotational speed of the front left wheel; ω is the acceleration of the front left wheel. fr This refers to the rotational speed of the front right wheel; ω is the acceleration of the front right wheel; rl This refers to the rotational speed of the rear left wheel; ω is the acceleration of the rear left wheel. rr This refers to the rotational speed of the rear right wheel; T is the acceleration of the rear right wheel; qfl T is the driving torque of the front left wheel; sfl T is the braking torque of the front left wheel; qfr T is the driving torque of the front right wheel; sfr T is the braking torque of the front right wheel; qrl T is the driving torque of the rear left wheel; srl The braking torque of the rear left wheel; T qrr T is the driving torque of the rear right wheel; srr R is the braking torque of the right rear wheel; m is the vehicle mass; g is the acceleration due to gravity. The longitudinal acceleration of the vehicle; denoted as lateral acceleration of the vehicle; μ as the road adhesion coefficient; D as the peak factor in the tire magic formula; C as the shape factor in the tire magic formula; B as the stiffness factor in the tire magic formula; λ as the tire slip ratio; E as the curvature factor in the tire magic formula; and ξ as the tire slip angle.

4. The AFS, ARS, and DYC cooperative control method for distributed electric drive vehicles based on DDPG according to claim 1, 2, or 3, characterized in that: The steps for active safety control of distributed electric drive vehicles based on the fading EKF observer and DDPG-AAD model are as follows: Step 1: Vehicle initialization; Initialize the dynamic model and the module for calculating the desired yaw rate and center of mass sideslip angle; Initialize the active front wheel steering subsystem, the active rear wheel steering subsystem, and the direct yaw moment control subsystem; The vehicle performs actions based on driving signals provided by the driver and vehicle status signals from the vehicle sensor array. Step 2: The EKF observer is gradually reduced to process the yaw rate and the sideslip angle of the center of mass. The fading EKF observer receives driving signals, vehicle state signals, and dynamic model parameters on one hand; and on the other hand, it is based on the whole vehicle dynamic model. And based on the yaw rate ω r , centroid side slip angle β EKF Lateral velocity v y and longitudinal velocity v x The process involves: performing digital differential processing; using a single Euler method to discretize the continuous vehicle state; adjusting the vehicle's turning angle information as a parameter for the fading EKF observer; and establishing the state prediction equation and observation prediction equation for the fading EKF observer. The fifth aspect involves local linearization in the fading EKF observer based on Φ(k) and c(k), and around the estimated value. The first step is to linearize Φ(k) and c(k); the sixth step is to update the current vehicle state under the fading factor λ(k) and Kalman filter gain K(k). Step 3: Establish the linear two-degree-of-freedom spatial state equations, and combine them with the calculation of the desired yaw rate and the sideslip angle of the center of mass; Establish the space equation as follows Where, k f and k r These are the equivalent lateral stiffnesses of the front and rear wheels, respectively. The yaw rate is affected not only by vehicle speed but also by road surface adhesion conditions; the desired sideslip angle is minimal when driving on low-adhesion surfaces; therefore, the accurate formulas for calculating the desired yaw rate and the desired sideslip angle are as follows: Step 4: In the vehicle stability judgment module, determine whether the current vehicle state is under stability control or not. The vehicle stability judgment module receives the optimal solution output by the fading EKF observer on one hand, and the expected yaw rate and center of gravity sideslip angle output by the expected yaw rate and center of gravity sideslip angle calculation module on the other hand; and thirdly, it judges the vehicle stability state based on the yaw rate threshold method and the bilinear method. When the vehicle state exceeds the stability boundary of the bilinear phase diagram or the yaw rate threshold, the AFS, ARS and DYC subsystems are activated and output an instability start signal. In this invention, the stability boundary of the double-line phase diagram is |B1β|. EFK +B2β|≤ξ, where B1 is the first road surface adhesion coefficient, B2 is the second road surface adhesion coefficient, and β EFK ξ is the centroid sideslip angle after filtering by the fading EKF observer, β is the vehicle heading angle, and ξ is the tire sideslip angle. Based on ω rd β ref ω and β are used to determine the vehicle's stability using a combination of the yaw rate threshold method and the bilinear method. The bilinear method is used to determine the influence of the center of gravity sideslip angle on the vehicle, and the stability boundary of the bilinear phase diagram can be expressed as... The upper and lower thresholds of yaw rate are ω h ω is the upper threshold of the yaw rate. l ω is the lower threshold of the yaw rate. rd For the desired yaw rate, Δω h The upper offset is Δω l This is the lower offset; Step 5: Process the vehicle motion space based on the DDPG-AAD model; The DDPG-AAD model receives vehicle state signals from the vehicle sensor array and instability start signals from the vehicle stability assessment module. Thirdly, it establishes the four-wheel steering vehicle differential equations for the AFS, ARS, and DYC subsystems, and then discretizes them. Fourthly, to better consider speed tracking, it uses MPC to track the desired speed instead of PID, outputting the longitudinal force F required to achieve the desired speed. desl =F xfl +F xfr +F xrl +F xrr The fifth aspect involves calculating the tracking error e of the centroid sideslip angle and yaw rate. β e ω e; The wheel load utilization rate is used to characterize the vehicle stability margin. The smaller the wheel load, the greater the vehicle stability margin. The sixth aspect is to set the cooperative control target of EDV and the DDPG reward function. The seventh aspect is to output the vehicle motion space parameters and apply them to the whole vehicle to complete the coordinated control AFS, ARS and DYC subsystems under instability. Determine the control objective of DDPG and construct the state space s t Action space a t Train the policy offline with a reward function r, specify the stopping condition, and output the optimal policy. The system state of the agent is determined as s t s t =(e β ,e ω ,v x ,v y The actions of the intelligent agent also need to satisfy... The action space of the intelligent agent is a t a t =(Δδ fl ,Δδ fr ,δ rr ,F xfl ,F xfr ,F xrl ,F xrr ); Initialize neural network parameters, state, and training parameters; clear the experience replay pool M; Based on the Actor network generation, the optimal action is selected for execution and {s,a,r,s'} is stored in M. Q and ψ are then updated. The update process is as follows: in accordance with Perform policy gradient updates, and then update the target network parameters using the set update frequency C. Determine whether the state is a terminated state after each transition. If it is a terminated state, proceed to the next learning cycle; otherwise, continue the action. The training iteration count N is not less than the set maximum iteration count W, then the process ends, and the optimal control strategy is output from the Actor network.

5. The AFS, ARS, and DYC cooperative control method for distributed electric drive vehicles based on DDPG according to claim 1, characterized in that... The steps to construct a fading EKF observer are as follows: Step A: Initialize the fading EKF observer; receive driving signals and vehicle status signals collected by the sensors; execute step B; Step B, based on the vehicle dynamics model And based on the yaw rate ω r , centroid side slip angle β EKF Lateral velocity v y and longitudinal velocity v x Performing digital differential equations Processed and differentiated, then used for vehicle motion control; Execute step C; In differential equations In the middle, the said in, Let δ be the yaw acceleration of the vehicle. fl δ is the tire steering angle of the front left wheel. fr δ is the tire steering angle of the front right wheel. rl δ is the tire steering angle of the rear left wheel. rr The tire steering angle of the rear right wheel, t f I is the track width of the front axle. z Let be the yaw moment of inertia of the vehicle about the z-axis; In differential equations In the middle, the said ω EKF The yaw rate is the yaw rate after filtering by the fading EKF observer; In differential equations In the middle, the said In differential equations In the middle, the said Step C involves discretizing the vehicle differential equations. A section of the Euler method is used to discretize the continuous vehicle state equations, resulting in the discrete-vehicle differential equations [ω]. r (k+1),β EKF (k+1),v x (k+1),v y [(k+1)], execute step D; In the discrete-vehicle differential equation [ω r (k+1),β EKF (k+1),v x (k+1),v y In (k+1)], the ω r (k+1) represents the vehicle's yaw rate at time k+1, ω r Let k be the yaw rate of the vehicle at time k. Let Δt be the yaw acceleration of the vehicle, and Δt be the time step. In the discrete-vehicle differential equation [ω r (k+1),β EKF (k+1),v x (k+1),v y In [(k+1)], the β EKF (k+1)=β EKF (k)+(β EKF ×Δt), β EKF (k+1) is the centroid sideslip angle at time k+1, β EKF (k) is the centroid sideslip angle at time k; In the discrete-vehicle differential equation [ω r (k+1),β EKF (k+1),v x (k+1),v y In [k+1], the ν x (k+1)=ν x (k)+(v x +ν y ω r )Δt, v x (k+1) represents the longitudinal velocity of the vehicle at time k+1, v x (k) represents the longitudinal velocity of the vehicle at time k; In the discrete-vehicle differential equation [ω r (k+1),β EKF (k+1),v x (k+1),v y In [k+1], the ν y (k+1)=ν y (k)+(v y -ν x ω r )Δt, v y (k+1) represents the lateral velocity of the vehicle at time k+1, v y (k) represents the lateral velocity of the vehicle at time k; Step D: Use the vehicle turning angle information as the adjustment parameter for the fading EKF observer; execute step D. Step E, based on X(k)=[β EKF ω EKF v x v y ] T , Z(k)=h[X(k),u(k)]+ε(k)=[v x v y ] T Establish the state prediction equation and observation prediction equation of the fading EKF observer using X(k+1)=g[X(k),u(k)]+w(k), and execute step F; Wherein, the state variable is X(k) and the measurement variable is Z(k); the state equation g[X(k),u(k)] and the measurement output equation h[X(k),u(k)] of the DEV vehicle motion can be estimated through the state variable and the measurement variable; Step F involves local linearization in the fading EKF observer based on Φ(k) and c(k), and around the estimated value. Linearize Φ(k) and c(k); execute step G; Step G: Introduce the fading factor λ(k) to predict the vehicle motion covariance; Among them, the covariance prediction matrix of the fading EKF observer after introducing the fading factor λ(k) is p(k|k-1). When λ(k)≥1, Q(k-1) is the covariance matrix of the noise, and step H is executed. Step H: Calculate the Kalman filter gain K(k); then proceed to step I. Step I, based on Perform vehicle state update according to p(k) = [I n The vehicle motion covariance is updated by performing the operation of [-K(k)c(k)]×p(k∣k-1). If the updated result does not satisfy the optimal solution output of the fading EKF observer, the process returns to step D for the next round of adjustment until the optimal solution is output. At this point, the estimation of the fading EKF observer ends. The optimal solution is characterized by the yaw rate ω. r And the centroid side slip angle β EKF .

6. The AFS, ARS, and DYC cooperative control method for distributed electric drive vehicles based on DDPG according to claim 1, characterized in that... The steps to construct a DDPG-AAD model are as follows: S1, initialize the DDPG-AAD model; establish the vehicle differential equations of AFS, ARS and DYC in coordination according to formula (25); Based on the vehicle's yaw moment requirement, the vehicle's additional yaw moment is expressed as: Then formula (25) is modified into the state-space equation formula (26); execute step S2; S2, Data processing within the DDPG-AAD model: After receiving data from the sensor group and observer, and combining formulas (32) to (36), the longitudinal force constraint of the desired speed, the tracking error of the centroid sideslip angle and yaw rate, and the vehicle stability margin are calculated, and step S3 is executed. S3, set the control objective of the DDPG-AAD model, which is formula (37), and execute step S4; S4, Design the reward function based on parameters S2 and S3, which is formula (38), and execute step S5; The reward function r is determined by the tracking reward r ad With stable margin reward r torque It consists of two parts, corresponding to two targets, J1 and J2, respectively; in equation (38), H ad The angular velocity tracking indicator means that if equation (35) is less than 0.1 rad / s, the tracking error between the yaw angular velocity and the sideslip angle of the center of mass is small, and a reward is given; H β The centroid sideslip angle tracking indicator is defined as follows: if the centroid sideslip angle tracking error is less than 0.1 rad / s according to equation (33), a reward is given. ω For yaw rate tracking, the meaning is that if equation (34) is less than 0.1 rad / s, the yaw rate tracking error is small, and a reward is given; H torque The angular velocity penalty indicator means that if the value of equation (35) is greater than 1 rad / s, the error is too large and a penalty is imposed; L torque The vehicle stability margin penalty sign indicates the following meaning: Insufficient vehicle stability margin will result in additional penalties; H ad H torque L torque All of these are qualitative goal rewards, while the remainder are quantitative goal extreme value rewards. S5, based on the DDPG algorithm and combined with e β ,e ω Given the vehicle state and 'e', ​​calculate the state space of the DDPG reinforcement learning agent that satisfies EDV, denoted as s. t =(e β ,e ω ,v x ,v y ), proceed to step S6; S6: According to the state equation Given the input, calculate the action space of the DDPG reinforcement learning agent that satisfies EDV, denoted as a. t =(Δδ fl ,Δδ fr ,δ rr ,F xfl ,F xfr ,F xrl ,F xrr Meanwhile, since the longitudinal speed is subject to the desired longitudinal force and vehicle turning angle constraints, it must satisfy formula (39) and execute step S7. S7, Initialize the network parameters and training parameters in deep reinforcement learning, and then execute step S8; S8, clear the experience replay pool M, and proceed to step S9; S9, determine whether the number of iterations N exceeds the set maximum number of iterations W. In this invention, the maximum number of iterations W is set to 50000. If N>M, then training is terminated and step S19 is executed. Conversely, proceed to step S10; S10, Initialize the state s of the agent, and execute step S11; S11, Determine the current state of the agent. t Is it in a terminated state? If it is in a terminated state, then proceed to step S9. Conversely, proceed to step S12 and continue training; S12, the current Actor network π(s) based on the DDPG-AAD model. t Generate the state s at p future time steps by generating m groups of p actions forward from θ. t+i '、Action a t+i Rewards r t+i (1≤i≤p), proceed to step S13; S13, calculate the total reward of group m, select the action a with the highest total reward to execute, transition the state to state s', obtain reward r, and execute step S14; S14, store {s,a,r,s'} into the experience replay pool M, and execute step S15; S15, randomly select n samples from M, calculate the target value and the actual value according to formula (40) and formula (41), and execute step S16; S16, update the current Critic network Q and the current Actor network ψ of the DDPG-AAD model, and execute formulas (40-43) during the update process; the policy gradient update formula is (44), and execute S17; S17, determine whether the number of iterations is an integer multiple of the set update frequency C. If not, proceed to step S11; otherwise, proceed to step S18. S18, Update the target network parameters of the DDPG-AAD model according to formulas (45) and (46), and execute S19; S19, Output the optimal control strategy π of the EDV system from the Actor network of the DDPG-AAD model. * (a*,θ)=a*; π * (a*,θ) is the optimal strategy under parameter θ. It receives parameter θ and outputs the corresponding optimal action a*. a* represents the optimal action.

7. The AFS, ARS, and DYC cooperative control method for distributed electric drive vehicles based on DDPG according to claim 1, 2, or 3, characterized in that: When a distributed electric drive vehicle system is driving stably, if instability occurs during the vehicle stability control process based on the center of gravity sideslip angle and yaw rate, an effusive EKF observer and a DDPG-AAD model are used for active safety control of the distributed electric drive vehicle.

Images