Intelligent electric vehicle path tracking control method based on Q learning genetic algorithm

Through the intelligent electric vehicle path tracking control method based on Q-learning genetic algorithm, the dynamic stability problem during driving at high speed and high curvature is solved, and the precise control of four-wheel steering angle and driving torque is achieved, which improves the path tracking ability and stability are improved. , suitable for smart electric vehicles under extreme operating conditions.

CN118928401AActive Publication Date: 2024-11-12JILIN UNIVERSITY +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202410971315.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-12
Estimated Expiration
2044-07-19

AI Technical Summary

Technical Problem

When driving at high speed and high curvature, the path tracking control of smart electric vehicles is difficult to meet the dynamic stability requirements, especially under high-speed and low-attached road surfaces, which are prone to dangerous working conditions such as side slippage and tail swing. The existing active steering is coordinated with DYC. Control takes less trade-offs on tire slip, controller parameters, and system dynamic performance.

Method used

Using the intelligent electric vehicle path tracking control method based on Q-learning genetic algorithm, the system output is redefined and decomposed into input and output subsystem and zero dynamic subsystem, and an adaptive generalized sliding mode control strategy and control based on Q-learning genetic algorithm optimization are proposed. The device parameter design method realizes precise control of four-wheel steering angle and driving torque, ensuring the dynamic stability of the vehicle under extreme operating conditions.

Benefits of technology

It improves the path tracking capability and dynamic stability of smart electric vehicles, prevents excessive sliding of wheels, and achieves rapid convergence of heading angles and lateral displacements. It has a wide range of applications and is suitable for widespread promotion and application.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118928401A_ABST
    Figure CN118928401A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent electric vehicle path tracking control method based on a Q learning genetic algorithm, and relates to the field of intelligent electric vehicle path tracking control. A path tracking model is established according to a dynamic motion mechanism of a vehicle; secondly, considering the slip state, trajectory tracking error and yaw stability of the four wheels, redefining the output of the path tracking system, and decomposing the path tracking system into an input and output subsystem and a zero dynamic subsystem through input and output linearization; a self-adaptive generalized sliding-mode control method is provided for the input and output subsystem, so that the state of the input and output subsystem quickly follows an ideal value of the input and output subsystem; stable conditions of a zero-dynamic subsystem are obtained through stability analysis, and a controller parameter design method based on Q learning genetic algorithm optimization is provided on the basis, so that asymptotic stability of an intelligent electric vehicle path tracking control system near a balance point is achieved. According to the method, the path tracking capability of the intelligent electric vehicle is effectively improved, and the dynamic stability of the intelligent electric vehicle under the limit working condition is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent electric vehicle path tracking control, and mainly to an intelligent electric vehicle path tracking control method based on Q learning genetic algorithm. Background Art

[0002] Path tracking control is a key technology for realizing the intelligentization of automobiles. It controls the size of the front wheel steering angle in real time through vehicle status information and pre-planned desired paths, aiming to minimize the lateral offset and heading error between the vehicle and the target path, and ensure that the vehicle travels along the expected path. At present, intelligent vehicles can achieve good path tracking effects under most working conditions. However, when the car is driving at high speed and large curvature, the lateral nonlinear factors of the tires are enhanced, and the path tracking accuracy under steering control will be greatly affected. The front wheel steering angle determined by the path tracking controller is difficult to meet the control requirements; on high-speed and low-adhesion roads, the car is prone to side slip, tail swing and other dangerous conditions when turning. Since there is only one control quantity, the front wheel steering angle, under extreme conditions, it is difficult for the steering controller to ensure the dynamic stability of the car while tracking the path.

[0003] The steering system directly generates lateral acceleration of the vehicle through the lateral force of the tire, which directly affects the lateral displacement, while active yaw moment control (DYC) indirectly affects the lateral response of the vehicle by affecting the yaw angular velocity. The coordinated control of DYC and active steering can effectively improve the dynamic performance of the vehicle in the process of path tracking. The driving / braking torque of the four wheels of a four-wheel independent drive electric vehicle is independently controllable, and the driving / braking torque control of a single wheel and the torque distribution control between axles and wheels can be accurately and quickly realized within the capacity of the motor, providing additional degrees of freedom for the lateral control of the vehicle, which can significantly improve the maneuverability and stability of the vehicle. The control methods commonly used for the coordination mechanism of active steering and DYC in four-wheel independent drive intelligent electric vehicles include: optimal control, sliding mode control, adaptive inversion control, model predictive control, etc. Reference 1 [Liang J, Lu Y, Yin G, et al. A Distributed Integrated Control Architecture of AFS and DYC based on MAS for Distributed Drive Electric Vehicles [J]. IEEE Transactions on Vehicular Technology, 2021, (99): 1-1.] regards the AFS system and the DYC system as multi-agents, and proposes a new distributed framework for integrated control of the AFS+DYC system based on the MAS idea. The collaborative control strategy of the two agents is obtained through the Pareto optimal theory to improve the lateral stability of the vehicle and reduce the amount of calculation of the driver during path tracking. Reference 2 [Hang P, Chen X, Luo F. LPV / H-infinity Controller Design for Path Tracking of Autonomous Ground Vehicles Through Four-Wheel Steering and Direct Yaw-Moment Control [J]. International Journal of Automotive Technology, 2019 (4): 20.] A four-wheel steering (4WS) and direct yaw moment control (DYC) system is used to design the path tracking control of an autonomous ground vehicle (AGV). The linear variable parameter (LPV) / H∞ control is used as the upper-level controller, and the weighted least squares (WLS) allocation algorithm is used at the lower level for torque distribution.Reference 3 [Mashadi, B., Ahmadizadeh, P., Majidi, M. et al. Integrated robust controller for vehicle path following. Multibody Syst Dyn 33, 207–228 (2015).] For the integrated 4WS+DYC control system, a robust controller is designed based on the μ synthesis method to realize the path tracking of AGV. This method has the powerful ability to enable the vehicle to track the required path in the presence of parameter uncertainty. The above active steering and DYC coordinated control schemes both utilize the redundant control freedom brought by the active yaw torque to improve the path tracking capability of the four-wheel drive smart electric vehicle while taking into account the dynamic stability. However, the above methods rarely consider the effective trade-off between tire slip, controller parameters and system dynamic performance during implementation.

[0004] The main idea of ​​sliding mode control is to design a dynamic, continuous or discrete control strategy so that the state of the system can slide on the preset sliding surface to achieve the desired performance. Therefore, the present invention considers the slip state, trajectory tracking error and yaw stability of the four wheels, redefines the output of the path tracking system, and decomposes the path tracking system into an input-output subsystem and a zero dynamic subsystem through input-output linearization; for the input-output subsystem, an adaptive generalized sliding mode control method is proposed to make the state of the input-output subsystem quickly follow its ideal value; for the zero dynamic subsystem, the constraint conditions for the stability of the zero dynamic subsystem are obtained through stability analysis, and on this basis, a controller parameter design method based on Q-learning genetic algorithm optimization is proposed to achieve the asymptotic stability of the intelligent electric vehicle path tracking control system near the equilibrium point, improve the path tracking capability of the intelligent electric vehicle, and ensure its dynamic stability under extreme working conditions. Summary of the invention

[0005] In order to improve the path tracking capability of intelligent electric vehicles and ensure their dynamic stability under extreme working conditions, the present invention adopts a method of redefining the system output and proposes an intelligent electric vehicle path tracking control method based on Q-learning genetic algorithm, which improves the dynamic stability of the system while ensuring good tracking control effects under different working conditions.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] A path tracking control method for an intelligent electric vehicle based on a Q-learning genetic algorithm comprises the following steps:

[0008] Step 1: Based on the kinematics and dynamics of the vehicle, a path tracking model of the four-wheel steering and four-wheel independent drive smart electric vehicle is established;

[0009] Step 2: Based on the path tracking model obtained in step 1, redefine the output of the system, and decompose the path tracking model into an input-output subsystem and a zero dynamic subsystem by input-output linearization;

[0010] Step 3: Use the CarSim car model to obtain the real-time parameters of the vehicle: Y-axis coordinates, X-axis coordinates, heading angle, lateral speed and acceleration, longitudinal speed and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc.

[0011] Step 4: Based on the Y-axis coordinates, X-axis coordinates, heading angle, lateral velocity and acceleration, longitudinal velocity and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc. output by the CarSim car model in step 3, a controller parameter design method based on Q-learning genetic algorithm optimization is proposed with the goal of rapid convergence of the zero dynamic subsystem state to obtain the optimized controller parameters;

[0012] Step 5. According to the given reference trajectory, the input-output subsystem obtained in step 2, the Y-axis coordinate, X-axis coordinate, heading angle, lateral velocity and acceleration, longitudinal velocity and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc. output by the CarSim car model in step 3, and the controller parameters obtained in step 4, the four-wheel steering angle and four-wheel drive torque are obtained through the input-output subsystem adaptive generalized sliding mode control module, and input them into the CarSim car model.

[0013] The beneficial effects of the present invention are as follows:

[0014] 1) According to the dynamic motion mechanism of the vehicle, the present invention establishes a mathematical model of the path tracking system, considers the slip state of the four wheels, trajectory tracking error and yaw stability, redefines the output of the system, and uses input-output linearization to decompose the path tracking system into an input-output subsystem and a zero dynamic subsystem, thereby achieving dimensionality reduction and decoupling of the model.

[0015] 2) The present invention proposes an adaptive generalized sliding mode control strategy for the input-output subsystem, so that the state of the input-output subsystem: the four-wheel slip state Δω σ , the lateral motion of the vehicle Δγ and Δv y It converges to zero in a finite time, while controlling the vehicle's handling stability, preventing excessive wheel slip.

[0016] 4) The present invention aims to quickly converge the heading angle following error and lateral displacement following error of the zero dynamic subsystem near the equilibrium point, and proposes a controller parameter design method based on Q-learning genetic algorithm optimization, so as to ensure the asymptotic stability of the trajectory tracking control system of the intelligent electric vehicle.

[0017] 5) The method proposed in the present invention is simple, easy to implement, and has a wide range of applications, and is suitable for widespread promotion and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a principle block diagram of the intelligent electric vehicle path tracking control method based on the Q learning genetic algorithm of the present invention.

[0019] Figure 2 This is a principle block diagram of the controller parameter design method based on Q learning genetic algorithm optimization of the present invention. DETAILED DESCRIPTION

[0020] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 As shown, the intelligent electric vehicle path tracking control method based on Q learning genetic algorithm of the present invention comprises: step one, according to the kinematic and dynamic mechanism of the vehicle, a path tracking model of the four-wheel steering and four-wheel independent drive intelligent electric vehicle is established; step two, according to the path tracking model obtained in step one, the output of the system is redefined, and the path tracking model is decomposed into an input-output subsystem and a zero dynamic subsystem through input-output linearization; step three, using the CarSim car model to obtain the real-time parameters of the vehicle: Y-axis coordinates, X-axis coordinates, heading angle, lateral speed and acceleration, longitudinal speed and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc.; step four, according to the Y-axis coordinates, X-axis coordinates, heading angle output by the CarSim car model in step three, Angle, lateral velocity and acceleration, longitudinal velocity and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc., with the goal of rapid convergence of the zero dynamic subsystem state, a controller parameter design method based on Q learning genetic algorithm optimization is proposed to obtain the optimized controller parameters; Step five, according to the given reference trajectory, the input and output subsystem obtained in step two, the Y-axis coordinates, X-axis coordinates, heading angle, lateral velocity and acceleration, longitudinal velocity and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc. output by the CarSim car model in step three, and the controller parameters obtained in step four, the four-wheel steering angle and four-wheel drive torque are obtained through the input and output subsystem adaptive generalized sliding mode control module, and they are input into the CarSim car model.

[0022] like Figure 2 As shown, the genetic algorithm optimization steps based on Q learning of the present invention include: step 1, population and parameter initialization; step 2, penalty factor optimization based on Q learning; step 3, termination criterion 1 judgment; step 4, selection, adaptive crossover and mutation operations; step 5, generation of a new generation of population; step 6, termination criterion 2 judgment.

[0023] The intelligent electric vehicle path tracking control method based on the Q learning genetic algorithm of the present invention has the following specific implementation steps:

[0024] Step 1: Based on the dynamic mechanism of the vehicle, a path tracking model of a four-wheel drive intelligent electric vehicle is established. The specific implementation method of the path tracking model of a four-wheel drive intelligent electric vehicle is as follows:

[0025] 1.1 Vehicle dynamics model

[0026] In the design of the controller, the handling dynamics model of the four-wheel independent drive smart electric vehicle can be described as:

[0027]

[0028] Among them, v x and v y are the longitudinal and lateral velocities of the vehicle, respectively. r is the yaw rate. m and I z is the vehicle mass and yaw inertia. F yf and F yr is the lateral force of the front and rear tires. f and δ r is the steering angle of the front and rear wheels. f and l r ΔM is the distance from the center of mass of the vehicle to the front axle and the rear axle respectively. z The additional yaw moment generated by the longitudinal force difference between the left and right tires is:

[0029]

[0030] Among them, T qσ is the driving torque of each wheel, σ=fl, fr, rl, rr, representing the left front wheel, right front wheel, left rear wheel, right rear wheel, t wσ Satisfy t wfl =t wfr =t wf , t wrl =t wrr =t wr , t wf ,t wr are the wheelbases of the front and rear wheels of the vehicle, R e is the wheel rolling radius.

[0031] When the lateral acceleration is less than 0.4g (g is the acceleration of gravity), the slip angle does not exceed 4° to 5°. The relationship between the tire lateral force and the slip angle can be expressed as a linear relationship, which can be written as:

[0032] F yf =-k f α f , F yr=-k r α r (4)

[0033] Among them, k f and k r are the cornering stiffness of the front and rear tires respectively. f and α r is the front and rear tire slip angle, defined as:

[0034]

[0035] Assuming that the steering angles of the front and rear wheels are small, cosδ in equations (1) and (2) is f ≈cosδ r ≈1. Substituting equations (4) and (5), the vehicle model described by equations (1) and (2) can be simplified to the following linear dynamic equation:

[0036]

[0037] Wheel dynamics equation:

[0038]

[0039] In the formula, ω σ is the rolling angular velocity of each wheel, I ω is the wheel moment of inertia, F zσ The vertical force on each wheel during the vehicle's roll and pitch motion is not considered, and μ is the road friction coefficient.

[0040]

[0041] In the formula, m ω is the sprung mass, and h is the height of the sprung center of mass.

[0042] The desired wheel rolling angular velocity ω is calculated from the wheel center velocity dσ :

[0043]

[0044] Where η wfl =η wrl =-1,η wfr =η wrr =1.

[0045] Desired wheel rolling angular velocity ω dσ Derivatives:

[0046]

[0047] In order to control the tire slip rate, the desired wheel angular velocity is set to the angular velocity of the wheel when it is in pure rolling, so the desired tire slip state is Δω σ =ω σ -ω dσ =0, indicating that the tire has no slip.

[0048]

[0049] 1.2 Path tracking model

[0050] In order to enable the vehicle to accurately track the target path, a path tracking control strategy is designed to reduce the heading angle error The lateral position error Δy is minimized. Δy is the lateral deviation between the center of mass of the vehicle and the closest point d on the reference path, R is the arc length along the reference path, and are the desired heading angle on the reference path and the actual heading angle of the car, respectively.

[0051] Actual heading angle and target heading angle The difference is the heading angle error, which is

[0052]

[0053] Based on the Serret-Frenet coordinate system and linearized by the small angle assumption, we can obtain:

[0054]

[0055] Where ρ is the curvature corresponding to the reference path point.

[0056] The lateral position error satisfies the following equation:

[0057]

[0058] Under ideal steady-state conditions, Therefore, according to equations (13) and (14), γ d =v x ρ; v yd = 0, let Δγ = γ - γ d , Δv y =v y -v yd is the desired yaw rate γ d Tracking deviation, expected lateral velocity v yd tracking deviation.

[0059]

[0060]

[0061] make The path tracking model is obtained from equations (11), (13)-(16):

[0062]

[0063] u z =[δ β ,δ γ ,T fl ,T fr ,T rl ,T rr ] T ,

[0064]

[0065]

[0066]

[0067] Step 2: Based on the path tracking model obtained in step 1, redefine the output of the system, and decompose the path tracking model into an input-output subsystem and a zero dynamic subsystem through input-output linearization.

[0068] The specific implementations of the input-output subsystem and the zero dynamic subsystem are as follows:

[0069] In order to control the wheel slip rate while controlling the vehicle's handling stability and prevent excessive wheel slip, the wheel slip state Δω is calculated. σ , the lateral motion of the vehicle Δγ and Δv y As a state variable, we control x z1 =Δv y ,x z2 =Δγ,x z3 =Δω fl ,x z4 =Δω fr ,x z5 =Δω rl ,x z6 =Δω rr , considering the influence of the slip state of the four wheels on the trajectory tracking error and yaw stability, the output of the system is redefined as:

[0070]

[0071] In the formula, the design parameter k i1 , k i2 , k i3is a constant to be determined, i = 1, 2, ..., 6. By selecting appropriate design parameters k i1 , k i2 , k i3 , which can ensure the internal dynamic stability of the system.

[0072] By taking the derivative of equation (18), according to the path tracking model (17), the heading angle error (12) and the lateral position error (13), the input-output subsystem can be obtained:

[0073]

[0074] In the formula,

[0075]

[0076] K z3 =[k 12 ,k 22 ,k 32 ,k 42 ,k 52 ,k 62 ] T

[0077]

[0078] Since the trajectory tracking model (17) is 8-dimensional and the input-output subsystem (19) is only 6-dimensional, the remaining 2-dimensional system states, namely the heading angle following error (13) and the lateral displacement following error (14), constitute the internal subsystem of the trajectory tracking control system. It can be seen that by adopting the method of redefining the system output (18), the path tracking control system (17) is decomposed into the input-output subsystem (19) and the internal subsystems (13)-(14).

[0079] When a specific control input u z , so that when the output z of the input-output subsystem (19) is zero, the internal subsystems (13) and (14) are zero dynamic subsystems. Ensuring the local stability of the zero dynamic subsystem can ensure the local asymptotic stability of the entire closed-loop system. Therefore, the present invention proposes an input-output subsystem control strategy based on adaptive piecewise sliding mode for the input-output subsystem, so that the input-output subsystem converges to zero in a finite time; for the internal subsystem, when a specific control input u z When the output of the input-output subsystem is zero, the internal subsystem is the zero dynamic subsystem. By selecting reasonable controller design parameters, the zero dynamic subsystem is asymptotically stable near the equilibrium point, thereby ensuring the asymptotic stability of the intelligent electric vehicle trajectory tracking control system.

[0080] Step 3: Use the CarSim car model to obtain the real-time parameters of the vehicle: Y-axis coordinates, X-axis coordinates, heading angle, lateral speed and acceleration, longitudinal speed and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc.

[0081] Step 4: Based on the Y-axis coordinates, X-axis coordinates, heading angle, lateral velocity and acceleration, longitudinal velocity and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc. output by the CarSim car model, with the goal of fast convergence of the zero dynamic subsystem state, a controller parameter design method based on Q-learning genetic algorithm optimization is proposed, and the optimized controller parameters are sent to the input-output subsystem adaptive generalized sliding mode control module. The specific implementation method is as follows:

[0082] The 2D state variables of the internal subsystem (heading angle following error The lateral displacement following error Δy) is derivatized twice so that the control u z Appear:

[0083]

[0084]

[0085] General Substituting (20) and (21), we get

[0086]

[0087] In the formula,

[0088]

[0089] From the input and output subsystem, we can see that the system state converges to zero in a finite time, that is, When u z =-(K z1 B z ) -1 [(K z1 A z +K z2 )x z +D z ], and bring it into the internal subsystem (22), we get the zero dynamic subsystem:

[0090]

[0091] make:

[0092]

[0093] We can get:

[0094]

[0095] Substituting equation (24) into (23), we can obtain

[0096]

[0097] From formula (25), we can see that if the matrix [A ο -B ο (K z1 B z ) -1 K N2 K N3 ] is strictly guaranteed to be in the left half plane of the complex plane, then the equilibrium point of the original nonlinear system is asymptotically stable. Therefore, select an appropriate redefine the system output parameter k i1 , k i2 , k i3 , i = 1, 2, ..., 6, so that the matrix [A ο -B ο (K z1 B z ) -1 K N2 K N3 ] are all negative, then the zero dynamic subsystem (25) will be at the equilibrium point The path tracking control system is asymptotically stable.

[0098] In order to better achieve rapid stabilization of the path tracking control system, the present invention takes the rapid convergence of the zero dynamic subsystem state as the goal and proposes a controller parameter design method based on Q-learning genetic algorithm optimization to optimize the set system performance indicators.

[0099] Step 1 Population and parameter initialization

[0100] The parameter k to be optimized i1 , k i2 , k i3 (i=1,2,…,6) represents the gene of the genetic algorithm. All genes are connected in series to form individuals, which are encoded with real numbers. Multiple individuals form a population. The initial population is generated by random method, the population size is Size, and the number of evolution generations is m. G 、Maximum evolutionary generations m Gmax , the number of Q learning iterations is m Q , maximum number of iterations m Qmax , let m G =0, m Q =0, current action a k =0.2, current state s k =x o T, k is the current moment, and other related parameters are initialized.

[0101] Step 2: Penalty factor optimization based on Q-learning

[0102] The quadratic performance index related to the heading control error and its rate of change is used as the optimization objective function:

[0103]

[0104] Where Θ∈R 4×4 is a positive symmetric matrix.

[0105] At the same time, each individual in the population must satisfy [A ο -B ο (K z1 B z ) -1 K N2 K N3 ](25)The real part of the eigenvalue is less than zero:

[0106] I Γ =Re[λ Γ (A o )]<0 (27)

[0107] Where Γ is an integer, 1≤Γ≤4. Γ (A o ) represents the matrix A o The Γth eigenvalue of .

[0108] According to the optimization objective function (26) and the inequality constraint (27), the following penalty function (28) is selected:

[0109]

[0110] In the formula, β J (m G ) is the penalty factor, β J (m G )=a k , m G For evolutionary algebra.

[0111] In order to strike a balance between the objective function and the constraints, the present invention adopts the Q learning method to optimize the penalty factor in equation (28) in real time. The specific implementation steps are as follows:

[0112] Step 2.1: Calculate s k Execute action a in state k Q k (s k ,a k )value

[0113] Let the fitness function:

[0114] F(m G )=1 / J β (m G )

[0115] According to the current status k and action a k Calculate the fitness values ​​of all individuals in the current population, and then use formula (29) to calculate Q k (s k ,a k ), Q k (s k ,a k ) is in s k Execute action a in state k Q value.

[0116]

[0117] In the formula, ∑F feasible Represents the sum of the fitness function values ​​of all feasible solutions in the population, num feasible represents the number of feasible solutions in the population, max(F Ξ ) represents the maximum fitness function value of individuals in the population, Ξ is an integer, 0<Ξ<size, size is the population size, ∑sum viol The total number of individuals violating the constraint, ∑mum viol The total number of individuals violating the constraint.

[0118] Step 2.2: Calculate the next state s k+1 and the immediate return value R k

[0119] According to the current status k , bring the design parameters corresponding to each individual in the population into the zero dynamic subsystem (25), and calculate the state s at the next moment k+1 .

[0120] Executions k+1 All possible actions corresponding to k a is the number of all possible actions, Υ=1,2,...,k a , according to formula (30) to calculate the immediate reward value R k ;

[0121]

[0122] In the formula, ε J is a small positive number, r (k+1,Υ) For k+1 Execute A in state k+1A single reward value obtained when taking any action in .

[0123] Step 2.3: Update Q k (s k ,a k ) value, and obtain the optimal action a k *

[0124] Order k+1 Any action among all possible actions corresponding to the state is a′, and s k+1 Substitute a′ into equations (28) and (29) to calculate Q k (s k+1 ,a′). Combined with Q k (s k ,a k ), update Q using equation (31) k (s k ,a k ).

[0125]

[0126] Where ε1 is the learning factor and ε2 is the discount factor.

[0127] Use a greedy strategy to select the optimal action:

[0128]

[0129] Order k =s k+1 , a k =a k * , m Q =m Q +1.

[0130] Step 3 Termination Criteria 1

[0131] If the number of Q learning iterations is m Q Equal to the maximum number of Q learning iterations m Qmax , then the iteration ends and outputs a k , jump to step 4; if the number of Q learning iterations is m Q Not equal to the maximum number of Q learning iterations m Qmax , return to step 2.1.

[0132] Step 4: Selection, adaptive crossover and mutation operations

[0133] A kSubstitute into formula (28), calculate the fitness function value of each individual, arrange the population in order from best to worst according to the fitness function value, and determine the selection probability of the individual according to the roulette method, thus forming the paternal parent.

[0134] Using the average, maximum, and minimum fitness function values ​​F of the current population avg (m G ), F max (m G ), F min (m G ), estimate the individual diversity of the current population:

[0135] f d (m G )=F avg (m G ) / [ε+F max (m G )-F min (m G )] 0≤m G ≤m Gmax

[0136] In the formula, m G is the evolutionary algebra; ε is a small positive number to ensure that no singularity occurs.

[0137] Design adaptive crossover and mutation operators P c , P m , so that it can be adaptively adjusted according to population diversity and evolutionary generations:

[0138]

[0139]

[0140] In the formula, m Gmax is the maximum evolutionary generation; P m0 ,P c0 ∈[0,1],b1,b2∈R + , can be any value. In this way, when the population distribution is relatively concentrated (i.e., tends to local convergence, f d (m G ) is larger), p m will increase, and p c decreases, otherwise the opposite; in the overall trend, p m and p c All of them decrease slowly with the genetic process.

[0141] The population is updated by performing selection, crossover, and mutation operations on the population.

[0142] Step 5 Generate a new generation of population

[0143] The original population and the updated population after selection, crossover and mutation operations are sorted according to the fitness value, and the top 50% are selected as the new generation population.

[0144] Let m G =m G +1, m Q =0.

[0145] Step 6 Termination Criteria 2

[0146] If the number of generations of genetic evolution is m G Equal to the maximum evolutionary number m Gmax , then the iteration ends and the controller parameter k is output i1 , k i2 , k i3 (i=1,2,…,6); if the genetic evolution generation m G Not equal to the maximum evolutionary number m Gmax , skip to step 2.1.

[0147] Step 5: Based on the given reference trajectory, the input-output subsystem obtained in step 2, the Y-axis coordinate, X-axis coordinate, heading angle, lateral velocity and acceleration, longitudinal velocity and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc. output by the CarSim car model in step 3, and the controller parameters obtained in step 4, the four-wheel steering angle and four-wheel drive torque are obtained through the input-output subsystem adaptive generalized sliding mode control module, and input into the CarSim car model. The specific implementation is as follows:

[0148] For the input-output subsystem (19), the following generalized sliding surface is adopted:

[0149]

[0150] In the formula, s g =[s g1 ,s g2 ,s g3 ,s g4 ,s g5 ,s g6 ] T , c s =diag(c s1 ,c s2 ,c s3 ,c s4 ,c s5 ,c s6 ), Γ=diag(G1,G2,G3,G4,G5,G6),

[0151] is the differential operator, csi >0 is the design parameter, i=1,2,...,6.

[0152] Select a complementary sliding surface in space that is orthogonal to equation (34):

[0153]

[0154] In the formula, s h =[s h1 ,s h2 ,s h3 ,s h4 ,s h5 ,s h6 ] T .

[0155] Define s = s g +s h , from (34) and (35) we can get:

[0156]

[0157] For the input-output subsystem (19), the following adaptive generalized sliding mode control strategy is proposed:

[0158]

[0159] In the formula, sgn(s)=[sgn(s1),sgn(s2),sgn(s3),sgn(s4),sgn(s5),sgn(s6)] T , D z The estimated upper bounds of the elements in the matrix, The adaptive rate is designed as follows:

[0160]

[0161] Among them, α D is the design parameter, α D =diag(α D1 ,α D2 ,α D3 ,α D4 ,α D5 ,α D6 ), α Di >0, s|=diag(|s1|,|s2|,|s3|,|s4|,|s5|,|s6|).

Claims

1. A path tracking control method for intelligent electric vehicles based on Q-learning genetic algorithm, characterized in that: The method comprises the following steps: Step 1: Based on the kinematics and dynamics of the vehicle, a path tracking model of the four-wheel steering and four-wheel independent drive smart electric vehicle is established; Step 2: Based on the path tracking model obtained in step 1, redefine the output of the system, and decompose the path tracking model into an input-output subsystem and a zero dynamic subsystem by input-output linearization; Step 3: Use the CarSim car model to obtain the real-time parameters of the vehicle: Y-axis coordinates, X-axis coordinates, heading angle, lateral speed and acceleration, longitudinal speed and acceleration, heading angular velocity, rolling angular velocity of each wheel, and vertical force of each wheel; Step 4: Based on the Y-axis coordinates, X-axis coordinates, heading angle, lateral velocity and acceleration, longitudinal velocity and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel, etc. output by the CarSim car model in step 3, a controller parameter design method based on Q-learning genetic algorithm optimization is proposed with the goal of rapid convergence of the zero dynamic subsystem state to obtain the optimized controller parameters; Step 5. According to the given reference trajectory, the input-output subsystem obtained in step 2, the Y-axis coordinate, X-axis coordinate, heading angle, lateral velocity and acceleration, longitudinal velocity and acceleration, heading angular velocity, rolling angular velocity of each wheel, vertical force of each wheel output by the CarSim car model in step 3, and the controller parameters obtained in step 4, the four-wheel steering angle and four-wheel drive torque are obtained through the input-output subsystem adaptive generalized sliding mode control module, and input them into the CarSim car model.

2. The intelligent electric vehicle path tracking control method based on Q learning genetic algorithm as claimed in claim 1, characterized in that: The process of establishing the path tracking model of the four-wheel steering and four-wheel independent drive smart electric vehicle described in step 1 is as follows: 1.1 Vehicle dynamics model In the design of the controller, the handling dynamics model of the four-wheel independent drive intelligent electric vehicle is: Among them, v x and v y are the longitudinal and lateral velocities of the vehicle, respectively; r is the yaw rate; m and I z is the vehicle mass and yaw inertia; F yf and F yr is the lateral force of the front and rear tires; δ f and δ r is the front and rear wheel steering angle; l f and l r are the distances from the center of mass of the vehicle to the front axle and the rear axle respectively; ΔM z The additional yaw moment generated by the longitudinal force difference between the left and right tires is: Among them, T qσ is the driving torque of each wheel, σ=fl, fr, rl, rr, representing the left front wheel, right front wheel, left rear wheel, right rear wheel, t wσ Satisfy t wfl =t wfr =t wf , t wrl =t wrr =t wr , t wf ,t wr are the wheelbases of the front and rear wheels of the vehicle, R e is the wheel rolling radius; When the lateral acceleration is less than 0.4g (g is the acceleration of gravity), the slip angle does not exceed 4° to 5°. The relationship between the tire lateral force and the slip angle can be expressed as a linear relationship, which can be written as: F yf =-k f a f ,F yr =-k r a r (4) Among them, k f and k r are the cornering stiffness of the front and rear tires respectively; α f and α r is the front and rear tire slip angle, defined as: Assuming that the steering angles of the front and rear wheels are small, cosδ in equations (1) and (2) is f ≈cosδ r ≈1; Substituting equations (4) and (5), the vehicle model described by equations (1) and (2) is simplified to the following linear dynamic equation: Wheel dynamics equation: In the formula, ω σ is the rolling angular velocity of each wheel, I ω is the wheel moment of inertia, F zσ is the vertical force of each wheel when the vehicle rolls and pitches, μ is the road friction coefficient; In the formula, m ω is the sprung mass, h is the height of the sprung mass center; The desired wheel rolling angular velocity ω is calculated from the wheel center velocity dσ : where η wfl = η wrl = -1, η wfr = η wrr = 1; Desired wheel rolling angular velocity ω dσ Derivatives: In order to control the tire slip rate, the desired wheel angular velocity is set to the angular velocity of the wheel when it is in pure rolling, so the desired tire slip state is Δω σ =ω σ -ω dσ =0, indicating that the tire has no slip; 1.2 Path tracking model In order to enable the vehicle to accurately track the target path, a path tracking control strategy is designed to reduce the heading angle error The lateral position error Δy is minimized; Δy is the lateral deviation between the center of mass of the vehicle and the nearest point d on the reference path, R is the arc length along the reference path, and They are the desired heading angle on the reference path and the actual heading angle of the car; Actual heading angle and target heading angle The difference is the heading angle error, which is Based on the Serret-Frenet coordinate system and linearized by the small angle assumption, we can obtain: Where ρ is the curvature corresponding to the reference path point; The lateral position error satisfies the following equation: Under ideal steady-state conditions, Therefore, according to equations (13) and (14), γ d =v x ρ; v yd = 0, let Δγ = γ - γ d , Δv y =v y -v yd is the desired yaw rate γ d Tracking deviation, expected lateral velocity v yd tracking deviation; make The path tracking model is obtained from equations (11), (13)-(16): u z =[δ β ,δ γ ,T fl ,T fr ,T rl ,T rr ] T , 3. The intelligent electric vehicle path tracking control method based on Q learning genetic algorithm as claimed in claim 1, characterized in that: The implementation process of redefining the system output by step 2 to decompose the path tracking model into input-output subsystem and zero dynamic subsystem is as follows: In order to control the wheel slip rate while controlling the vehicle's handling stability and prevent excessive wheel slip, the wheel slip state Δω is calculated. σ , the lateral motion of the vehicle Δγ and Δv y As a state variable, control; let x z1 =Δv y ,x z2 =Δγ,x z3 =Δω fl ,x z4 =Δω fr ,x z5 =Δω rl ,x z6 =Δω rr , considering the influence of the slip state of the four wheels on the trajectory tracking error and yaw stability, the output of the system is redefined as: In the formula, the design parameter k i1 , k i2 , k i3 is an undetermined constant, i=1,2,…,6; by selecting a suitable design parameter k i1 , k i2 , k i3 , which can ensure the internal dynamic stability of the system; By taking the derivative of equation (18), according to the path tracking model (17), the heading angle error (12) and the lateral position error (13), the input-output subsystem can be obtained: In the formula, K z3 =[k 12 ,k 22 ,k 32 ,k 42 ,k 52 ,k 62 ] T 4. The intelligent electric vehicle path tracking control method based on Q learning genetic algorithm as claimed in claim 1, characterized in that: The implementation process of the controller parameter design method based on Q learning genetic algorithm optimization described in step 4 is as follows: The 2D state variables of the internal subsystem (heading angle following error The lateral displacement following error Δy) is derivatized twice so that the control u z Appear: General Substituting (20) and (21), we get In the formula, From the input and output subsystem, we can see that the system state converges to zero in a finite time, that is, When u z =-(K z1 B z ) -1 [(K z1 A z +K z2 )x z +D z ], and bring it into the internal subsystem (22), we get the zero dynamic subsystem: make: We can get: Substituting equation (24) into (23), we can obtain From formula (25), we can see that if the matrix [A ο -B ο (K z1 B z ) -1 K N2 K N3 ] is strictly guaranteed to be in the left half plane of the complex plane, then the equilibrium point of the original nonlinear system is asymptotically stable; therefore, select an appropriate redefine the system output parameter k i1 , k i2 , k i3 , i = 1, 2, ..., 6, so that the matrix [A ο -B ο (K z1 B z ) -1 K N2 K N3 ] are all negative, then the zero dynamic subsystem (25) will be at the equilibrium point The path tracking control system is asymptotically stable. In order to better achieve rapid stabilization of the path tracking control system, a controller parameter design method based on Q-learning genetic algorithm optimization is proposed with the goal of rapid convergence of the zero dynamic subsystem state, so that the set system performance indicators are optimal. Step 1 Population and parameter initialization The parameter k to be optimized i1 , k i2 , k i3 (i=1,2,…,6) represents the gene of the genetic algorithm. All genes are connected in series to form individuals, which are encoded with real numbers. Multiple individuals form a group. The initial population is generated randomly, the population size is Size, and the number of evolution generations is m. G 、Maximum evolutionary generations m Gmax , the number of Q learning iterations is m Q , maximum number of iterations m Qmax , let m G =0, m Q =0, current action a k =0.2, current state s k =x o T , k is the current moment, initialize other related parameters; Step 2: Penalty factor optimization based on Q-learning The quadratic performance index related to the heading control error and its rate of change is used as the optimization objective function: minf J =exp(s k Θs k T ) (26) Where Θ∈R 4×4 is a positive definite symmetric matrix; At the same time, each individual in the population must satisfy [A ο -B ο (K z1 B z ) -1 K N2 K N3 ](25)The real part of the eigenvalue is less than zero: I Γ =Re[λ Γ (A o )]<0 (27) Where Γ is an integer, 1≤Γ≤4. Γ (A o ) represents the matrix A o The Γth eigenvalue of ; According to the optimization objective function (26) and the inequality constraint (27), the following penalty function (28) is selected: In the formula, β J (m G ) is the penalty factor, β J (m G )=a k , m G for evolutionary algebra; In order to strike a balance between the objective function and the constraints, the present invention adopts the Q learning method to optimize the penalty factor in equation (28) in real time. The specific implementation steps are as follows: Step 2.1: Calculate s k Execute action a in state k Q k (s k ,a k )value Let the fitness function: F(m G )=1 / J β (m G ) According to the current status k and action a k Calculate the fitness values ​​of all individuals in the current population, and then use formula (29) to calculate Q k (s k ,a k ), Q k (s k ,a k ) is in s k Execute action a in state k Q value; In the formula, ∑F feasible Represents the sum of the fitness function values ​​of all feasible solutions in the population, num feasible represents the number of feasible solutions in the population, max(F Ξ ) represents the maximum fitness function value of individuals in the population, Ξ is an integer, 0<Ξ<size, size is the population size, ∑sum viol The total number of individuals violating the constraint, ∑mum viol The total number of individuals violating the constraint; Step 2.2: Calculate the next state s k+1 and the immediate return value R k According to the current status k , bring the design parameters corresponding to each individual in the population into the zero dynamic subsystem (25), and calculate the state s at the next moment k+1 ; Executions k+1 All possible actions corresponding to k a is the number of all possible actions, Υ=1,2,...,k a , according to formula (30) to calculate the immediate reward value R k ; In the formula, ε J is a small positive number, r (k+1,γ) For k+1 Execute A in state k+1 A single reward value obtained when taking any action in Step 2.3: Update Q k (s k ,a k ) value, and obtain the optimal action a k * Order k+1 Any action among all possible actions corresponding to the state is a′, and s k+1 Substitute a′ into equations (28) and (29) to calculate Q k (s k+1 ,a′); combined with Q k (s k ,a k ), update Q using equation (31) k (s k ,a k ); In the formula, ε1 is the learning factor and ε2 is the discount factor; Use a greedy strategy to select the optimal action: Let s k = s k+1 , a k = a k * , m Q = m Q + 1; Step 3 Termination Criteria 1 If the number of Q learning iterations is m Q Equal to the maximum number of Q learning iterations m Qmax , then the iteration ends and outputs a k , jump to step 4; If the number of Q learning iterations is m Q Not equal to the maximum number of Q learning iterations m Qmax , return to step 2.1; Step 4: Selection, adaptive crossover and mutation operations A k Substitute into formula (28), calculate the fitness function value of each individual, arrange the population in order from best to worst according to the fitness function value, and determine the selection probability of the individual according to the roulette method, thus forming the father parent; Using the average, maximum, and minimum fitness function values ​​F of the current population avg (m G ), F max (m G ), F min (m G ), estimate the individual diversity of the current population: f d (m G )=F avg (m G ) / [ε+F max (m G )-F min (m G )] 0≤m G ≤m Gmax In the formula, m G is the evolutionary algebra; ε is a small positive number to ensure that no singularity occurs; Design adaptive crossover and mutation operators P c , P m , so that it can be adaptively adjusted according to population diversity and evolutionary generations: In the formula, m Gmax is the maximum evolutionary generation; P m0 ,P c0 ∈[0,1],b1,b2∈R + , can be any value; thus, when the population distribution is relatively concentrated (i.e., tends to local convergence, f d (m G ) is larger), p m will increase, and p c decreases, otherwise the opposite; in the overall trend, p m and p c All of them decrease slowly with the genetic process; The population is updated by performing selection, crossover, and mutation operations on the population; Step 5 Generate a new generation of population Sort the original population and the updated population after selection, crossover, and mutation operations according to the fitness value, and select the top 50% as the new generation population; Let m G =m G +1, m Q =0; Step 6 Termination Criteria 2 If the number of generations of genetic evolution is m G Equal to the maximum evolutionary number m Gmax , then the iteration ends and the controller parameter k is output i1 , k i2 , k i3 (i=1,2,…,6); if the genetic evolution generation m G Not equal to the maximum evolutionary number m Gmax , skip to step 2.

1.

5. The intelligent electric vehicle path tracking control method based on Q learning genetic algorithm as claimed in claim 1, characterized in that: The implementation process of step five is as follows: For the input-output subsystem (19), the following generalized sliding surface is adopted: In the formula, s g =[s g1 ,s g2 ,s g3 ,s g4 ,s g5 ,s g6 ] T , c s =diag(c s1 ,c s2 ,c s3 ,c s4 ,c s5 ,c s6 ), Γ=diag(G1,G2,G3,G4,G5,G6), is the differential operator, c si >0 is the design parameter, i=1,2,...,6; Select a complementary sliding surface in space that is orthogonal to equation (34): In the formula, s h =[s h1 ,s h2 ,s h3 ,s h4 ,s h5 ,s h6 ] T ; Define s = s g +s h , from (34) and (35) we can get: For the input-output subsystem (19), the following adaptive generalized sliding mode control strategy is proposed: In the formula, sgn(s)=[sgn(s1),sgn(s2),sgn(s3),sgn(s4),sgn(s5),sgn(s6)] T , D z The estimated upper bounds of the elements in the matrix, The adaptive rate is designed as follows: Among them, a D For design parameters, a D =diag(a D1 ,a D2 ,a D3 ,a D4 ,a D5 ,a D6 ),a Di >0, s|=diag(s1|,|s2|,|s3|,|s4|,|s5|,|s6|).

Citation Information

Patent Citations

  • Multi-objective hierarchical collaborative control and optimization method for vehicles with driving intentions

    CN108773376A

  • Multi-agent-based track tracing control method of reconfigurable modular flexible manipulator

    CN109240092A

  • Intelligent electric vehicle path tracking model prediction control method

    CN109795502A

  • And inputting saturated automatic driving automovable path tracking control method

    CN111176302A

  • Autonomous vehicle path tracking control method based on high-speed nonsingular switching sliding mode

    CN117227744A