Path tracking control method fusing reinforcement learning adaptive preview

Abstract

The invention relates to a path tracking control method fusing reinforcement learning and adaptive preview, which is used for coping with the change of a dynamic environment and solving the problem of complex multi-target path tracking. The invention relates to the field of intelligent driving. The method comprises an upper layer and a lower layer, the upper layer determines an adaptive preview distance through reinforcement learning and inputs the distance to the lower layer, and the lower layer determines a control action through optimization solution and controls a vehicle to realize intelligent path tracking. Wherein the upper layer comprises an information acquisition module, an information processing module and a reinforcement learning module, and the lower layer comprises a reference trajectory sequence generation module, a model prediction control module and a transverse action conversion module. The upper layer reinforcement learning module optimizes an adaptive preview distance through interaction with a driving environment; and the lower reference trajectory sequence generation module outputs a reference trajectory sequence based on the self-adaptive preview distance and inputs the reference trajectory sequence into the model prediction control module to obtain control output, the control output is converted into control action through the transverse action conversion module, and the vehicle is controlled to realize intelligent path tracking.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving, and in particular is a path tracking control method integrating reinforcement learning and adaptive preview. Technical Background

[0002] With the continuous development of intelligent automobile technology and computer technology, the automation level of autonomous driving systems is also gradually improving. As one of the core technologies, the goal of vehicle trajectory tracking control is to achieve real-time and accurate tracking of the desired trajectory, thereby effectively improving driving safety.

[0003] The preview distance plays a vital role in trajectory tracking performance. Compared with the path tracking control method that uses a fixed preview distance, the adaptive preview distance can reasonably select the preview distance according to external factors such as different path curvatures, thereby significantly improving the tracking effect. In complex road conditions or high-dynamic environments, the adaptive preview strategy can also effectively improve the vehicle's handling stability, safety, and driving comfort.

[0004] At present, in the research of adaptive preview distance, many methods have been proposed, each with its own advantages but also limitations. Patent CN 118707953 uses a trained BP neural network to output the optimal preview distance in real time according to vehicle speed, road adhesion coefficient and road curvature, significantly improving trajectory tracking performance, but requires calibration of a large amount of data to adapt to different scenarios, and the training cost is high. Patent CN 116795110 simulates human driving response based on preview time control of anthropomorphic evaluation function, but the parameter calibration is complex and it is difficult to flexibly respond to diverse scenarios. Patent CN 107097785 uses fuzzy control to achieve adaptive preview, which has strong adaptability, but the control rules are relatively complex and may not work well under extreme conditions. Patent CN116853293 establishes an adaptive preview distance model based on road curvature, safe speed and empirical coefficient. When the road curvature changes rapidly, the preview distance will fluctuate greatly. While improving trajectory tracking performance, the above methods also face challenges in adaptability, complexity and cost. Summary of the invention

[0005] In view of the shortcomings of the prior art and in order to solve the problems existing in the above-mentioned technical background, the present invention provides a path tracking control method integrating reinforcement learning with adaptive preview. This method uses reinforcement learning to interact with the environment to dynamically adjust strategies to adapt to diverse scenarios without the need for cumbersome parameter calibration. In addition, reinforcement learning adopts an end-to-end optimization method, directly optimizing the whole from input to output, avoiding the complexity of traditional control rule design. Compared with supervised learning, reinforcement learning autonomously explores the optimal strategy through a trial-and-error mechanism, reducing data calibration and training costs, and at the same time has strong generalization capabilities and can adapt to new scenarios through migration or continuous learning. Adaptive adjustment of the preview distance is achieved by maximizing the cumulative return.

[0006] Based on the adaptive preview distance to determine the target reference trajectory sequence, combined with the model predictive control algorithm, the reference trajectory sequence is tracked, which can effectively cope with dynamic environmental changes and solve complex multi-target path tracking problems, thereby realizing intelligent path tracking control of the vehicle.

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0008] The present invention is a path tracking control method integrating reinforcement learning and adaptive preview, the method comprising an upper layer optimizing the preview distance through reinforcement learning and inputting it to a lower layer, the lower layer determining the control action through model predictive control optimization solution, and controlling the vehicle to realize intelligent driving; wherein the upper layer comprises an information acquisition module, an information processing module, and a reinforcement learning module, and the lower layer comprises a reference trajectory sequence generation module, a model predictive control module, and a lateral action conversion module.

[0009] The method comprises the following steps:

[0010] Step 1: Driving environment information collection:

[0011] The information acquisition module is responsible for acquiring vehicle status information and expected path information, providing key support for path tracking. Vehicle status information includes the lateral and longitudinal position coordinates, heading angle, and longitudinal and lateral speed of the vehicle, reflecting the real-time motion status of the vehicle; expected path information includes the coordinates of the expected path points and the expected heading angle, providing the target trajectory and heading that the vehicle needs to track. By collecting status information in real time, the system can provide accurate data support for the control algorithm.

[0012] Step 2: Information processing:

[0013] The information processing module is used to obtain the current comprehensive deviation and the path curvature at the initial preview point according to the current vehicle state information and the expected path information; specifically, the comprehensive deviation includes the lateral deviation y between the current position of the vehicle and the expected path. e and yaw angle deviation θ e , due to the lateral deviation ye and yaw angle deviation θ e The units are different, and after normalization and weighted calculation, the comprehensive deviation C is obtained. e , as shown in equations (1) to (5),

[0014] y e =yy ref (1)

[0015] θ e =θ-θ ref (2)

[0016]

[0017]

[0018]

[0019] in, and are the lateral deviation and yaw angle deviation after normalization respectively; emax and θ emax are the maximum lateral deviation and yaw angle deviation respectively; emin and θ emin are the minimum lateral deviation and yaw angle deviation respectively; κ is the weight coefficient.

[0020] The information processing module is based on the current longitudinal vehicle speed v x and preview factor k p Determine the initial preview distance L i , that is, L i =k p v x , and then obtain L by table lookup method i The path curvature ρ i .

[0021] Step 3: Design reinforcement learning module:

[0022] Step 3.1, state space design:

[0023] The state space of the reinforcement learning module is defined as s t ={C e ,ρ i ,v x ,v y}; Among them, C e is the comprehensive deviation, ρ i is the initial preview distance L i The path curvature at v x and v y are the longitudinal speed and lateral speed of the vehicle respectively; the action space of the reinforcement learning module is defined as at =[L c ], L c It is the adaptive preview distance.

[0024] Step 3.2, reward function design:

[0025] The reinforcement learning module is trained by interacting with the driving environment. The present invention designs a corresponding reward function based on the vehicle tracking expected path accuracy. The reward function of the reinforcement learning module is defined as follows:

[0026]

[0027] In the formula, C e is the comprehensive deviation, p1 is the reward amplification factor, and p2 is the reward penalty number for failure in the tracking path.

[0028] Step 4: Generate reference trajectory sequence:

[0029] The reference trajectory sequence generation module determines the path information at the preview point according to the adaptive preview distance, and generates the reference trajectory sequence according to the current vehicle speed and the predicted time domain. in, Represents the expected heading angle at time s+1 in the current state s, and so on to time s+n; in, Represents the expected lateral displacement at time s+1 under the current state at time s, and so on to time s+n.

[0030] Step 5: Design model predictive control module:

[0031] Step 5.1: Establish a prediction model:

[0032] The prediction model is established using the vehicle kinematics model, vehicle dynamics model and nonlinear tire model, as shown in formula (7):

[0033]

[0034] In the formula, x, y are the horizontal and vertical position coordinates of the vehicle, v x and v y are the longitudinal and lateral speeds of the vehicle, θ is the heading angle of the vehicle, and F y,f is the lateral force of the front wheel, F y,r is the rear wheel lateral force, m is the vehicle mass, γ is the vehicle yaw rate, a, b is the distance from the vehicle center of mass to the front and rear axles, I z The vehicle's yaw moment of inertia.

[0035] Step 5.2: Model discretization:

[0036] The above model is discretized using the fourth-order Runge-Kutta method, with a sampling time of T s , the input-output relationship of the incremental discrete system model can be expressed as formula (8),

[0037]

[0038] In the formula, u(s)=[δ f ],

[0039] Among them, ξ(s) is the state variable of the vehicle at the current time s, u(s) and △u(s) are the control input and control input increment of the system at the current time s, respectively, and y c (s) is the predicted output of the system at the current time s, and C is the coefficient matrix used to determine the number of predicted outputs of the system.

[0040] Step 5.3, optimization problem construction:

[0041] The optimization problem construction takes into account the ride comfort and control action stability, imposes hard constraints on the control action, and defines the optimization objective of the model predictive control, such as Equation (9) and Equation (10).

[0042]

[0043]

[0044] Where J1 is the objective function representing the weighted combination of yaw angle and lateral displacement tracking deviation, Γ y is the predicted output weight matrix, Based on the preview point L i is the reference heading angle for the next n steps; J2 is the objective function of the control input increment, Γ u is the control input incremental weight matrix, δ fmin ,δ fmax Represent the minimum steering angle and the maximum steering angle respectively; △δ fmin ,△δ fmax Represent the sampling time T s The minimum and maximum steering angle changes allowed within the range.

[0045] Step 5.4, dynamic optimization solution:

[0046] Dynamic optimization is to solve the optimization problem and obtain the optimal control instruction u at the current moment. o =[δ o ],δ o is the front wheel turning angle.

[0047] Step 6: Design the horizontal action conversion module:

[0048] The lateral motion control module takes the optimal control instruction u solved by the model predictive control module as o =[δ o ] is converted into the PWM signal required by the steering motor to control the vehicle to achieve intelligent path tracking.

[0049] The beneficial effects of the present invention are as follows: the present invention relates to a path tracking control method integrating reinforcement learning adaptive preview, the method comprising an upper and lower layer, the upper layer outputting an adaptive preview distance by integrating a reinforcement learning algorithm, and the lower layer using the preview point information determined by the adaptive preview distance as the expected information of the model predictive control, so that the method can maximize the degree of exploration of the path tracking control strategy while ensuring safety, and avoid falling into the dilemma of local optimality, and at the same time, the method improves the generalization and robustness of the path tracking control strategy, and has good application prospects in the field of intelligent vehicle path tracking control. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the adaptive preview distance hierarchical control based on reinforcement learning of the present invention. DETAILED DESCRIPTION

[0051] The present invention is described in detail below with reference to the accompanying drawings.

[0052] The present invention proposes a path tracking control method that integrates reinforcement learning and adaptive preview. The upper layer uses the information acquisition module to collect vehicle and expected path information, and obtains the comprehensive deviation and path curvature as the input of the reinforcement learning method through the information processing module to train the reinforcement learning module. The lower layer uses the reference trajectory sequence generation module to receive the expected path information and the adaptive preview distance to generate a reference trajectory sequence as the expected tracking template of the model predictive control module, and obtains the control output through dynamic optimization solution. It is converted into the optimal control action through the lateral action conversion module, so that the vehicle completes the path tracking task. Reinforcement learning uses the SAC method. Refer to Figure 1 The schematic diagram specifically includes the following steps:

[0053] Step 1: Information collection module:

[0054] The intelligent vehicle information collection module is responsible for obtaining vehicle status information C t ={x,y,θ,v x ,v y} and expected path information P t ={x ref ,y ref ,θ ref}. Among them, x, y are the horizontal and vertical position coordinates of the vehicle, θ is the heading angle of the vehicle, and v x and v y are the longitudinal and lateral speeds of the vehicle, x ref With y ref are the coordinates of the desired path points, θ ref is the desired heading angle. By collecting status information in real time, the system can provide accurate data support for the control algorithm, realize the adaptive adjustment of the vehicle to the preview distance, and accurately track the reference trajectory through the model predictive control algorithm, and finally realize intelligent path tracking in complex dynamic environments.

[0055] Step 2: Information processing module:

[0056] It is used to obtain the current comprehensive deviation and the path curvature at the initial preview point according to the current vehicle state information and the expected path information; specifically, according to the lateral deviation y between the current position information of the vehicle and its corresponding expected path e and yaw angle deviation θ e , due to the lateral deviation y e and yaw angle deviation θ e The units are different, and after normalization and weighted calculation, the comprehensive deviation C is obtained. e , as shown in equations (11) to (15),

[0057] y e =yy ref (11)

[0058] θ e =θ-θ ref (12)

[0059]

[0060]

[0061]

[0062] Among them, y is the longitudinal position coordinate of the vehicle, θ is the heading angle of the vehicle, and are the lateral deviation and yaw angle deviation after normalization respectively; emax and θ emax are the maximum lateral deviation and yaw angle deviation respectively; emin and θ emin are the minimum lateral deviation and yaw angle deviation respectively; κ is the weight coefficient.

[0063] The information processing module is based on the current longitudinal vehicle speed v x and preview factor k p Determine the initial preview distance Li , that is, L i =k p v x , and then obtain L by table lookup method i The path curvature ρ i .

[0064] Step 3: Design reinforcement learning module:

[0065] Step 3.1, state space design:

[0066] The state space of the reinforcement learning module is defined as s t ={C e ,ρ i ,v x ,v y}; Among them, C e is the comprehensive deviation, ρ i is the initial preview distance L i The path curvature at v x and v y are the longitudinal speed and lateral speed of the vehicle respectively; the action space of the reinforcement learning module is defined as a t =[L c ], L c It is the adaptive preview distance.

[0067] Step 3.2, action space design:

[0068] In order to achieve the output of adaptive preview distance, the action space of the reinforcement learning module is defined as a single preview distance control action. This action outputs the ideal preview distance of the vehicle through the reinforcement learning algorithm to ensure that the vehicle can achieve good tracking effect under different target paths. The output value of the preview distance will be dynamically adjusted according to the current comprehensive deviation and path curvature, and the interval is set to [d min ,d max ], where d min is the minimum preview distance, d max is the maximum preview distance. Through this single output action, the reinforcement learning algorithm can optimize the adjustment strategy of the preview distance, thereby achieving adaptive adjustment of the preview distance.

[0069] According to the actual situation, the preview distance limit is limited to d min =1,d max =20.

[0070] Step 3.3, reward function design:

[0071] The reinforcement learning module is trained by interacting with the formal environment. Based on the expected path accuracy of vehicle tracking, the present invention designs a corresponding reward function. The reward function of the reinforcement learning module is defined as formula (16);

[0072]

[0073] In the formula, C e For comprehensive deviation, the reward amplification factor is p1=10, and the reward penalty number for failure in the tracking path midway is p2=200.

[0074] Step 3.4, experience playback:

[0075] In deep reinforcement learning methods, the experience replay mechanism plays an important role in accelerating the training process and achieving algorithm convergence. Experience replay enables the agent to repeatedly use historical experience, thereby improving the utilization efficiency of experience samples. This scheme defines an experience pool and stores the training data at each moment in the pool. When the number of experiences in the experience pool reaches the preset value, a certain number of experience samples are randomly selected from the experience pool to form a batch for training. Specifically, at the kth moment in the tth round of training, N experiences are randomly selected from the experience pool to form an experience batch for training, where N is set to 64.

[0076] Step 3.5. Select reinforcement learning algorithm:

[0077] Different reinforcement learning methods have their own advantages and disadvantages. The present invention selects the SAC algorithm as the reinforcement learning algorithm in step 3 to complete the adaptive preview distance optimization task. The SAC algorithm can select the optimal action in the continuous action space, and by introducing the maximum entropy mechanism, the strategy learned by the agent is more random, which helps to improve the generalization ability of the control strategy.

[0078] Step 3.6, reinforcement learning module training:

[0079] Set the regularization parameter α = -3, the soft update step size τ = 0.005, the discount factor γ = 0.99, and initialize the neural network parameters in the SAC algorithm, including the Actor network parameter θ actor 、Critic1 network parameters θ critic1 、Critic2 network parameters θ critic2 、Target Critic1 network parameters θ targetcritic1 and target Critic2 network parameters θ targetcritic2 In the tth training round, the vehicle speed, comprehensive deviation and path curvature are obtained from the information processing module in step 2, and the current state s in step 3.1 is formed. t This state is input into the Actor network, and after global optimization, the action a in step 3.2 is output.t , the action is within the predetermined action space. Execute action a t After that, the information collection module returns the reward value r according to the current state t and the new state s t+1 Then, the state s t 、Action a t , reward value r t and the new state s t+1 Store them in the experience pool in step 3.4. When the number of experiences in the experience pool reaches 512, randomly extract 128 experiences from the pool. Input these experiences into the Critic1 network, Critic2 network, target Critic1 network, and target Critic2 network, and calculate Q respectively. critic1 , Q critic2 , target value Q targetcritic1 and Q targetcritic2 , and use formula (17) to calculate the timing difference (TD) target value.

[0080] η t =r t +γmin{Q targetcritic1 ,Q targetcritic2}-αlogπ(a t+1 |s t+1 )(17)

[0081]

[0082]

[0083] By minimizing the loss functions in equations (18) and (19), the Critic1 network and the Critic2 network are updated.

[0084] Sampling actions using reparameterization

[0085]

[0086] The Actor network is updated using the loss function in equation (20).

[0087] The target Critic1 network and the target Critic2 network are updated using equations (21) and (22).

[0088] Q targetcritic1 ←τQ critic1 +(1-τ)Q targetcritic1 (twenty one)

[0089] Q targetcritic1 ←τQ critic2 +(1-τ)Q targetcritic1(twenty two)

[0090] The above training process will continue for multiple iterations until the algorithm converges. After the algorithm converges, the optimal network parameters are selected and loaded into the Actor network, thereby completing the training process of the reinforcement learning module in the present invention.

[0091] Step 4: Reference trajectory sequence generation module:

[0092] The reference trajectory sequence generation module determines the path information at the preview point according to the adaptive preview distance, and generates the reference trajectory sequence according to the current vehicle speed and the predicted time domain. in, Represents the expected heading angle at time s+1 in the current state s, and so on to time s+n; in, Represents the expected lateral displacement at time s+1 under the current state at time s, and so on to time s+n.

[0093] Step 5: Design the MPC control module:

[0094] Step 5.1: Establish a prediction model:

[0095] The prediction model is established using the vehicle kinematics model, vehicle dynamics model and nonlinear tire model, as shown in formula (23):

[0096]

[0097] In the formula, F y,f is the lateral force of the front wheel, F y,r is the rear wheel lateral force, m is the vehicle mass, γ is the vehicle yaw rate, a, b is the distance from the vehicle center of mass to the front and rear axles, I z The vehicle's yaw moment of inertia.

[0098] The tire lateral force is calculated as follows:

[0099] F y =μD y sin(C y arctan(B y α-E y (B y α-arctan(B y α))))+S vy (twenty four)

[0100] In the formula, F y is the tire lateral force, μ is the road adhesion coefficient, α is the tire slip angle, B y is the tire characteristic curve stiffness factor, C y is the tire characteristic curve shape factor, Dy is the peak factor of the tire characteristic curve, E y is the tire characteristic curve curvature factor, S vy is the vertical offset of the tire characteristic curve, except C y In addition, other tire characteristic curve parameters are related to the tire vertical load, and α is the tire side slip angle.

[0101] The equivalent slip angles of the front and rear axle tires of the vehicle are as shown in equations (25) and (26):

[0102]

[0103]

[0104] Among them, α f , α r Represent the equivalent sideslip angles of the front and rear axle tires respectively.

[0105] The vertical loads of the front and rear axle tires of the vehicle are as shown in equations (27) and (28):

[0106]

[0107]

[0108] The above model is discretized using the fourth-order Runge-Kutta method, with a sampling time of T s , the input-output relationship of the incremental discrete system model can be expressed as formula (29),

[0109]

[0110] In the formula, u(s)=[δ f ],

[0111] Among them, ξ(s) is the state variable of the vehicle at the current time s, u(s) and △u(s) are the control input and control input increment of the system at the current time s, respectively, and y c (s) is the predicted output of the system at the current time s, and C is the coefficient matrix used to determine the number of predicted outputs of the system.

[0112] The optimization problem construction takes into account the ride comfort and control action stability, imposes hard constraints on the control action, and defines the optimization objective of the model predictive control, such as Equation (30) and Equation (31).

[0113]

[0114]

[0115] Where J1 is the objective function representing the weighted combination of yaw angle and lateral displacement tracking deviation, Γ y is the predicted output weight matrix, Based on the preview point L i is the reference heading angle for the next n steps; J2 is the objective function of the control input increment, Γ u is the control input incremental weight matrix, δ fmin ,δ fmax Represent the minimum steering angle and the maximum steering angle respectively; △δ fmin ,△δ fmax Represent the sampling time T s The minimum and maximum steering angle changes allowed within the range.

[0116] Dynamic optimization is to solve the optimization problem and obtain the optimal control instruction u at the current moment. o =[δ o ],δ o is the front wheel turning angle.

[0117] Step 6: Design the horizontal action conversion module:

[0118] The lateral motion control module takes the optimal control instruction u solved by the model predictive control module as o =[δ o ] is converted into the PWM signal required by the steering motor to control the vehicle to achieve intelligent path tracking.

[0119] In summary: the present invention relates to a path tracking control method integrating reinforcement learning and adaptive preview. The method comprises two layers, the upper layer outputs an adaptive preview distance by integrating a reinforcement learning algorithm, and the lower layer uses a reference trajectory sequence determined by the adaptive preview distance as the expected information of the model predictive control, so that the method can improve the generalization and robustness of the path tracking control strategy while ensuring safety, and has good application prospects in the field of intelligent vehicle path tracking control.

Claims

1. A path tracking control method integrating reinforcement learning and adaptive preview, characterized in that: The method includes two layers, the upper layer optimizes the preview distance through reinforcement learning and inputs it to the lower layer, and the lower layer determines the control action through model predictive control optimization solution to control the vehicle to achieve intelligent driving; wherein, the upper layer includes an information acquisition module, an information processing module, and a reinforcement learning module, and the lower layer includes a reference trajectory sequence generation module, a model predictive control module, and a lateral action conversion module; The information collection module in the upper layer is used to obtain vehicle status information C t ={x,y,θ,v x ,v y } and expected path information P t ={x ref ,y ref ,θ ref }; x, y are the lateral and longitudinal position coordinates of the vehicle, θ is the heading angle of the vehicle, v x and v y are the longitudinal and lateral speeds of the vehicle, x ref With y ref are the coordinates of the desired path points, θ ref is the desired heading angle; The information processing module in the upper layer is used to obtain the comprehensive deviation and the path curvature at the initial preview point according to the vehicle state information and the expected path information; specifically, the comprehensive deviation includes the lateral deviation y between the current position of the vehicle and the expected path e and yaw angle deviation θ e , due to the lateral deviation y e and yaw angle deviation θ e The units are different, and after normalization and weighted calculation, the comprehensive deviation C is obtained. e , as shown in equations (1) to (5), and e =yy ref (1) i e =θ-θ ref (2) in, and are the lateral deviation and yaw angle deviation after normalization respectively; emax and θ emax are the maximum lateral deviation and yaw angle deviation respectively; emin and θ emin are the minimum lateral deviation and yaw angle deviation respectively; κ is the weight coefficient; The information processing module is based on the current longitudinal vehicle speed v x and preview factor k p Determine the initial preview distance L i , that is, L i =k p v x , and then obtain L by table lookup method i The path curvature ρ i ; The state space of the reinforcement learning module in the upper layer is defined as s t ={C e ,ρ i ,v x ,v y }; Among them, C e is the comprehensive deviation, ρ i is the initial preview distance L i The path curvature at v x and v y are the longitudinal speed and lateral speed of the vehicle respectively; the action space of the reinforcement learning module is defined as a t =[L c ], L c is the adaptive preview distance; the reinforcement learning module is trained by interacting with the driving environment. The present invention designs a corresponding reward function based on the vehicle tracking expected path accuracy. The reward function of the reinforcement learning module is defined as formula (6): In the formula, C e is the comprehensive deviation, p1 is the reward amplification factor, and p2 is the reward penalty number for failure in the tracking path; The reference trajectory sequence generation module in the lower layer generates a reference trajectory sequence based on the adaptive preview distance according to the adaptive preview distance and the current expected path information; The model predictive control module in the lower layer optimizes the control output using a reference trajectory sequence based on an adaptive preview distance. Specifically, the model predictive control module includes three parts: a prediction model, an optimization problem construction, and a dynamic optimization. The prediction model is established using a vehicle kinematic model, a vehicle dynamics model, and a nonlinear tire model, as shown in formula (7). In the formula, x, y are the horizontal and vertical position coordinates of the vehicle, v x and v y are the longitudinal and lateral speeds of the vehicle, θ is the heading angle of the vehicle, and F y,f is the lateral force of the front wheel, F y,r is the rear wheel lateral force, m is the vehicle mass, γ is the vehicle yaw rate, a, b is the distance from the vehicle center of mass to the front and rear axles, I z Vehicle yaw moment of inertia; The above model is discretized using the fourth-order Runge-Kutta method, with a sampling time of T s , the input-output relationship of the incremental discrete system model can be expressed as formula (8), In the formula, In the ceremony, u(s)=[δ f ], Among them, ξ(s) is the state variable of the vehicle at the current time s, u(s) and △u(s) are the control input and control input increment of the system at the current time s, respectively, and y c (s) is the predicted output of the system at the current time s, and C is the coefficient matrix used to determine the number of predicted outputs of the system; The optimization problem construction takes into account the ride comfort and control action stability, imposes hard constraints on the control action, and defines the optimization objective of the model predictive control, such as Equation (9) and Equation (10). st(10) Where J1 is the objective function representing the weighted combination of yaw angle and lateral displacement tracking deviation, Γ y is the prediction output weight matrix, Based on the preview point L i is the reference heading angle for the next n steps; J2 is the objective function of the control input increment, Γ u is the control input incremental weight matrix, δ fmin ,δ fmax Represent the minimum steering angle and the maximum steering angle respectively; △δ fmin ,△δ fmax Represent the sampling time T s The minimum change in steering angle and the maximum change in steering angle allowed within the system; Dynamic optimization is to solve the optimization problem and obtain the optimal control output u at the current moment. o =[δ o ],δ o is the front wheel turning angle; Finally, the lateral motion control module uses the optimal control output u solved by the model predictive control module o =[δ o ] is converted into the PWM signal required by the steering motor to control the vehicle to achieve intelligent path tracking.

2. The path tracking control method integrating reinforcement learning and adaptive preview according to claim 1, characterized in that: The reference trajectory sequence generation module determines the path information at the preview point according to the adaptive preview distance, and generates the reference trajectory sequence according to the current vehicle speed and the predicted time domain. in, Represents the expected heading angle at time s+1 in the current state s, and so on to time s+n; in, Represents the expected lateral displacement at time s+1 under the current state at time s, and so on to time s+n.

Images