Vehicle trajectory tracking method based on model prediction and learning driving

Abstract

The invention relates to a vehicle trajectory tracking method based on model prediction and learning driving, which belongs to the technical field of unmanned vehicle trajectory tracking control, combines deep learning and control theories, and comprises two parts of reducing calculation time consumption and adjusting weight parameters in real time. The purposes of reducing the calculation burden of the vehicle-mounted controller and improving the trajectory tracking precision and the dynamic adaptability are achieved respectively; real-time adjustment of model parameters is realized by learning dynamic characteristics and environment changes of the system; the learning-based adaptive model can better adapt to different track modes and environmental dynamics through continuous learning and adaptation; deep learning models such as a neural network are utilized to learn nonlinear mapping of the system, so that prediction accuracy is improved, and real-time adjustment and optimization of the system are realized in combination with a model prediction control framework; the application of the method in trajectory tracking is expected to enhance the robustness of the system and improve the adaptability to complex environments and dynamic changes.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned vehicle trajectory tracking control, and in particular to a vehicle trajectory tracking method based on model prediction and learning drive, and in particular to a vehicle trajectory tracking method based on a learning-driven nonlinear time-varying model predictive controller. Background Art

[0002] Trajectory tracking is the basic part of unmanned driving. It effectively and stably controls the self-driving car to track the determined reference trajectory output by the path planning module. Among them, this part includes model-free control and model-based control; the former does not need to establish a vehicle model and directly controls according to feedback, which is simple to implement and has small computational complexity, but the control accuracy is average; the latter establishes a kinematic or dynamic model, pays attention to the error while paying attention to the internal state changes of the system, and has a good control effect. In particular, the model predictive control algorithm (MPC) can handle the problem of possible interactions between the input and output of the multiple-input multiple-output (MIMO) system, systematically consider the constraints of the state and control behavior, and explain the expected future behavior of the system. Therefore, it is widely used in the field of unmanned driving trajectory tracking.

[0003] Although the model predictive control algorithm MPC has high accuracy and good performance, it needs to solve the problem of online optimization at each sampling moment, which requires a powerful on-board controller and large memory, and there is a contradiction between computing power and system performance. When the sampling time is short, the system performance is enhanced and the amount of calculation is high; when the sampling time is long, the calculation and system performance will decrease. In the current context of emphasizing the balance between control accuracy and computing resource consumption, the two cannot be taken into account. On the other hand, trajectory tracking tasks often face complex and dynamically changing environments, and the adaptive model can flexibly adjust its own parameters and structure under constantly changing conditions, thereby improving the adaptability and robustness of the tracking system. It can adapt to different trajectory patterns and environmental changes in real time, allowing the tracking system to track the target motion trajectory more accurately.

[0004] 1. The current trajectory tracking control algorithm of unmanned vehicles is generally limited to non-holonomic constraints at a fixed speed, which cannot meet the control requirements when facing high-speed driving scenarios. Especially in emergency operations during high-speed driving, such as steering, changing lanes on low-adhesion roads, and avoiding obstacles, the tire adhesion is easily saturated, and the cornering force will approach the limit, which significantly increases the control complexity. When driving on a curve, the loss of trajectory caused by the sideslip of the front axle and the drift instability induced by the sideslip of the rear axle are both safety issues that need to be solved urgently. To solve the above problems, it is particularly important to build a trajectory tracking system that can predict and adapt to dynamic constraints such as slip and roll. The system uses an active front wheel steering strategy to accurately track the predetermined trajectory while ensuring vehicle stability, thereby effectively preventing accidents. The realization of this goal is inseparable from the in-depth understanding and application of vehicle and tire dynamics models. However, although the use of nonlinear time-varying MPC methods can improve system performance, its computational complexity has become a bottleneck for real-time applications. Therefore, research needs to focus on algorithm optimization and computational efficiency improvement to ensure the real-time and effectiveness of trajectory tracking control in high-speed and complex environments.

[0005] 2. Most of the existing unmanned vehicle trajectory tracking control algorithms do not make effective dynamic adjustments to the MPC cost function weights, especially when the system's dynamic characteristics and real environment are constantly changing. The adaptability is very poor. Whether it is a logic-based parameter adjustment method or a simple linear adjustment method, the existing adjustment methods cannot fully cover all dynamic changes. More importantly, there is no ability to continuously learn and adapt, and it is impossible to enhance the robustness of the system. In actual driving, engineers are often required to manually adjust parameters or add judgment logic. This method is very dependent on specific working conditions. Any slight change will lead to a significant decrease in control effect, and even safety issues such as logic processing failure, and it is impossible to give full play to the vehicle's dynamic advantages. Summary of the invention

[0006] In view of the above problems, the present invention provides a vehicle trajectory tracking method based on model prediction and learning drive. The present invention combines deep learning and control theory, including reducing the computational time and adjusting the weight parameters in real time, respectively achieving the purpose of reducing the computational burden of the vehicle controller and improving the trajectory tracking accuracy and dynamic adaptability, and better balancing resource consumption and control accuracy. By learning the dynamic characteristics of the system and environmental changes, real-time adjustment of model parameters is achieved. The learning-based adaptive model can better adapt to different trajectory patterns and environmental dynamics through continuous learning and adaptation. The nonlinear mapping of the system is learned by deep learning models such as neural networks, thereby improving the accuracy of the prediction, and combined with the framework of model predictive control, real-time adjustment and optimization of the system are achieved; the application in trajectory tracking is expected to enhance the robustness of the system, improve the adaptability to complex environments and dynamic changes, and open up new possibilities for achieving high-precision trajectory tracking tasks.

[0007] The present invention provides a vehicle trajectory tracking method based on model prediction and learning drive, comprising:

[0008] Step S1, constructing a dynamic model of the unmanned vehicle;

[0009] Preferably, the expression of the dynamic model of the unmanned vehicle in step S1 is:

[0010]

[0011]

[0012]

[0013]

[0014]

[0015]

[0016] Among them, v x represents the lateral velocity of the vehicle, v y represents the longitudinal velocity of the vehicle, represents the yaw angle of the vehicle, represents the vehicle's yaw rate, r represents the vehicle's yaw angular velocity, represents the vehicle's yaw angular acceleration, δ f represents the front wheel turning angle of the vehicle, δ r is the rear wheel turning angle, F lf Indicates the lateral force of the front wheel, F cf is the front wheel longitudinal force, F lr Represents the rear wheel lateral force, F cr is the rear wheel longitudinal force, I z represents the rotational torque of the vehicle around the Z axis, l f represents the distance from the front axle to the center of mass of the vehicle, l r is the distance from the rear axle to the vehicle's center of mass, represents the lateral velocity of the vehicle, represents the longitudinal velocity of the vehicle, represents the lateral velocity of the vehicle, Indicates the longitudinal velocity of the vehicle.

[0017] Furthermore, the front wheel lateral force F lf , front wheel longitudinal force F cf , rear wheel lateral force F lr and the front wheel longitudinal force F cr , the expressions are:

[0018] Fcf =C cf α f ,F cr =C cr α r

[0019] F lf =C lf s f ,F lr =C lr s r

[0020] Among them, F cf Represents the friction of the front wheel, F cr is the friction force of the rear wheel, C cf Represents the lateral stiffness coefficient of the front wheel, C cr Represents the lateral stiffness coefficient of the rear wheel, α f represents the sideslip angle of the front wheel, α r Indicates the side slip angle of the rear wheel, F lf represents the longitudinal force of the front wheel, F lr represents the longitudinal force of the rear wheel, C lf Indicates the longitudinal stiffness coefficient of the front wheel, C lr Represents the longitudinal stiffness coefficient of the rear wheel, s f Indicates the slip rate of the front wheel, s r Indicates the slip rate of the rear wheels.

[0021] Furthermore, the front wheel partial stiffness α f and the rear wheel eccentric stiffness α r The expressions are:

[0022]

[0023]

[0024] Step S2, determining an MPC trajectory tracking controller;

[0025] Preferably, the expression of the MPC trajectory tracking controller is:

[0026]

[0027] Where A is the objective function of the MPC trajectory tracking controller; is the stage cost function at the kth moment, describing the state x at the kth moment k and the control input u at the kth moment k The immediate cost of is the estimate of the kth parameter, is the cost of the control input change at the kth moment, Δu krepresents the change of control input at the kth moment relative to the previous moment, D is the weight matrix, m(x K ) is the cost function at the Kth moment, that is, the terminal cost function.

[0028] is a constraint, is the initial state constraint, requiring the state at the beginning of the optimization to be a known initial state is the system dynamic constraint, describing how the state changes at each time step according to the current state x k , the control input u at the current moment k and the parameter estimates at the current moment To evolve, An estimation function representing the system dynamics.

[0029] are state constraints that ensure that the state and control inputs at each time step meet certain safety or performance criteria, such as dynamic limits, physical constraints, etc.

[0030] is the K-th moment constraint, i.e., the terminal constraint, which is used to ensure that the state x at the K-th moment K and the K-th moment parameter estimate Satisfy the constraints.

[0031] Step S3: Establish a nonlinear time-varying model predictive controller based on the MPC trajectory tracking controller, and the expression is:

[0032]

[0033] Where J is the cost function of the nonlinear time-varying model, η(k+Δk|k) is the predicted value of the k+Δkth time step in the prediction domain, Δk is the time interval, and η ref (k+Δk|k) is the reference quantity of the k+Δkth time step in the prediction time domain, Δu(k+Δk|k) is the control increment constraint of the k+Δkth time step in the prediction time domain, Q is the quadratic matrix of weighted state or output error, and its elements are related to the performance indicators of the system, such as control accuracy or steady-state error. A larger Q value will increase the corresponding state or output error to the objective function. R is the quadratic matrix of weighted control increment, which reflects the degree of penalty for changes in control input. A larger R value will make the control increment smaller, thereby reducing the severity of the control action and maintaining the stability of the system; ε is the relaxation factor.

[0034] Step S4, designing an event-triggered control law and an event-triggered reward function;

[0035] Step S5, optimizing the nonlinear time-varying model predictive controller based on the unmanned vehicle's dynamic model, event-triggered control law, event-triggered reward function and strategy learning method, obtaining an optimized MPC trajectory tracking controller weight parameter and inputting it into the nonlinear time-varying model predictive controller for re-solving, and obtaining an optimized nonlinear time-varying model predictive controller;

[0036] Step S6: Tracking the vehicle trajectory using the optimized nonlinear time-varying model predictive controller.

[0037] Preferably, the event triggering control law in step S4 is expressed as:

[0038]

[0039] Among them, u i,k is the control input of the ith control variable at the kth moment, is the initial state constraint at the kth moment, is the initial state constraint of the ith control variable, π θ is the event trigger mechanism, a i is the output action corresponding to the ith control variable, s i is the enhanced state corresponding to the i-th control quantity, π ET (·) is the event trigger function, u 0,k is the initial control input at the kth moment.

[0040] Preferably, the reward function triggered by the event is expressed as:

[0041]

[0042] Among them, s′=t(s,a), which means the state obtained by taking action a in state s, T is the length of the round, and R G = J represents the control target of ET-MPC; λ p is the weight of the penalty condition, p indicates whether the state s′ triggers the penalty condition for the end of the round, which is set to whether the lateral deviation exceeds the threshold, λ p =1 / t is inversely proportional to the time steps in the current round; a represents the action, when a=1, it indicates whether to trigger the recalculation event, λ A is the action weight, which reflects the relative relationship between the event triggering frequency and the control effect.

[0043] Preferably, the specific steps of obtaining the optimized nonlinear time-varying model predictive controller in step S5 include:

[0044] Step S51, let k=1, when k=1, it indicates the initial time;

[0045] Step S52: Solve the nonlinear time-varying model predictive controller to obtain the optimal predicted state trajectory at time k and the optimal control input

[0046] Step S53, let i=1, when i=1, it represents the first control variable in the optimal predicted state trajectory at time k;

[0047] Step S54, inputting the ith control variable in the optimal predicted state trajectory at time k into the dynamic model of the unmanned vehicle, and obtaining the vehicle state, environmental state, MPC trajectory tracking error and curvature corresponding to the ith control variable;

[0048] Exemplarily, the vehicle state in step S54 includes vehicle position, speed, heading, yaw angle, and yaw angular velocity;

[0049] The environmental conditions include road curvature and slope.

[0050] Step S55, generating an enhanced state of the i-th control variable based on the i-th MPC trajectory tracking error, curvature, vehicle state and environmental state;

[0051] Step S56: Calculate the corresponding reward function based on the vehicle state and environment state obtained after the i-th control variable is input into the vehicle model; update the critic network B based on the corresponding reward function i,k , and output the advantage value of the ith control quantity; use the advantage value of the ith control quantity to adjust the strategy network W i Update and get the updated strategy network W i+1 , update the strategy network W i As a policy network W i+1 ; Input the enhanced state of the i-th control quantity into the update strategy network W i ; Determine whether the enhanced state of the i-th control quantity is stable through the strategy learning method;

[0052] Step S57: If it is stable, determine whether the vehicle state and the environment state corresponding to the i-th control amount have reached the termination condition. If so, return to step S51 and set k=k+1; if not, set i=i+1 and return to step S54;

[0053] If it is unstable, it means that the vehicle state and environmental state corresponding to the i-th control quantity do not have an acceptable control effect, the event triggers the strategy to decide to recalculate, the strategy network continues to output actions, the output actions are input into the MPC trajectory tracking controller, the cost function weight factor in the MPC trajectory tracking controller is updated, and an updated MPC trajectory tracking controller is obtained; based on the establishment of an updated nonlinear time-varying model predictive controller in the MPC trajectory tracking controller, return to step S52, and replace the nonlinear time-varying model predictive controller with the updated nonlinear time-varying model predictive controller;

[0054] Step S58, determine whether the optimal predicted state trajectory at time k reaches the end of the path. If so, calculate the total reward, obtain the nonlinear time-varying model predictive controller, and complete the parking control. If not, k=k+1, return to step S52.

[0055] It can be understood that the policy network in step S56 determines whether an event is triggered.

[0056] It can be understood that the stability described in step S56 is whether the i-th control amount can continue to be invested in the unmanned vehicle for subsequent control;

[0057] For example, if the current vehicle control error is too large or the current environment is a sharp turn, the enhanced state of the ith control quantity will reflect possible or imminent instability in the control effect. When the strategy network is updated, it will be based on the enhanced state of the input ith control quantity, through feature extraction of the enhanced state in the network and the strategy output of the activation function, to obtain the final action selection, which is expressed as whether to choose to recalculate.

[0058] Preferably, the termination condition in step S57 is a condition that causes the simulation to be restarted; the termination condition includes the lateral deviation exceeding a threshold or exceeding a maximum simulation time.

[0059] Preferably, the total reward expression is:

[0060]

[0061] Among them, R is the reward value, R Y represents the lateral deviation term, Represents the heading deviation term, R γ represents the yaw angle term, represents the lateral deviation change rate term, represents the heading deviation change rate term, Represents the angle change rate term.

[0062] The learning-driven agent in the present invention only acts as an optimizer on the MPC controller, and does not directly interact with the vehicle. The control output still depends on the calculation result of the MPC controller.

[0063] Compared with the prior art, the present invention has at least the following beneficial effects:

[0064] (1) The deep reinforcement learning of the present invention continuously interacts with the MPC system to learn the event triggering strategy, and eliminates unnecessary control outputs in the trajectory tracking process through the triggering mechanism, which not only fully utilizes the computing power of the MPC system in the multi-step prediction time domain, but also can greatly reduce the consumption of computing resources;

[0065] (2) The weight coefficient of the cost function of the MPC controller is adaptively adjusted based on strategy learning, which can avoid vehicle instability caused by model nonlinearity under different road conditions and different curvature sections, improve control accuracy and stability, and greatly reduce the tedious work of manually adjusting parameters;

[0066] (3) The present invention incorporates event triggering mechanism and strategy learning into a unified framework, performs event triggering and parameter optimization simultaneously, combines the advantages of the two parts through the design of reward function, and comprehensively jointly optimizes the traditional MPC controller. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The drawings are only for the purpose of illustrating particular embodiments and are not to be construed as limiting the invention.

[0068] Figure 1 This is a schematic diagram of the optimization framework of the nonlinear time-varying model predictive controller of the present invention;

[0069] Figure 2 It is a flow chart of the comprehensive optimization method of the nonlinear time-varying model predictive controller according to the present invention. DETAILED DESCRIPTION

[0070] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0071] A specific embodiment of the present invention, as Figure 1-2 , discloses a vehicle trajectory tracking method based on model prediction and learning-driven,

[0072] In order to illustrate the effectiveness of the method proposed in the present invention, the above technical solution of the present invention is described in detail below through a specific embodiment, and the specific implementation steps are as follows:

[0073] The present invention provides a vehicle trajectory tracking method based on model prediction and learning drive, comprising:

[0074] Step S1, constructing a dynamic model of the unmanned vehicle;

[0075] Preferably, the expression of the dynamic model of the unmanned vehicle in step S1 is:

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] Among them, v x represents the lateral velocity of the vehicle, v y represents the longitudinal velocity of the vehicle, represents the yaw angle of the vehicle, represents the vehicle's yaw rate, r represents the vehicle's yaw angular velocity, represents the vehicle's yaw angular acceleration, δ f represents the front wheel turning angle of the vehicle, δ f is the rear wheel turning angle, F lr Indicates the lateral force of the front wheel, F cf is the front wheel longitudinal force, F lr Represents the rear wheel lateral force, F cr is the rear wheel longitudinal force, I z represents the rotational torque of the vehicle around the Z axis, l f represents the distance from the front axle to the center of mass of the vehicle, l r is the distance from the rear axle to the vehicle's center of mass, represents the lateral velocity of the vehicle, represents the longitudinal velocity of the vehicle, represents the lateral velocity of the vehicle, Indicates the longitudinal velocity of the vehicle.

[0083] Furthermore, the front wheel lateral force F lf , front wheel longitudinal force F cf , rear wheel lateral force F lr and the front wheel longitudinal force F cr , the expressions are:

[0084] F cf =C cfα f ,F cr =C cr α r

[0085] F lf =C lf s f ,F lr =C lr s r

[0086] Among them, F cf Represents the friction of the front wheel, F cr is the friction force of the rear wheel, C cf Represents the lateral stiffness coefficient of the front wheel, C cr Represents the lateral stiffness coefficient of the rear wheel, α f represents the sideslip angle of the front wheel, α r Indicates the side slip angle of the rear wheel, F lf represents the longitudinal force of the front wheel, F lr represents the longitudinal force of the rear wheel, C lf Indicates the longitudinal stiffness coefficient of the front wheel, C lr Represents the longitudinal stiffness coefficient of the rear wheel, s f Indicates the slip rate of the front wheel, s r represents the slip rate of the rear wheel. Further, the front wheel eccentric stiffness α f and the rear wheel eccentric stiffness α r The expressions are:

[0087]

[0088]

[0089] Step S2, determining an MPC trajectory tracking controller;

[0090] Preferably, the expression of the MPC trajectory tracking controller is:

[0091]

[0092] Where A is the objective function of the MPC trajectory tracking controller; is the stage cost function at the kth moment, describing the state x at the kth moment k and the control input u at the kth moment k The immediate cost of is the estimate of the kth parameter, is the cost of the control input change at the kth moment, Δu k represents the change of control input at the kth moment relative to the previous moment, D is the weight matrix, m(x K) is the cost function at the Kth moment, that is, the terminal cost function.

[0093] is a constraint, is the initial state constraint, requiring the state at the beginning of the optimization to be a known initial state is the system dynamic constraint, describing how the state changes at each time step according to the current state x k , the control input u at the current moment k and the parameter estimates at the current moment To evolve, An estimation function representing the system dynamics.

[0094] are state constraints that ensure that the state and control inputs at each time step meet certain safety or performance criteria, such as dynamic limits, physical constraints, etc.

[0095] is the K-th moment constraint, i.e., the terminal constraint, which is used to ensure that the state x at the K-th moment K and the K-th moment parameter estimate Satisfy the constraints.

[0096] Step S3: Establish a nonlinear time-varying model predictive controller based on the MPC trajectory tracking controller, and the expression is:

[0097]

[0098] Where J is the cost function of the nonlinear time-varying model, η(k+Δk|k) is the predicted value of the k+Δkth time step in the prediction domain, Δk is the time interval, and η ref (k+Δk|k) is the reference quantity of the k+Δkth time step in the prediction time domain, Δu(k+Δk|k) is the control increment constraint of the k+Δkth time step in the prediction time domain, Q is the quadratic matrix of weighted state or output error, and its elements are related to the performance indicators of the system, such as control accuracy or steady-state error. A larger Q value will increase the corresponding state or output error to the objective function. R is the quadratic matrix of weighted control increment, which reflects the degree of penalty for changes in control input. A larger R value will make the control increment smaller, thereby reducing the severity of the control action and maintaining the stability of the system; ε is the relaxation factor.

[0099] Step S4, designing an event-triggered control law and an event-triggered reward function;

[0100] Step S5, optimizing the nonlinear time-varying model predictive controller based on the unmanned vehicle's dynamic model, event-triggered control law, event-triggered reward function and strategy learning method, obtaining an optimized MPC trajectory tracking controller weight parameter and inputting it into the nonlinear time-varying model predictive controller for re-solving, and obtaining an optimized nonlinear time-varying model predictive controller;

[0101] Step S6: Tracking the vehicle trajectory using the optimized nonlinear time-varying model predictive controller.

[0102] Preferably, the event triggering control law in step S4 is expressed as:

[0103]

[0104] Among them, u i,k is the control input of the ith control variable at the kth moment, is the initial state constraint at the kth moment, is the initial state constraint of the ith control variable, π θ is the event trigger mechanism, a i is the output action corresponding to the ith control variable, s i is the enhanced state corresponding to the i-th control quantity, π ET (·) is the event trigger function, u 0,k is the initial control input at the kth moment.

[0105] Preferably, the reward function triggered by the event is expressed as:

[0106]

[0107] Among them, s′=T(s,a), which means the state obtained by taking action a in state s, T means the length of the round, R G = J represents the control target of ET-MPC; λ p is the weight of the penalty condition, p indicates whether the state s′ triggers the penalty condition for the end of the round, which is set to whether the lateral deviation exceeds the threshold, λ p =1 / t is inversely proportional to the time steps in the current round; a represents the action, when a=1, it indicates whether to trigger the recalculation event, λ A is the action weight, which reflects the relative relationship between the event triggering frequency and the control effect.

[0108] Preferably, the specific steps of obtaining the optimized nonlinear time-varying model predictive controller in step S5 include:

[0109] Step S51, let k=1, when k=1, it indicates the initial time;

[0110] Step S52: Solve the nonlinear time-varying model predictive controller to obtain the optimal predicted state trajectory at time k and the optimal control input

[0111] Step S53, let i=1, when i=1, it represents the first control variable in the optimal predicted state trajectory at time k;

[0112] Step S54, inputting the ith control variable in the optimal predicted state trajectory at time k into the dynamic model of the unmanned vehicle, and obtaining the vehicle state, environmental state, MPC trajectory tracking error and curvature corresponding to the ith control variable;

[0113] Exemplarily, the vehicle state in step S54 includes vehicle position, speed, heading, yaw angle, and yaw angular velocity;

[0114] The environmental conditions include road curvature and slope.

[0115] Step S55, generating an enhanced state of the i-th control variable based on the i-th MPC trajectory tracking error, curvature, vehicle state and environmental state;

[0116] Step S56: Calculate the corresponding reward function based on the vehicle state and environment state obtained after the i-th control variable is input into the vehicle model; update the critic network B based on the corresponding reward function i,k , and output the advantage value of the ith control quantity; use the advantage value of the ith control quantity to adjust the strategy network W i Update and get the updated strategy network W i+1 , update the strategy network W i As a policy network W i+1 ; Input the enhanced state of the i-th control quantity into the update strategy network W i ; Determine whether the enhanced state of the i-th control quantity is stable through the strategy learning method;

[0117] Step S57: If it is stable, determine whether the vehicle state and the environment state corresponding to the i-th control amount have reached the termination condition. If so, return to step S51 and set k=k+1; if not, set i=i+1 and return to step S54;

[0118] If it is unstable, it means that the vehicle state and environmental state corresponding to the i-th control quantity do not have an acceptable control effect, the event triggers the strategy to decide to recalculate, the strategy network continues to output actions, the output actions are input into the MPC trajectory tracking controller, the cost function weight factor in the MPC trajectory tracking controller is updated, and an updated MPC trajectory tracking controller is obtained; based on the establishment of an updated nonlinear time-varying model predictive controller in the MPC trajectory tracking controller, return to step S52, and replace the nonlinear time-varying model predictive controller with the updated nonlinear time-varying model predictive controller;

[0119] Step S58, determine whether the optimal predicted state trajectory at time k reaches the end of the path. If so, calculate the total reward, obtain the nonlinear time-varying model predictive controller, and complete the parking control. If not, k=k+1, return to step S52.

[0120] It can be understood that the policy network in step S56 determines whether an event is triggered.

[0121] It can be understood that the stability described in step S56 is whether the i-th control amount can continue to be invested in the unmanned vehicle for subsequent control;

[0122] For example, if the current vehicle control error is too large or the current environment is a sharp turn, the enhanced state of the ith control quantity will reflect possible or imminent instability in the control effect. When the strategy network is updated, it will be based on the enhanced state of the input ith control quantity, through feature extraction of the enhanced state in the network and the strategy output of the activation function, to obtain the final action selection, which is expressed as whether to choose to recalculate.

[0123] Preferably, the termination condition in step S57 is a condition that causes the simulation to be restarted; the termination condition includes the lateral deviation exceeding a threshold or exceeding a maximum simulation time.

[0124] Preferably, the total reward expression is:

[0125]

[0126] Among them, R is the reward value, R Y represents the lateral deviation term, Represents the heading deviation term, R γ represents the yaw angle term, represents the lateral deviation change rate term, represents the heading deviation change rate term, Represents the angle change rate term.

[0127] The learning-driven agent in the present invention only acts as an optimizer on the MPC controller, and does not directly interact with the vehicle. The control output still depends on the calculation result of the MPC controller.

[0128] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A vehicle trajectory tracking method based on model prediction and learning drive, characterized in that: include: Step S1, constructing a dynamic model of the unmanned vehicle; Step S2, determining an MPC trajectory tracking controller; Step S3, establishing a nonlinear time-varying model predictive controller based on the MPC trajectory tracking controller; Step S4, designing an event-triggered control law and an event-triggered reward function; Step S5, optimizing the nonlinear time-varying model predictive controller based on the unmanned vehicle's dynamic model, event-triggered control law, event-triggered reward function and strategy learning method, obtaining an optimized MPC trajectory tracking controller weight parameter and inputting it into the nonlinear time-varying model predictive controller for re-solving, and obtaining an optimized nonlinear time-varying model predictive controller; Step S6: Tracking the vehicle trajectory using the optimized nonlinear time-varying model predictive controller.

2. The vehicle trajectory tracking method based on model prediction and learning drive according to claim 1 is characterized in that: The expression of the dynamic model of the unmanned vehicle in step S1 is: in, represents the lateral velocity of the vehicle, represents the longitudinal velocity of the vehicle, represents the lateral velocity of the vehicle, Indicates the longitudinal velocity of the vehicle; v x represents the lateral velocity of the vehicle, v y represents the longitudinal velocity of the vehicle, Indicates the yaw angle of the vehicle, F lf Indicates the lateral force of the front wheel, F cf is the front wheel longitudinal force, F lr Represents the rear wheel lateral force, F cr is the rear wheel longitudinal force, δ f represents the front wheel turning angle of the vehicle, δ r is the rear wheel turning angle, m is the vehicle weight, represents the vehicle's yaw rate, r represents the vehicle's yaw angular velocity, represents the vehicle's yaw angular acceleration, I z represents the rotational torque of the vehicle around the Z axis, l f represents the distance from the front axle to the center of mass of the vehicle, l r is the distance from the rear axle to the vehicle's center of mass.

3. The vehicle trajectory tracking method based on model prediction and learning drive according to claim 1 is characterized in that: The expression of the MPC trajectory tracking controller is: Where A is the output of the MPC trajectory tracking controller; is the stage cost function at the kth moment, describing the state x at the kth moment k and the control input u at the kth moment k The immediate cost of is the estimate of the kth parameter, is the cost of the control input change at the kth moment, Δu k represents the change of control input at the kth moment relative to the previous moment, D is the weight matrix, m(x N ) is the terminal cost function.

4. The vehicle trajectory tracking method based on model prediction and learning drive according to claim 1 is characterized in that: A nonlinear time-varying model predictive controller is established based on the MPC trajectory tracking controller, and the expression is: Where J is the cost function of the nonlinear time-varying model, η(k+Δk|k) is the predicted value of the k+Δkth time step in the prediction domain, Δk is the time interval, and η ref (k+Δk|k) is the reference quantity of the k+Δkth time step in the prediction domain, Δu(k+Δk|k) is the control increment constraint of the k+Δkth time step in the prediction domain, Q is the quadratic term matrix of the weighted state or output error, R is the quadratic term matrix of the weighted control increment, and ε is the relaxation factor.

5. The vehicle trajectory tracking method based on model prediction and learning drive according to claim 1 is characterized in that: The specific steps of obtaining the optimized nonlinear time-varying model predictive controller in step S5 include: Step S51, let k=1, when k=1, it indicates the initial time; Step S52: Solve the nonlinear time-varying model predictive controller to obtain the optimal predicted state trajectory at time k and the optimal control input Step S53, let i=1, when i=1, it represents the first control variable in the optimal predicted state trajectory at time k; Step S54, inputting the ith control variable in the optimal predicted state trajectory at time k into the dynamic model of the unmanned vehicle, and obtaining the vehicle state, environmental state, MPC trajectory tracking error and curvature corresponding to the ith control variable; Step S55, generating an enhanced state of the i-th control variable based on the i-th MPC trajectory tracking error, curvature, vehicle state and environmental state; Step S56: Calculate the corresponding reward value based on the vehicle state and the environment state corresponding to the i-th control amount; update the critic network B based on the corresponding reward value i,k , and output the advantage value of the ith control quantity; use the advantage value of the ith control quantity to adjust the strategy network W i Update and get the updated strategy network W i+1 , update the strategy network W i As a policy network W i+1 ; The enhanced state of the i-th control quantity is input into the update strategy network W i ; Determine whether the enhanced state of the i-th control quantity is stable through the strategy learning method; Step S57: If it is stable, determine whether the vehicle state and the environment state corresponding to the i-th control amount have reached the termination condition. If so, return to step S51 and set k=k+1; if not, set i=i+1 and return to step S54; If unstable, obtain the updated strategy network W i Output action, input the output action into the MPC trajectory tracking controller, update the cost function weight factor in the MPC trajectory tracking controller, and obtain an updated MPC trajectory tracking controller; based on establishing an updated nonlinear time-varying model predictive controller in the MPC trajectory tracking controller, return to step S52, and replace the nonlinear time-varying model predictive control with the updated nonlinear time-varying model predictive controller; Step S58, determine whether the optimal predicted state trajectory at time k reaches the end of the path. If so, calculate the total reward to obtain the optimized nonlinear time-varying model predictive controller and complete the parking control. If not, k=k+1 and return to step S52.

6. The vehicle trajectory tracking method based on model prediction and learning drive according to claim 5 is characterized in that: Step S54: the vehicle state includes the vehicle position, speed, heading, yaw angle and yaw angular velocity; The environmental conditions include road curvature and slope.

7. The vehicle trajectory tracking method based on model prediction and learning drive according to claim 5 is characterized in that: The stabilization in step S57 is that the vehicle state and the environmental state corresponding to the i-th control variable have an acceptable control effect.

8. The vehicle trajectory tracking method based on model prediction and learning drive according to claim 5 is characterized in that: The termination conditions include the lateral deviation exceeding a threshold or exceeding a maximum simulation time.

9. The vehicle trajectory tracking method based on model prediction and learning drive according to claim 5, characterized in that: The total reward expression is: Among them, R is the reward value, R Y represents the lateral deviation term, Represents the heading deviation term, R γ represents the yaw angle term, represents the lateral deviation change rate term, represents the heading deviation change rate term, Represents the angle change rate term.

Images