Intelligent vehicle trajectory tracking control method based on double-quintic polynomial and multi-target reward DDPG algorithm

Abstract

The invention belongs to the technical field of path planning and tracking, and particularly relates to an intelligent vehicle trajectory tracking control method based on a double-quintic polynomial and a multi-target reward DDPG algorithm, which establishes a vehicle path model and gives a tracking error function by introducing a double-quintic polynomial lane changing path planning algorithm. And constructing a trajectory tracking control model in combination with the vehicle dynamics model and the Markov decision process. Simulation experiments show that under different vehicle speed conditions, compared with a traditional DDPG algorithm, the method shows better performance in the aspects of tracking precision, lateral acceleration control, course angle stability and yaw angle stability, and effectiveness and robustness of the method are verified.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of path planning and tracking, and particularly relates to an intelligent vehicle trajectory tracking control method based on a double quintic polynomial and a multi-target reward DDPG algorithm. BACKGROUND

[0002] In the field of intelligent vehicle trajectory tracking control, the path planning method based on quintic polynomial is widely used because the trajectory generated by the method has continuity in position, speed and acceleration. However, the method gradually exposes several key defects in the actual application process: first, the safety distance required for lane changing operation is large and the lane changing time is long, which reduces the maneuvering efficiency of the vehicle in dynamic complex scenes such as highways, and may increase the collision risk with other traffic participants due to the long lane changing duration; second, the adaptability of traditional control strategies (such as PID and LQR) to the strong nonlinear dynamics of the vehicle (such as the coupling effect of lateral and longitudinal motion) is insufficient, which causes the lateral position tracking accuracy to decrease and the lateral acceleration to fluctuate intensively in high-speed working conditions, making it difficult to meet the requirements of driving comfort; third, the existing tracking control methods based on deep reinforcement learning (such as the DDPG algorithm) mostly use single target reward function design, which fails to optimize multiple performance indicators such as tracking accuracy, steering smoothness, driving efficiency and speed stability, restricting the improvement of the overall performance of the system. These problems jointly limit the reliability and practicality of intelligent vehicles in high-speed dense traffic environments, and urgent breakthroughs are needed through the collaborative innovation of trajectory planning and control methods. SUMMARY

[0003] In view of the problems in the prior art, the application provides an intelligent vehicle trajectory tracking control method based on a double quintic polynomial and a multi-target reward DDPG algorithm.

[0004] The application is implemented by the following technical solutions: An intelligent vehicle trajectory tracking control method based on a double quintic polynomial and a multi-target reward DDPG algorithm, comprising the following steps: S1, generating a motion trajectory equation for the first lane change and the second lane change using a double quintic polynomial; S2, obtaining tracking error signals, including lateral position error and yaw rate deviation ; S3, constructing a DDPG control model, involving state input, action output and an innovative multi-target reward function; S4, performing DDPG algorithm training: the DDPG algorithm accumulates interaction data through an experience replay mechanism, and uses a double target network to stabilize the training process; S5, performing simulation training.

[0005] Further, the specific operation of S1 is as follows: S1.1, taking the vehicle starting point as the initial state and the adjacent lane center point as the terminal point, establishing the motion trajectory equation of the first lane change, the lateral displacement being equal to the lane width:

[0006]

[0007]

[0008] wherein, is the time used for the first lane change; and are the initial time and the speed after lane change, respectively; is the distance traveled for overtaking lane change; is the lane width; S1.2, taking the terminal point of the first lane change as the starting point of the second lane change and the original lane target point as the terminal point, establishing the motion trajectory equation of the second lane change, the lateral displacement being zero relative to the vehicle starting point:

[0009]

[0010] wherein, is the time used for returning to the original lane; is the speed of returning to the original lane; is the distance traveled for returning to the original lane.

[0011] Further, the specific operation of S2 is as follows: The lateral trajectory tracking is performed through the lateral position error and the yaw rate deviation . The calculation method of the lateral position error is to obtain the current vehicle position information by using the sensors and other devices installed on the vehicle, compare it with the lateral position information of the reference trajectory at the current time, and thus calculate the lateral position error .

[0012] The calculation method of the yaw rate deviation is to replace the actual road curvature with the planned reference trajectory curvature, and the curvature expression is as follows:

[0013] The expected yaw rate is obtained by combining the vehicle longitudinal speed and the road curvature , and the yaw rate deviation is obtained by comparing the current yaw rate of the vehicle.

[0014] Furthermore, in S3, the state input S t Lateral position error between vehicle and track yaw rate deviation Horizontal error rate Yaw rate error rate Path curvature lateral speed of the vehicle Composition; Action Output A t By adjusting the front wheel steering angle To achieve lateral motion control; The multi-objective reward function is optimized through the collaborative optimization of four sub-objectives, resulting in a final comprehensive reward function. for: = +M R1 represents the reward for tracking accuracy. For comfort rewards, For efficiency-based rewards, M represents the vehicle speed stability reward, and M represents the model penalty. In addition, termination conditions are set for the control process, which are met when the lateral displacement of the vehicle relative to its initial position is equal to the lane width or the lateral error. When the value is greater than 1m, terminate the current round, proceed to the next round, and reconfigure the environment state values; at the same time, when the termination condition is met, a penalty M is applied to prevent excessive lateral error.

[0015] Furthermore, the tracking accuracy reward is specifically as follows: tracking accuracy is evaluated based on lateral position error, and the calculation of lateral position error is used as the basis for designing the tracking accuracy reward function, denoted as... A penalty is imposed when the lateral error exceeds 0.15m, and a positive reward (H) is given when the error decreases to 0.1m, encouraging the vehicle to reduce the error from the reference trajectory. The weighting parameters for this reward;

[0016] The comfort reward is specifically defined as follows: the smaller the front wheel steering angle, the smoother the change. Therefore, a comfort reward function is designed based on the front wheel steering angle. :

[0017] Efficiency rewards specifically refer to: providing positive rewards related to lateral displacement. To encourage vehicles to complete the tracking process as quickly as possible and ensure driving efficiency, a penalty P is imposed if the lateral displacement of the vehicle is still 0 if the time exceeds 1 second, so that the vehicle completes the lane-changing process within the specified time. The reward function is set as follows:

[0018] The vehicle speed stability bonus is set as follows: Based on the longitudinal speed bonus function, a fixed bonus C is applied when the deviation between the actual vehicle speed and the reference speed is penalized, provided the speed deviation is within the allowable range (±1 km / h). + C Furthermore, the specific operations of S4 are as follows: S4.1 In each interaction step, the Actor network adjusts its behavior according to the current state. Generate Actions ,in These are the parameters of the Actor network. The environment returns a reward after receiving an action. and the next state and transfer samples Store in the experience revisit buffer; S4.2 The training of the Critic network is based on minimizing the temporal difference, and the loss function is defined as:

[0019] Among them, the target value The following was obtained through calculation using the target network: ,here and The target networks are Critic and Actor, respectively. As a discount factor, and These are the parameters of the target network; the target network is updated using a soft update strategy. to alleviate Fluctuations in value estimates; S4.3 The optimization of the Actor network is achieved through policy gradient ascent, and its gradient calculation formula is as follows:

[0020] The gradient reflects how the Critic network's evaluation of actions guides the direction of policy improvement. The Critic network uses the chain rule to... The gradient of the value pair is backpropagated to the Actor network, thereby adjusting the policy parameters. To maximize long-term cumulative rewards.

[0021] Beneficial technical effects of the present invention: To address the control problem of intelligent vehicles during trajectory tracking, this invention proposes a deep reinforcement learning-based trajectory tracking control method with a multi-objective reward function. By introducing a dual 5th-order polynomial lane-changing path planning algorithm, a vehicle path model is established, and a tracking error function is provided. A trajectory tracking control model is constructed by combining a vehicle dynamics model and a Markov decision process. At different vehicle speeds (36 km / h, 72 km / h, and 100 km / h), this algorithm significantly improves overall performance compared to the traditional DDPG method: the average lateral tracking error is reduced by approximately 40%, the maximum lateral acceleration is controlled within 0.55 m / s², and the fluctuation range of heading and yaw angle errors is reduced by over 30%, effectively balancing tracking accuracy, driving comfort, and system stability. This invention provides a reliable solution for high-performance trajectory tracking control of intelligent vehicles. This method has significant effects on improving tracking accuracy, stability, and comfort. Attached Figure Description

[0022] Figure 1 Lane-changing path map for intelligent vehicles; Figure 2 The trajectory of a double quintic polynomial; Figure 3 The overall algorithm framework diagram for the tracking control model; Figure 4 This is a diagram of the DDPG network architecture. Figure 5 This is a framework diagram of the simulation model; Figure 6 This is a diagram of the agent training process; Figure 7 This is a lateral position tracking diagram at 36 km / h; Figure 8 The lateral acceleration at 36 km / h; Figure 9 The heading angle error at 36 km / h; Figure 10 The yaw angle error at 36 km / h; Figure 11 Lateral position tracking at 72 km / h; Figure 12 The lateral acceleration at 72 km / h; Figure 13 The heading angle error at 72 km / h; Figure 14 The yaw angle error at 72 km / h; Figure 15 Lateral position tracking at 100 km / h; Figure 16 The lateral acceleration at 100 km / h; Figure 17 The heading angle error at 100 km / h; Figure 18 This represents the yaw angle error at 100 km / h. Detailed Implementation

[0023] The present invention will now be further described with reference to the accompanying drawings.

[0024] 1. Trajectory planning In the field of intelligent vehicle path planning, while path planning methods based on fifth-order polynomials can meet the basic requirements for trajectory continuity, they still suffer from problems such as large lane-changing safety distances and long lane-changing times in practical applications. These issues affect the safety and real-time performance of lane changes to some extent. To address this problem, this invention proposes a lane-changing path planning algorithm based on dual fifth-order polynomials. This algorithm introduces a transition state between the initial and target states, using the transition position as the endpoint of the first planning and the starting point of the second planning. The two paths are then fitted to generate the final lane-changing path. The lane-changing process is as follows: Figure 1 As shown, the obtained double quintic polynomial trajectory diagram is shown below. Figure 2 .

[0025] 1.1 First-order quintic polynomial When a vehicle changes lanes, during the first lane change, the trajectory of the vehicle at any given moment can be defined as a function of its lateral and longitudinal positions with respect to time t, resulting in the expression shown in (1):

[0026] Initial time conditions ( =0): Assume the vehicle starts at the origin, with an initial lateral velocity of 0, acceleration of 0, and longitudinal velocity of 0. ,Right now:

[0027] Intermediate time conditions ( Let the total lane change time be... When the vehicle terminates, its lateral displacement is equal to the lane width. The longitudinal displacement is And the velocity remains constant, while the acceleration is 0, that is:

[0028] By substituting the polynomial coefficients into the above equations, we obtain the specific motion equations for the first lane change:

[0029]

[0030] In the above expressions, Time taken to change lanes; and The initial speed and the speed after lane change are both 20 m / s; The distance traveled to overtake or change lanes; The lane width is set to 3.75m.

[0031] 1.2 Quadratic and quintic polynomials The final time condition of the above process is used as the initial time for the second return to the original lane. The final time condition is:

[0032] Based on the above equations, the specific motion equations for the second lane change are obtained:

[0033]

[0034] 1.3 Tracking Error Function After completing the double quintic polynomial programming, the reference trajectory needs to be tracked. When lateral trajectory tracking is involved, it is mainly achieved through lateral position error. and yaw rate deviation Tracking is performed, therefore a corresponding tracking error function is constructed. The current vehicle position information is obtained using sensors and other devices installed on the vehicle, and this information is compared with the lateral position information of the reference trajectory at the current moment to calculate the lateral position error. Since road curvature cannot be obtained through cameras, a planned reference trajectory curvature is used to replace the actual road curvature. The curvature expression can be calculated from the aforementioned reference trajectory as follows:

[0035] Combined with vehicle longitudinal speed and road curvature Find the desired yaw rate The yaw rate deviation can then be obtained by comparing the vehicle's current yaw rate. .

[0036] 2. Trajectory Tracking Control Model Combining the above trajectory tracking error function The DDPG algorithm is used to control the intelligent vehicle to complete trajectory tracking. The overall algorithm framework is as follows: Figure 3 As shown.

[0037] In the reinforcement learning training framework, the agent acts as the core controller, achieving precise vehicle tracking of the reference trajectory by adjusting the front wheel steering angle. A lane-changing tracking control model is constructed by integrating the vehicle's three-degree-of-freedom dynamics model with the DDPG framework. First, a reference trajectory is generated based on a double fifth-order polynomial, and then combined with vehicle position information and input into the tracking model to obtain the current state information. The output is given to the agent. A reward function is also constructed as an evaluation metric. The agent combines the state information and the reward value to output control actions. Control the vehicle to adjust its movement state, and then enter the next control state. The reward function evaluates and updates the reward based on state changes and action effects. This guides the intelligent agent to continuously optimize the control strategy. After training, the optimal control strategy is output to the decision layer, and based on the vehicle's motion state and reference trajectory information in the perception layer, precise control commands are finally output to achieve efficient trajectory tracking of the intelligent vehicle.

[0038] 2.1 Vehicle Dynamics Model A vehicle dynamics model is built based on vehicle trajectory tracking, considering only the roll motion along the x-axis, the lateral motion along the y-axis, and the yaw motion along the z-axis. According to Newton's second law, the vehicle dynamics equations are:

[0039]

[0040]

[0041]

[0042] in, For the quality of the car; It is the moment of inertia of the vehicle about the z-axis; These are the distances from the center of mass to the front and rear axles, respectively. It refers to the steering angle of the vehicle's front wheels; These are the stiffness of the front and rear tires, respectively. It is the lateral angle; It is the yaw rate; These are the longitudinal and lateral velocities, respectively.

[0043] 2.2 Markov Decision Process (1) Establishment of the state space: The state space provides a decision-making and evaluation framework for the autonomous driving system. It fully characterizes the vehicle's operating characteristics at a specific moment by selecting representative state variables. The state variables in vehicle trajectory tracking control mainly include the lateral error between the vehicle and the trajectory. Yaw rate error Horizontal error rate Yaw rate error rate Path curvature lateral speed of the vehicle Its state space can be defined as:

[0044] (2) Establishment of the motion space: In vehicle trajectory tracking control, the motion space defines the operational commands that the controlled vehicle can execute. Based on tracking the reference trajectory, this is mainly achieved by adjusting the front wheel steering angle. To achieve lateral motion control, based on a three-degree-of-freedom vehicle dynamics model, the controller transforms the front wheel steering angle command output by the reinforcement learning algorithm into adjustments to state parameters such as the vehicle yaw angle ψ, thereby causing changes in the vehicle's position information. Therefore, the action space of this control system can be defined as:

[0045] (3) Setting of multi-objective reward function: The reward function plays a key role in autonomous driving trajectory tracking control. It optimizes the control strategy by evaluating the decision-making effect of the agent. However, the traditional DDPG algorithm only considers the design of a single reward function. This paper introduces a multi-objective reward function. Based on the scenario set in this paper, the reward function is mainly used to adjust the front wheel steering angle. The reward mechanism is constructed based on key indicators, which can effectively guide the agent to learn the optimal control strategy.

[0046] For tracking accuracy, it can be evaluated based on lateral position error. The calculation of lateral position error is used as the basis for designing the tracking accuracy reward function, denoted as... A penalty is imposed when the lateral error exceeds 0.15m, and a positive reward (H) is given when the error decreases to 0.1m, encouraging the vehicle to reduce its error from the reference trajectory. The weighting parameter for this reward.

[0047]

[0048] For comfort, the smaller the front wheel steering angle, the smoother the change; therefore, a comfort reward function is designed based on the front wheel steering angle. :

[0049] To improve the efficiency of trajectory tracking, a positive reward related to lateral displacement is provided. This encourages vehicles to complete the tracking process as quickly as possible to ensure driving efficiency. If the lateral displacement of the vehicle remains 0 after 1 second, a penalty P is applied to ensure the vehicle completes the lane-changing process within the specified time. The reward function is set as follows:

[0050] Regarding the setting of the longitudinal speed reward function, adding a speed-maintaining reward function can significantly improve the overall performance of autonomous vehicles. By penalizing the deviation between the actual vehicle speed and the reference speed, it encourages the vehicle to maintain a stable longitudinal speed. When the speed deviation is within the allowable range (±1km / h), a fixed reward C is applied.

[0051] + C (18) Final comprehensive reward function for: = +M (19) Where M is the model penalty.

[0052] In addition, a termination condition is set for the control process: the lateral displacement of the vehicle relative to its initial position is 3.75m or the lateral error is satisfied. When the value is greater than 1m, terminate the current round, proceed to the next round, and reconfigure the environment state values; at the same time, when the termination condition is met, a penalty M is applied to prevent excessive lateral error.

[0053] The DDPG algorithm is a deep reinforcement learning algorithm based on the Actor-Critic framework, primarily used for problems with continuous action spaces. It combines the ideas of Q-learning and the policy gradient method, where the Actor network is responsible for determining the policy, while the Critic network is used to estimate the action-value function. The DDPG network architecture diagram is shown below. Figure 4 .

[0054] The core working mechanism of DDPG revolves around experience replay and the target network. In each interaction step, the Actor network adjusts its operation based on the current state. Generate Actions ,in These are the parameters of the Actor network. The environment returns a reward after receiving an action. and the next state and transfer samples Stored in the experience revisit buffer. The Critic network is trained based on minimizing temporal differences. Its loss function is defined as:

[0055] Among them, the target value The following was obtained through calculation using the target network: ,here and The target networks are Critic and Actor, respectively. As a discount factor, and These are the parameters of the target network. The target network is updated using a soft update strategy. This method of slowly tracking network parameters effectively alleviates the problem. Fluctuations in value estimation. The optimization of the Actor network is achieved through policy gradient ascent, and its gradient calculation formula is:

[0056] This gradient reflects how the Critic network's evaluation of actions guides the direction of policy improvement. The Critic uses the chain rule to... The gradient of the value pair is backpropagated to the Actor network, thereby adjusting the policy parameters. To maximize long-term cumulative rewards.

[0057] 3. Model Simulation and Analysis To verify the effectiveness of the proposed trajectory tracking control model based on deep reinforcement learning, a simulation environment was constructed using the powerful simulation platform MATLAB / Simulink, employing the DDPG algorithm and utilizing the Agent module in Simulink.

[0058] 3.1 Simulation Environment Setup The parameters used in the reinforcement learning simulation training are shown in Table 1 below. Table 1. Parameters used in reinforcement learning simulation training Parameter Value Discount factor 0.99 Actor learning rate 1 x 10 -4 ]] Critic learning rate 1 x 10 -3 ]] Soft update coefficient 1 x 10 -4 ]] Maximum number of training episodes 1500 Action noise variance 0.3 Noise decay rate 1 x 10 -5 ]] 3.2 Simulation Model Construction After configuring the DDPG network and simulation environment in MATLAB, a trajectory tracking control simulation model based on the DDPG algorithm needs to be built in Simulink. This involves setting the position of the controlled vehicle and its interaction with the agent, including its state, actions, and rewards. Figure 5 As shown, the simulation system mainly consists of a path planning module and a control module. The path planning module uses a double fifth-order polynomial algorithm to generate a reference trajectory. Error calculation integrates the vehicle's actual pose and the reference path information to obtain the lateral position deviation and heading angle deviation. After receiving these state observations, the DDPG controller outputs front wheel steering angle commands according to the trained control strategy to achieve accurate tracking of the vehicle trajectory. Throughout the control process, the system feeds back the action evaluation signal reflecting the control effect to the controller in the form of a reward value, forming a closed-loop control.

[0059] In the initial training phase, the reward value exhibited significant fluctuations, oscillating wildly from near -1000. This was because the agent was initially exploring through randomly selected actions and had not yet found an effective learning direction, leading to substantial changes in the reward value. In the subsequent 200-400 steps, while the reward value still fluctuated somewhat, it tended to stabilize. At this point, the agent began to gradually adjust towards a better direction, and after more than 400 training steps, the reward value gradually became relatively stable (see...). Figure 6 At this point, the algorithm gradually converges, and the agent learns an effective control strategy.

[0060] To verify the effectiveness of the designed control algorithm, the DDPG algorithm and the improved DDPG algorithm were used to track and control the reference trajectory, with initial vehicle speeds set at 36 km / h, 72 km / h, and 100 km / h. The simulation results were compared and analyzed. For the tracking control system, the control objective is to continuously reduce the error between the tracking trajectory and the reference trajectory by flexibly adjusting the control input of the front wheel steering angle, mainly including lateral position error. Therefore, it is required to reduce the lateral position error during the tracking of the reference trajectory.

[0061] The figures below show a comparison of the tracking performance, lateral position error, and longitudinal acceleration of the multi-target reward DDPG algorithm used in this paper and the traditional DDPG algorithm (considering only the lateral position error reward function) at vehicle speeds of 36 km / h, 72 km / h, and 100 km / h. Figure 7 , 11 As shown in Figure 15, under different vehicle speeds, the improved DDPG algorithm is significantly better than the traditional DDPG algorithm. In the initial stage, the traditional DDPG algorithm will exhibit jitter, which is because the traditional DDPG algorithm does not consider the tracking accuracy reward. However, the improved DDPG algorithm provides a very smooth and stable tracking trajectory throughout the entire tracking process.

[0062] from Figure 8 , 12 Figure 16 shows a comparison of the lateral acceleration of the intelligent vehicle at different speeds. Under DDPG control, the rate of change of lateral acceleration remains within a certain range throughout the simulation, ensuring driver comfort and preventing severe jolts. Furthermore, the improved DDPG algorithm does not exceed ±0.48m / s², ±0.5m / s², and ±0.55m / s² at 36km / h, 72km / h, and 100km / h, respectively. In contrast, the traditional DDPG algorithm can reach a maximum lateral acceleration of 0.55m / s² at only 36km / h, which is significantly higher than the improved DDPG algorithm. Although both algorithms can maintain a certain range, the lateral acceleration of the traditional DDPG algorithm is significantly too large, which may produce a large turning angle and cause discomfort to the occupants.

[0063] from Figure 9 , 13 From the perspective of 17, the improved DDPG algorithm exhibits a relatively smaller range of heading angle error fluctuation, with an average heading error of 0.05 rad at 72 km / h, and the error remaining close to 0 rad for most of the time; while the DDPG algorithm shows a larger error fluctuation. This indicates that the improved DDPG algorithm performs better in controlling the heading angle, and can more stably maintain the error within a smaller range, demonstrating better performance and stability compared to the traditional DDPG algorithm.

[0064] Figure 10 , 14 Figure 18 shows the variation of yaw angle error at different vehicle speeds. The yaw angle error of the DDPG algorithm fluctuates greatly. At 36km / h, 72km / h, and 100km / h, its maximum error is close to 0.28°, 0.35°, and 0.40°, respectively. In contrast, the maximum error of the improved DDPG algorithm is significantly lower, with absolute values ​​close to 0.18°, 0.21°, and 0.28°, respectively.

[0065] The proposed vehicle trajectory tracking control method based on deep reinforcement learning (DDPG) significantly improves the accuracy, stability, and comfort of intelligent vehicles in different speed scenarios by designing a multi-objective reward function mechanism through a dual quintic polynomial lane-changing path planning algorithm and the DDPG algorithm. Simulation experiments show that under different vehicle speed conditions (36km / h, 72km / h, 100km / h), the proposed method exhibits superior performance compared to the traditional DDPG algorithm in terms of tracking accuracy, lateral acceleration control, and stability of heading and yaw angles, verifying its effectiveness and robustness.

Claims

1. A method for intelligent vehicle trajectory tracking control based on a dual quintic polynomial and a multi-objective reward DDPG algorithm, characterized in that: Includes the following steps: S1. Use a double quintic polynomial to generate the motion trajectory equations for the first lane change and the second lane change. S2. Acquire tracking error signals, including lateral position error. e 1 and yaw rate deviation e 2 ; S3. Construct the DDPG control model, which involves state input, action output, and an innovative multi-objective reward function; S4. Train the DDPG algorithm: The DDPG algorithm accumulates interactive data through an experience replay mechanism and adopts a dual-objective network stabilization training process. S5. Conduct simulation training.

2. The intelligent vehicle trajectory tracking control method based on the dual quintic polynomial and multi-objective reward DDPG algorithm according to claim 1, characterized in that: The specific operation of S1 is as follows: S1.

1. Taking the vehicle's starting point as the initial state and the center point of the adjacent lane as the ending point, establish the motion trajectory equation for the first lane change, where the lateral displacement is equal to the lane width: in, Time taken for the first lane change; and These are the initial speed and the speed after changing lanes, respectively. The distance traveled to overtake or change lanes; Lane width; S1.

2. Using the end point of the first lane change as the starting point of the second lane change, and the target point of the original lane as the end point, establish the motion trajectory equation for the second lane change. The lateral displacement relative to the vehicle's starting point is zero. in, Time taken to return to the original lane; The speed at which you return to the original lane; The distance traveled to return to the original lane.

3. The intelligent vehicle trajectory tracking control method based on the dual quintic polynomial and multi-objective reward DDPG algorithm according to claim 1, characterized in that: The specific operation of S2 is as follows: Lateral trajectory tracking is achieved through lateral position error. e 1 and yaw rate deviation e 2 Track; Lateral position error e 1 The calculation method is as follows: The current vehicle position information is obtained using sensors and other equipment installed on the vehicle. This information is then compared with the lateral position information of the reference trajectory at the current moment to calculate the lateral position error. e 1 ; Yaw angular velocity deviation e 2 The calculation method is as follows: the curvature of the planned reference trajectory is used to replace the actual road curvature, and the curvature expression is as follows: The desired yaw rate is calculated by combining the vehicle's longitudinal velocity and the road curvature. The yaw rate deviation can then be obtained by comparing the vehicle's current yaw rate. .

4. The intelligent vehicle trajectory tracking control method based on the dual quintic polynomial and multi-objective reward DDPG algorithm according to claim 1, characterized in that: In S3, state input S t Lateral position error between vehicle and track yaw rate deviation Horizontal error rate Yaw rate error rate Path curvature lateral speed of the vehicle Composition; Action Output A t By adjusting the front wheel steering angle To achieve lateral motion control; The multi-objective reward function is optimized through the collaborative optimization of four sub-objectives, resulting in a final comprehensive reward function. for: = +M R1 represents the reward for tracking accuracy. For comfort rewards, For efficiency-based rewards, M represents the reward for vehicle speed stability, and M represents the model penalty. In addition, termination conditions are set for the control process, which are met when the lateral displacement of the vehicle relative to its initial position is equal to the lane width or the lateral error. If the value is greater than 1m, terminate the current round, proceed to the next round, and reconfigure the environment state values; at the same time, when the termination condition is met, a penalty M is applied to prevent excessive lateral error. The tracking accuracy reward is specifically as follows: Tracking accuracy is evaluated based on lateral position error, and the calculation of lateral position error is used as the basis for designing the tracking accuracy reward function, denoted as... A penalty is imposed when the lateral error exceeds 0.15m, and a positive reward (H) is given when the error decreases to 0.1m, encouraging the vehicle to reduce the error from the reference trajectory. The weighting parameters for this reward; The comfort reward is specifically defined as follows: the smaller the front wheel steering angle, the smoother the change. Therefore, a comfort reward function is designed based on the front wheel steering angle. ,in The weighting parameters for this reward are: Efficiency rewards specifically refer to: providing positive rewards related to lateral displacement. To encourage vehicles to complete the tracking process as quickly as possible and ensure driving efficiency, a penalty P is imposed if the lateral displacement of the vehicle is still 0 if the time exceeds 1 second, forcing the vehicle to complete the lane-changing process within the specified time. For this reward's weighting parameters, the reward function is set as follows: The vehicle speed stability bonus is defined as follows: Based on the longitudinal speed bonus function, a fixed bonus C is applied to penalize the deviation between the actual vehicle speed and the reference speed, provided the speed deviation is within the allowable range (±1 km / h). The weighting parameters for this reward are: + C。 5. The intelligent vehicle trajectory tracking control method based on the dual quintic polynomial and multi-objective reward DDPG algorithm according to claim 1, characterized in that: The specific operation of S4 is as follows: S4.1 In each interaction step, the Actor network adjusts its behavior according to the current state. Generate Actions ,in These are the parameters of the Actor network. The environment returns a reward after receiving an action. and the next state and transfer samples Store in the experience revisit buffer; S4.2 The training of the Critic network is based on minimizing the temporal difference, and the loss function is defined as: Among them, the target value The following was obtained through calculation using the target network: ,here and The target networks are Critic and Actor, respectively. As a discount factor, and These are the parameters of the target network; the target network is updated using a soft update strategy. to alleviate Fluctuations in value estimates; S4.3 The optimization of the Actor network is achieved through policy gradient ascent, and its gradient calculation formula is as follows: The gradient reflects how the Critic network's evaluation of actions guides the direction of policy improvement. The Critic network uses the chain rule to... The gradient of the value pair is backpropagated to the Actor network, thereby adjusting the policy parameters. To maximize long-term cumulative rewards.

Images