An unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning

Abstract

The application relates to a high-precision trajectory tracking control method for an unmanned vehicle based on deep reinforcement learning, and belongs to the technical field of trajectory tracking control of unmanned vehicles. The method comprises the following steps: obtaining target path information of the unmanned vehicle and current pose state information of the unmanned vehicle; determining a pose error of the unmanned vehicle according to the target path information of the unmanned vehicle and the current pose state information of the unmanned vehicle; designing a reinforcement learning state space, an action space and a reward function; constructing a reinforcement learning algorithm network based on the current pose state information of the unmanned vehicle and the pose error of the unmanned vehicle, and generating an action instruction through a reinforcement learning intelligent agent; inputting the action instruction into a vehicle dynamics model, controlling the unmanned vehicle to perform trajectory tracking, and feeding back pose state information output by the vehicle dynamics model for trajectory tracking control at the next moment. The application can solve the problems of weak dynamic adaptability and insufficient precision of a traditional trajectory tracking method for an unmanned vehicle in a complex dynamic environment.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned vehicle trajectory tracking control, and particularly relates to an unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning. BACKGROUND

[0002] The rise of automatic driving technology provides a key solution to cracking traffic safety pain points and relieving road congestion problems. As the core execution link of the automatic driving system, the performance of trajectory tracking control directly determines whether the vehicle can accurately follow the preset path and realize safe and stable driving in all scenarios. It is also the core premise and key support for intelligent vehicles to break through technical bottlenecks and move towards commercialization.

[0003] Traditional unmanned vehicle trajectory tracking control methods are mostly based on accurate mathematical models. For example, PID control is widely used due to its simple structure and easy implementation, but this method has poor adaptability to model parameter changes and external disturbances, and it is difficult to ensure tracking accuracy in complex road conditions. Linear quadratic regulator obtains the optimal control law by solving Riccati equation, but its performance depends on the accuracy of the linearization model, and for vehicle systems with strong nonlinear characteristics, the control effect is often unsatisfactory. Although model predictive control can handle constraints, it has high computational complexity and relies on accurate models and environmental information for future state prediction, which can easily lead to performance degradation in scenes with dramatic changes.

[0004] Under this background, reinforcement learning technology provides a new path to solve the above bottlenecks due to its unique advantages. Unlike traditional methods, reinforcement learning uses a self-learning mechanism of "trial and error-feedback-optimization" to establish a direct mapping from environmental state to control action without relying on pre-constructed accurate mathematical models. At the same time, its real-time adaptive capability can optimize the control strategy through continuous learning to dynamically respond to complex situations such as changes in road adhesion coefficient and sudden changes in path curvature, significantly improving the robustness and accuracy of control. Considering the limitations of existing technologies and actual application needs, researching an unmanned vehicle trajectory tracking control strategy based on deep reinforcement learning has important theoretical value and practical significance for breaking through the bottlenecks of automatic driving control technology and ensuring safe driving in complex scenarios. SUMMARY

[0005] The purpose of the present application is to solve the problem of weak dynamic adaptability and insufficient precision of traditional unmanned vehicle trajectory tracking methods in complex dynamic environments, and to provide an unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning.

[0006] To achieve the above purpose, the technical solution of the present application is: an unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning, comprising:

[0007] S1, obtaining target path information of the unmanned vehicle and current pose state information of the unmanned vehicle;

[0008] S2, determining a pose error of the unmanned vehicle according to the target path information of the unmanned vehicle and the current pose state information of the unmanned vehicle;

[0009] S3, designing a reinforcement learning state space, an action space and a reward function based on the target path information of the unmanned vehicle, the current pose state information of the unmanned vehicle and the pose error of the unmanned vehicle;

[0010] S4, constructing a reinforcement learning algorithm network based on the current pose state information of the unmanned vehicle and the pose error of the unmanned vehicle, and generating an action instruction through a reinforcement learning agent;

[0011] S5, inputting the action instruction into a vehicle dynamics model, controlling the unmanned vehicle to perform trajectory tracking, and feeding back pose state information output by the vehicle dynamics model for trajectory tracking control at the next time.

[0012] Further, in step S3, the reinforcement learning state space is defined as:

[0013]

[0014] wherein is a longitudinal speed of the unmanned vehicle, describing a forward dynamic along the expected trajectory; is an expected heading angle, serving as a direction reference for trajectory tracking; represents a lateral position error of the current position of the unmanned vehicle and a target point of the expected path, directly measuring a lateral deviation of the trajectory; is a heading angle error, representing a difference between the current heading angle of the unmanned vehicle and the expected heading angle, reflecting a directional tracking deviation; to break through the control hysteresis depending on only the current error, a differential compensation of the lateral position error and a differential compensation of the heading angle error are introduced, and the reinforcement learning state space is redefined as:

[0015]

[0016] wherein the differential compensation term is:

[0017] , .

[0018] Further, in step S3, the reward function is a dual-mechanism collaborative reward function, including a regionalized reward function based on a smooth step function and an adaptive weight reward function based on a Gaussian kernel function.

[0019] Further, the regionalized reward function based on the smooth step function constructs continuous and derivable multi-level reward and punishment regions through a Sigmoid-type smooth step function, and the regionalized reward function based on the smooth step function is:

[0020]

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] wherein represents the Sigmoid-type smooth step function adopted, represents the lateral position error, represents the heading angle error; the lateral position error four-level control domains are adopted: A: when less than 0.2 m (high-precision area), a high reward value is given; when between 0.2 and 0.5 m (stable tracking area), a medium reward is given; B: when between 1 and 2 m (pre-warning punishment area), a medium punishment is given; when greater than 2 m (dangerous strong punishment area), a high punishment is given; the heading angle error three-level safety boundaries are adopted: C: when the heading angle error less than 5 degrees, a high reward is given; when between 5 and 10 degrees, a medium reward is given; D: when the heading angle error greater than 20 degrees, a strong punishment is given.

[0027] Further, the adaptive weight reward function based on the Gaussian kernel function dynamically adjusts the punishment intensity of the lateral position error, the heading angle error and the front wheel steering angle through adaptive weight, and the adaptive weight reward function based on the Gaussian kernel function is:

[0028]

[0029]

[0030]

[0031]

[0032] wherein is a lateral position error is a corresponding adaptive weight, is a heading angle error is a corresponding adaptive weight, is a front wheel steering angle, is a current error, is a threshold parameter; the adaptive weight is dynamically adjusted using function: when error tends to 0, the weight tends to 2.0, which increases the loss function gradient and forces the controller to fine-tune the small error; when , (i.e., when the error is large), the weight tends to 1.0, which avoids system oscillation caused by excessive correction and ensures training stability.

[0033] Further, in step S4, the reinforcement learning algorithm network is constructed, including constructing a deep deterministic policy gradient algorithm network DDPG and a double-delay deep deterministic policy gradient algorithm network TD3.

[0034] Further, in step S4, the action instruction is generated by the reinforcement learning agent, including: constructing a reinforcement learning state space according to the pose error of the unmanned vehicle and the pose state information of the unmanned vehicle, setting the action space as the front wheel steering angle, using a reinforcement learning algorithm with an Actor-Critic structure, wherein the Actor network is responsible for learning the policy function, and outputs the front wheel steering angle that meets the constraint condition, and the Critic network evaluates the value of the current policy through the learning value distribution target function; the target reward value is calculated by combining the reward function, the policy function and the value function are optimized through continuous interaction learning, and finally the optimal front wheel steering angle that adapts to the current trajectory tracking state is output as the action instruction.

[0035] The application further provides an edge computing terminal device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning when executing the program.

[0036] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program implements the steps of the unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning when executed by a processor.

[0037] The application further provides an electronic device, which comprises a processor and a memory, wherein the memory stores a computer program, and the computer program causes the processor to execute the steps of the unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning when executed by the processor.

[0038] Compared with the prior art, the present application has the following beneficial effects: the present application provides a more flexible, intelligent and robust solution for unmanned vehicle trajectory tracking by deeply integrating reinforcement learning with unmanned vehicle trajectory tracking control. The method uses the decision mechanism of reinforcement learning to optimize the control strategy of unmanned vehicle trajectory tracking, so that it can better adapt to complex and variable traffic environments and task requirements. During the actual driving of the unmanned vehicle, the optimal action instruction can be adjusted in real time according to the current pose error and other states, so as to ensure that the unmanned vehicle accurately and safely tracks the expected trajectory. The present application not only solves the problems of weak dynamic adaptability and insufficient precision of traditional unmanned vehicle trajectory tracking methods in complex dynamic environments, but also improves the processing capacity and control stability of unmanned vehicles in different trajectory tracking scenarios, providing strong support for high-precision and safe driving of unmanned vehicles in logistics transportation, intelligent transportation and other fields. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 A flowchart of an unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning provided for the embodiments of the present application;

[0040] Figure 2 An Actor neural network structure for the embodiments of the present application;

[0041] Figure 3 A control framework diagram of an unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning provided for the embodiments of the present application;

[0042] Figure 4 Experimental results of the DDPG algorithm for the embodiments of the present application in straight line trajectory tracking;

[0043] Figure 5 Experimental results of the TD3 algorithm for the embodiments of the present application in straight line trajectory tracking;

[0044] Figure 6 Experimental results of the DDPG algorithm for the embodiments of the present application in sinusoidal trajectory tracking;

[0045] Figure 7 Experimental results of the TD3 algorithm for the embodiments of the present application in sinusoidal trajectory tracking. DETAILED DESCRIPTION

[0046] To make the features and advantages of the present application more obvious and easy to understand, the following embodiments are specifically described as follows:

[0047] It should be pointed out that the following detailed description is exemplary and is intended to provide further description of the present application. Unless otherwise specified, all technical and scientific terms used in the present description have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0048] It is to be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments consistent with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0049] An embodiment of the present application provides a high-precision trajectory tracking control method for unmanned vehicles based on deep reinforcement learning, a flowchart of which is shown in Figure 1 The method comprises the following steps:

[0050] S1, obtaining target path information of the unmanned vehicle and current pose state information of the unmanned vehicle;

[0051] S2, determining a pose error of the unmanned vehicle according to the target path information of the unmanned vehicle and the current pose state information of the unmanned vehicle;

[0052] S3, designing a reinforcement learning state space, an action space and a reward function based on the target path information of the unmanned vehicle, the current pose state information of the unmanned vehicle and the pose error of the unmanned vehicle;

[0053] S4, constructing a reinforcement learning algorithm network based on the current pose state information of the unmanned vehicle and the pose error of the unmanned vehicle, and generating an action instruction through a reinforcement learning agent;

[0054] S5, inputting the action instruction into a vehicle dynamics model, controlling the unmanned vehicle to perform trajectory tracking, and feeding back pose state information output by the vehicle dynamics model for trajectory tracking control at the next moment.

[0055] As a preferred scheme of the embodiment, in step S3, the reinforcement learning state space is defined as:

[0056]

[0057] wherein is a longitudinal speed of the unmanned vehicle, describing a forward dynamic along the expected trajectory; is an expected heading angle, serving as a direction reference for trajectory tracking; represents a lateral position error of the unmanned vehicle at a current position and an expected path target point, directly measuring a lateral deviation of the trajectory; is a heading angle error, representing a difference between a current heading angle of the unmanned vehicle and the expected heading angle, reflecting a directional tracking deviation; to break through the control hysteresis depending on only the current error, a differential compensation of the lateral position error and a differential compensation of the heading angle error Then the reinforcement learning state space is redefined as:

[0058]

[0059] where the differential compensation term is:

[0060]

[0061] As a preferred solution of the embodiment, in step S3, the reward function is a dual-mechanism collaborative reward function, including a regionalized reward function based on a smooth step function and an adaptive weight reward function based on a Gaussian kernel function.

[0062] The regionalized reward function based on the smooth step function constructs a continuous and derivable multi-level reward and punishment region through a Sigmoid-type smooth step function, and the regionalized reward function based on the smooth step function is:

[0063]

[0064]

[0065]

[0066]

[0067]

[0068]

[0069] wherein represents the Sigmoid-type smooth step function adopted, represents the lateral position error, represents the heading angle error; the lateral position error A four-level control domain is adopted: A: when is less than 0.2 m (high-precision area), a high reward value is given; when is between 0.2 and 0.5 m (stable tracking area), a medium reward is given; B: when is between 1 and 2 m (pre-warning punishment area), a medium punishment is given; when is greater than 2 m (dangerous strong punishment area), a high punishment is given; the heading angle error A three-level safety boundary is adopted: C: when the heading angle error is less than 5 degrees, a high reward is given; when is between 5 degrees and 10 degrees, a medium reward is given; D: when the heading angle error is greater than 20 degrees, a strong punishment is given.

[0070] ​​The adaptive weight reward function based on the Gaussian kernel function dynamically adjusts the punishment intensity of the lateral position error, the heading angle error and the front wheel steering angle through adaptive weight, and the adaptive weight reward function based on the Gaussian kernel function is:

[0071]

[0072]

[0073]

[0074]

[0075] wherein is the lateral position error is the corresponding adaptive weight, is the heading angle error is the corresponding adaptive weight, is the front wheel steering angle, is the current error, is a threshold parameter; the adaptive weight dynamic adjustment adopts function: when (i.e. the error tends to 0), the weight tends to 2.0, which increases the loss function gradient and forces the controller to finely correct the small error; when (i.e. in the case of a larger error), the weight tends to 1.0, which avoids system oscillation caused by excessive correction and guarantees the training stability.

[0076] As a preferred scheme of the embodiment, in step S4, the reinforcement learning algorithm network is constructed, including constructing a deep deterministic policy gradient algorithm network DDPG and a double-delay deep deterministic policy gradient algorithm network TD3.

[0077] As a preferred scheme of the embodiment, in step S4, the action instruction is generated by the reinforcement learning agent, including: constructing a reinforcement learning state space according to the pose error and the pose state information of the unmanned vehicle. The action space is defined as the continuous value range of the front wheel steering angle . The reinforcement learning algorithm with an Actor-Critic structure is adopted, wherein the policy network (Actor) adopts a full connection architecture, such as Figure 2As shown, this policy network constructs a multi-level architecture for achieving autonomous vehicle trajectory tracking and control. The input layer receives the autonomous vehicle's state information. Subsequently, a fully connected layer performs initial feature extraction and transformation, followed by a ReLU activation layer to introduce nonlinear factors, enhancing the network's ability to express complex trajectory patterns. Then, another fully connected layer further integrates and optimizes the features. The Tanh activation layer performs nonlinear mapping on the features, making the output more adaptable to subsequent processing requirements. The scaling layer adjusts the signal scale, allowing data to be transmitted within an appropriate range. The entire network, through the combination of a multi-level structure and different activation functions, finally outputs the autonomous vehicle's front wheel steering angle that satisfies the constraints. The Critic network learns the value distribution objective function. The value of the current strategy in trajectory tracking tasks is evaluated. Then, a regionalized reward function based on a smooth step function is combined. and adaptive weighted reward function based on Gaussian kernel function Dual-mechanism collaborative reward function The target reward value is calculated after each action interaction. Finally, through continuous interactive learning between the agent and the autonomous vehicle control environment, the policy function of the Actor network and the value function of the Critic network are iteratively optimized. Ultimately, the optimal front wheel steering angle adapted to the current trajectory tracking state is output as the action command, enabling the autonomous vehicle to accurately track the target trajectory.

[0078] like Figure 3 The diagram illustrates a deep reinforcement learning-based trajectory tracking control framework for autonomous vehicles. The modules work together to form a closed-loop system of "perception-decision-control." The path information of the target path is compared with the pose state information fed back from the vehicle dynamics model. The resulting deviation is input into the tracking module. The pose error output by the tracking module is transmitted to the reward module to generate reward information and also sent to the observation module to form the observation state. The reward information and the observation state are jointly input into the reinforcement learning agent. The agent iteratively optimizes the strategy through learning and outputs action commands to the vehicle dynamics model to drive the vehicle's movement. The vehicle dynamics model then feeds back the new pose state information to the comparison stage. This iterative process achieves accurate dynamic tracking of the target path by the autonomous vehicle, fully leveraging the adaptive decision-making advantages of reinforcement learning in complex scenarios and the role of multi-module closed-loop interaction in ensuring the accuracy and stability of trajectory tracking.

[0079] To more intuitively illustrate the effectiveness and feasibility of the methods proposed in the embodiments of this invention, this invention selects straight lines and sinusoidal trajectories as reference trajectories, and performs reinforcement learning training based on the DDPG and TD3 deep reinforcement learning algorithms according to the methods proposed above. Then, the trained model is used for simulation verification. In the straight line trajectory tracking task, the experimental results are as follows: Figure 4 and Figure 5As shown, under the proposed method, the tracking trajectories based on DDPG and TD3 can quickly and smoothly approach the reference trajectory, the transition in the initial stage is smooth and natural, and the subsequent continuous and stable fitting has no obvious fluctuation or deviation, and the precision performance is excellent. In the sinusoidal fluctuation trajectory tracking task, the experimental results are as shown in Figure 6 and Figure 7 As shown, the tracking trajectories of DDPG and TD3 are highly consistent with the reference trajectory,

[0080] The present application can not only accurately capture the overall fluctuation period of the sinusoidal trajectory, but also highly coincide with the reference trajectory at key dynamic nodes such as the peak of the wave and the lowest point of the trough, with minimal deviation. Whether it is the slope matching in the rising stage of the trajectory or the smooth transition in the falling stage, it perfectly matches the dynamic change law of the reference trajectory, and there is no "losing" or "lagging" situation caused by insufficient dynamic adaptability. In summary, the proposed method significantly enhances the dynamic adaptability of the trajectory tracking of the unmanned vehicle, greatly improves the tracking precision and stability, and provides strong support for the reliable driving of the unmanned vehicle in complex trajectory scenarios.

[0081] The present application also provides an edge computing terminal device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned high-precision trajectory tracking control method for unmanned vehicles based on deep reinforcement learning when executing the program.

[0082] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the above-mentioned high-precision trajectory tracking control method for unmanned vehicles based on deep reinforcement learning.

[0083] The present application also provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the above-mentioned high-precision trajectory tracking control method for unmanned vehicles based on deep reinforcement learning.

[0084] The above-mentioned is only a preferred embodiment of the present application, and is not a limitation on other forms of the present application. Any person skilled in the art can modify or change the above-mentioned disclosed technical content into equivalent embodiments with equivalent changes. However, any simple modification, equivalent change and modification of the above-mentioned embodiments without departing from the technical solution content of the present application, and according to the technical essence of the present application, still belongs to the protection scope of the technical solution of the present application.

[0085] The patent is not limited to the above best mode, and anyone can derive other various forms of a high-precision trajectory tracking control method for an unmanned vehicle based on deep reinforcement learning under the inspiration of the patent, and any equivalent changes and modifications made in the patent application scope shall be covered by the patent.

Claims

1. A deep reinforcement learning-based high-precision trajectory tracking control method for an unmanned vehicle, characterized in that, The method comprises the following steps: S1, obtaining target path information of the unmanned vehicle and current pose state information of the unmanned vehicle; S2, determining a pose error of the unmanned vehicle according to the target path information of the unmanned vehicle and the current pose state information of the unmanned vehicle; S3, designing a reinforcement learning state space, an action space and a reward function based on the target path information of the unmanned vehicle, the current pose state information of the unmanned vehicle and the pose error of the unmanned vehicle; S4, constructing a reinforcement learning algorithm network based on the current pose state information of the unmanned vehicle and the pose error of the unmanned vehicle, and generating an action instruction through a reinforcement learning agent; S5, inputting the action instruction into a vehicle dynamics model to control the unmanned vehicle to perform trajectory tracking, and feeding back pose state information output by the vehicle dynamics model for trajectory tracking control at the next moment; In step S3, the reward function is a dual-mechanism collaborative reward function, which comprises a regionalized reward function based on a smooth step function and an adaptive weight reward function based on a Gaussian kernel function; the regionalized reward function based on the smooth step function constructs a continuous and derivable multi-level reward and punishment region through a Sigmoid-type smooth step function, and the regionalized reward function based on the smooth step function is: wherein represents a sigmoid-type smoothing step function employed, represents a lateral position error, represents a heading angle error; lateral position error Four levels of control zones are employed: A: when a high reward value is given when the lateral position error is less than 0.2 m; when the lateral position error is between 0.2 and 0.5 m, a medium reward is given; B: When Medium penalty is given when between 1 and 2 m; when High penalty is given when greater than 2 m; heading angle error Three levels of safety margin are adopted: C: When heading angle error High reward is given when less than 5 degrees; when Medium reward is given when between 5 and 10 degrees; D: When the heading angle error is greater than 20 degrees, a strong penalty is given; The adaptive weight reward function based on the Gaussian kernel function dynamically adjusts the punishment intensity of the lateral position error, the heading angle error and the front wheel steering angle through adaptive weight, and the adaptive weight reward function based on the Gaussian kernel function is: wherein is a lateral position error is a corresponding adaptive weight, is a heading angle error is a corresponding adaptive weight, is a front wheel steering angle, is a current error, is a threshold parameter; Adaptive weight dynamic adjustment employs Function: When , the weight approaches 2.0; when , the weight approaches 1.

0.

2. The high-precision trajectory tracking control method for an unmanned vehicle based on deep reinforcement learning according to claim 1, characterized in that, In step S3, the reinforcement learning state space is defined as: wherein is the longitudinal velocity of the unmanned vehicle, describing the forward dynamic along the desired trajectory; is the desired heading angle, as the directional reference for trajectory tracking; represents the lateral position error between the current position of the unmanned vehicle and the target point of the desired path, directly measuring the lateral deviation of the trajectory; is the heading angle error, representing the difference between the current heading angle of the unmanned vehicle and the desired heading angle, reflecting the directional tracking deviation; to break through the control hysteresis of relying only on the current error, the differential compensation of the lateral position error is introduced and the differential compensation of the heading angle error The reinforcement learning state space is then redefined as: wherein the differential compensation term is: ， 。 3. The high-precision trajectory tracking control method for an unmanned vehicle based on deep reinforcement learning according to claim 1, characterized in that, In step S4, constructing the reinforcement learning algorithm network comprises constructing a deep deterministic policy gradient algorithm network DDPG and a double-delay deep deterministic policy gradient algorithm network TD3.

4. The high-precision trajectory tracking control method for an unmanned vehicle based on deep reinforcement learning according to claim 1, characterized in that, In step S4, the action instruction is generated through the reinforcement learning agent, which comprises: constructing a reinforcement learning state space according to the pose error of the unmanned vehicle and the pose state information of the unmanned vehicle, setting the action space as the front wheel steering angle, adopting a reinforcement learning algorithm with an Actor-Critic structure, wherein the Actor network is responsible for learning a policy function, outputs the front wheel steering angle meeting the constraint condition, and the Critic network evaluates the value of the current policy through learning a value distribution target function; the target reward value is calculated in combination with the reward function, the policy function and the value function are optimized through continuous interaction learning, and finally the optimal front wheel steering angle adapting to the current trajectory tracking state is output as the action instruction.

5. An edge computing terminal device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning according to any one of claims 1-4 when executing the program.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the unmanned vehicle high-precision trajectory tracking control method based on deep reinforcement learning according to any one of claims 1-4.

7. An electronic device comprising a processor and a memory, wherein, The memory stores a computer program, when the computer program is executed by the processor, makes the processor execute the steps of the deep reinforcement learning based high-precision trajectory tracking control method of the unmanned vehicle according to any one of claims 1-4.

Images