A driving control method for unmanned vehicle homing

Abstract

The present invention relates to the field of intelligent driving technology and discloses a driving control method for unmanned vehicle navigation, comprising: discretizing the path currently traveled by the unmanned vehicle; acquiring a target point and its attributes from the discrete path points by establishing a functional relationship between the foresight distance and the speed, wherein the attributes of the target point include the direction angle and the position of the target point; calculating the difference between the heading angle of the unmanned vehicle and the direction angle of the target point, recorded as a control angle; calculating the difference between the lateral position of the unmanned vehicle and the lateral coordinate of the target point; based on a deep reinforcement learning method, making the heading angle and lateral position of the unmanned vehicle consistent with the direction angle and the lateral coordinate of the target point, searching for the target point, and realizing training of a neural network; and using the trained neural network to make autonomous decisions. The present invention realizes lateral and longitudinal control of the unmanned vehicle in different driving tasks, and effectively improves trajectory tracking accuracy and stability.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a driving control method for an unmanned vehicle. Background Art

[0002] With the development of society and technology, the number of vehicles on the road has increased significantly in recent years. Autonomous driving is seen as an effective solution to alleviate traffic congestion and improve road safety. However, due to the complex and nonlinear control systems of autonomous vehicles and the ever-changing driving environment, autonomous vehicles are difficult to control, with poor control accuracy and stability.

[0003] The control of autonomous vehicles is mainly divided into lateral control and longitudinal control. Longitudinal movement is the focus of control in common driving tasks such as following a vehicle and passing through intersections. If the control strategy is not accurate enough or the response is not timely, it may cause the vehicle to collide with obstacles, and the traffic efficiency will also be affected. Lateral movement is also particularly important in the control of autonomous vehicles, especially for driving tasks such as lane changing, turning and parking. Existing technologies can only solve control problems in specific task scenarios, and the control accuracy is low. However, considering the diversity of driving tasks, a universal method is needed to adapt to the needs of various driving tasks. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a driving control method for an unmanned vehicle, which realizes lateral and longitudinal control of the unmanned vehicle in different driving tasks and effectively improves the trajectory tracking accuracy and stability.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for controlling the homing of an unmanned vehicle comprises the following steps:

[0007] Step 1: discretize the path currently traveled by the unmanned vehicle to obtain discrete path points; obtain the discrete path points within a set distance in front of the unmanned vehicle according to a set period;

[0008] Step 2: Determine the target point from the discrete path points by establishing a functional relationship between the foresight distance and the speed; obtain the attributes of the target point, which include the direction angle of the target point and the position of the target point; the position of the target point includes the horizontal coordinate and the vertical coordinate of the target point;

[0009] Step 3: Calculate the difference between the heading angle of the unmanned vehicle and the direction angle of the target point, which is recorded as the control angle; calculate the difference between the lateral position of the unmanned vehicle and the lateral coordinate of the target point, which is recorded as the lateral deviation; based on the deep reinforcement learning method, design the action space and observation space, construct the reward function, and train the neural network so that the heading angle and lateral position of the unmanned vehicle are consistent with the direction angle and lateral coordinate of the target point respectively, and search for the target point to achieve the training of the neural network;

[0010] Step 4: Use the trained neural network to make autonomous decisions: Input the unmanned vehicle's observation information into the trained neural network to obtain the optimal action corresponding to the optimal strategy; the unmanned vehicle acts according to the optimal action to achieve target point search; the observation information includes the size and direction of the unmanned vehicle's current control angle, as well as the size of the lateral deviation.

[0011] Furthermore, in step 2, obtaining the attributes of the target point specifically includes: the unmanned vehicle obtains the direction angle and position of the target point based on the on-board radar sensor, camera and inertial navigation system.

[0012] Furthermore, in step 2, determining the target point from the discrete path points by establishing a functional relationship between the forward sight distance and the speed specifically includes:

[0013] The functional relationship between the foresight distance and the speed is as follows:

[0014] dist=0.1×v

[0015] Among them, dist is the distance between the target point in front of the unmanned vehicle and the current position of the unmanned vehicle, and v is the current speed of the unmanned vehicle.

[0016] Furthermore, in step 3, the action space and observation space are designed based on the deep reinforcement learning method, the reward function is constructed, and the neural network is trained to make the heading angle and lateral position of the unmanned vehicle consistent with the heading angle and lateral coordinate of the target point, respectively, and the target point is searched to realize the training of the neural network, which specifically includes:

[0017] During the neural network training process, the unmanned vehicle is used as the vehicle agent. The action space of the vehicle agent is the steering angle, and the observation space of the vehicle agent is the control angle and lateral deviation.

[0018] The reward function r during training is:

[0019] r=r1+r2+r3;

[0020] in, represents the angle error penalty term; represents the lateral error penalty term; Represents items used to improve training efficiency;

[0021] The neural network adopts TD3 deep neural network;

[0022] The training process specifically includes:

[0023] First, initialize the TD3 deep neural network;

[0024] Secondly, the autonomous vehicle uses the TD3 deep neural network to decide its next action based on the state of itself and surrounding vehicles, and receives corresponding rewards and penalties. The current state, next state, actions taken, and rewards and penalties received by all vehicle agents are stored in the experience pool.

[0025] Finally, during the training process, samples are extracted from the experience pool based on the method of prioritized experience replay to train the evaluation network. The parameters of the evaluation network are updated by minimizing the loss function, and the parameters of the corresponding policy network are then updated using the gradient descent method. The target policy network and target evaluation network are updated using soft updates.

[0026] The loss function is:

[0027] Where, L i represents the loss function of the i-th evaluation network, N represents the number of samples extracted, Q represents the action value, θ i represents the parameters of the i-th evaluation network, s represents the state observation value, a represents the action, and y represents the predicted action value:

[0028]

[0029] In the formula, d represents the task completion flag, represents the action value calculated by the i-th target evaluation network, θ i ′ represents the parameters of the i-th target evaluation network, s′ represents the state observation value at the next moment, and a′ represents the action at the next moment.

[0030] Compared with the prior art, the beneficial technical effects of the present invention are:

[0031] The driving control method of the present invention involves the vehicle acquiring discrete path points within a certain distance ahead during each cycle. By establishing a functional relationship between forward-sight distance and speed, the vehicle locates the target point ahead. GPS and other equipment are then used to obtain the target point's attributes, including its azimuth, lateral coordinates, and longitudinal coordinates. Based on deep reinforcement learning methods, a neural network is trained to adjust the vehicle's heading angle so that its heading angle and lateral position are consistent with those of the target point. This method enables lateral and longitudinal control of unmanned vehicles in various driving tasks, effectively improving trajectory tracking accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 1 is a flow chart of a method for controlling a self-driving vehicle in accordance with an embodiment of the present invention;

[0033] Figure 2 2 is a schematic diagram of driving according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings, which are for reference and illustration purposes only and do not limit the scope of protection of the present invention.

[0035] like Figure 1 As shown, this embodiment discloses a driving control method for an unmanned vehicle, which includes the following steps.

[0036] Step 1: If Figure 2 As shown, the vehicle path is discretized, the interval between adjacent discrete path points is 0.1m, and the discrete path points within a certain distance ahead are obtained according to a set period. In this embodiment, the set period is 100ms.

[0037] The planned path is discretized into a series of points. At each moment, a point a certain distance ahead is chosen as the target point, or "d". Different driving tasks can be summarized as tracking "d". The control process is to ensure that the angle formed by the vehicle's heading and the tangent direction of the trajectory at the target point is zero.

[0038] Step 2: Find the target point ahead by establishing a functional relationship between the forward sight distance and the speed, and use GPS and other equipment to obtain the attributes of the target point, including the direction angle, horizontal coordinates, and vertical coordinates.

[0039] The functional relationship between the forward sight distance and the speed is as follows:

[0040] dist = 0.1 × v;

[0041] Among them, dist is the distance between the target point and the current position of the unmanned vehicle, and v is the current speed of the vehicle.

[0042] like Figure 2 As shown, the control angle e formed with the target point is obtained δ , and lateral deviation e y , used for training neural networks.

[0043] Step 3: Based on the deep reinforcement learning method, the action space and observation space are designed, the reward function is constructed, and the neural network is trained to make the heading angle and lateral position of the unmanned vehicle consistent with the heading angle and lateral coordinate of the target point, respectively, to achieve the purpose of finding the target point, that is, to achieve the purpose of trajectory tracking.

[0044] During the neural network training process, the unmanned vehicle is regarded as the vehicle agent, the action space of the vehicle agent is the steering angle, and the observation space of the vehicle agent is the control angle and lateral deviation.

[0045] The reward function r during training is as follows:

[0046] r=r1+r2+r3;

[0047] in,

[0048]

[0049] In this embodiment, the neural network may adopt the TD3 deep neural network.

[0050] The training process specifically includes:

[0051] First, initialize the TD3 deep neural network.

[0052] Next, the autonomous vehicle uses the TD3 deep neural network to determine its next action based on the state of itself and surrounding vehicles, and to determine the corresponding rewards and penalties. The current state, next state, actions taken, and rewards and penalties earned by all vehicle agents are stored in an experience pool.

[0053] Finally, during training, samples are drawn from the experience pool based on a prioritized experience replay method to train the evaluation network. The evaluation network parameters are updated by minimizing the loss function. The corresponding policy network parameters are then updated using gradient descent, and the target policy network and target evaluation network are updated using soft updates.

[0054] The method based on priority experience replay stores the current state, next state, actions taken, and rewards and penalties obtained of all vehicle agents in the form of tuples in the experience pool. During training, more important experiences will be preferentially extracted for training, thereby accelerating convergence.

[0055] The loss function is:

[0056]

[0057] Where, L i represents the loss function of the i-th evaluation network, N represents the number of samples extracted, Q represents the action value, θ i represents the parameters of the i-th evaluation network, s represents the state observation value, a represents the action, and y represents the predicted action value:

[0058]

[0059] In the formula, r represents the reward obtained, d represents the task completion mark, represents the action value calculated by the i-th target evaluation network, θ i ′ represents the parameters of the i-th target evaluation network, s′ represents the state observation value at the next moment, and a′ represents the action at the next moment.

[0060] Step 4: Use the trained neural network to make autonomous decisions. By inputting observation information into the trained neural network, the optimal action corresponding to the optimal strategy is obtained. The unmanned vehicle then acts according to the action, achieving target location.

[0061] Specifically, at each time step, the unmanned vehicle observes the size and direction of the current control angle, as well as the size of the lateral deviation. Based on the observed information, the policy network outputs the corresponding optimal action, that is, the action with the largest action value.

[0062] As shown in Table 1, the test results show that the unmanned vehicle driving control method of the present invention can achieve accurate trajectory tracking in the following, double lane change, horizontal parking, 45-degree inclined parking, and vertical parking scenarios.

[0063] Table 1 shows the test results of the homing driving control technology in different driving tasks.

[0064]

[0065] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. It is intended that all variations within the meaning and range of equivalents of the claims be embraced herein, and any reference signs in the claims should not be construed as limiting the claims to which they relate.

[0066] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A driving control method for an unmanned vehicle, characterized in that: The following steps are involved: Step 1: discretize the path currently traveled by the unmanned vehicle to obtain discrete path points; obtain the discrete path points within a set distance in front of the unmanned vehicle according to a set period; Step 2: Determine the target point from the discrete path points by establishing a functional relationship between the foresight distance and the speed; obtain the attributes of the target point, which include the direction angle of the target point and the position of the target point; the position of the target point includes the horizontal coordinate and the vertical coordinate of the target point; Step 3: Calculate the difference between the heading angle of the unmanned vehicle and the direction angle of the target point, and record it as the control angle; Calculate the difference between the lateral position of the unmanned vehicle and the lateral coordinate of the target point, which is recorded as the lateral deviation; Based on deep reinforcement learning methods, the action space and observation space are designed, the reward function is constructed, and the neural network is trained to ensure that the heading angle and lateral position of the unmanned vehicle are consistent with the heading angle and lateral coordinates of the target point, respectively. The target point is searched and the neural network training is implemented. Specifically, During the neural network training process, the unmanned vehicle is used as the vehicle agent, the action space of the vehicle agent is the steering angle, and the observation space of the vehicle agent is the control angle. and lateral deviation ; Reward function during training for: ； in, , represents the angle error penalty term; , represents the lateral error penalty term; , represents the item used to improve training efficiency; The neural network adopts TD3 deep neural network; The training process specifically includes: First, initialize the TD3 deep neural network; Secondly, the autonomous vehicle uses the TD3 deep neural network to decide its next action based on the state of itself and surrounding vehicles, and receives corresponding rewards and penalties. The current state, next state, actions taken, and rewards and penalties received by all vehicle agents are stored in the experience pool. Finally, during the training process, samples are extracted from the experience pool based on the method of prioritized experience replay to train the evaluation network. The parameters of the evaluation network are updated by minimizing the loss function, and the parameters of the corresponding policy network are then updated using the gradient descent method. The target policy network and target evaluation network are updated using soft updates. The loss function is: ; Where, Indicates the The loss function of the evaluation network is represents the number of samples drawn, represents the action value, Indicates the The parameters of the evaluation network, represents the state observation value, Indicates action, Represents the predicted action value: ； Where, Indicates that the task is completed. Indicates the The action value calculated by the target evaluation network, Indicates the The parameters of the target evaluation network, represents the state observation value at the next moment, Indicates the action at the next moment; Step 4: Use the trained neural network to make autonomous decisions: Input the unmanned vehicle's observation information into the trained neural network to obtain the optimal action corresponding to the optimal strategy; the unmanned vehicle acts according to the optimal action to achieve target point search; the observation information includes the size and direction of the unmanned vehicle's current control angle, as well as the size of the lateral deviation.

2. The method for controlling the homing of an unmanned vehicle according to claim 1, characterized in that: In step 2, obtaining the attributes of the target point specifically includes: the unmanned vehicle obtains the direction angle and position of the target point based on the on-board radar sensor, camera and inertial navigation system.

3. The method for controlling the homing of an unmanned vehicle according to claim 1, characterized in that: In step 2, determining the target point from the discrete path points by establishing a functional relationship between the forward sight distance and the speed specifically includes: The functional relationship between the foresight distance and the speed is as follows: in, is the distance between the target point in front of the unmanned vehicle and the current position of the unmanned vehicle, is the current speed of the autonomous vehicle.

Images