Unmanned vehicle homing driving control method

Abstract

The invention relates to the technical field of intelligent driving, and discloses a homing driving control method for an unmanned vehicle, and the method comprises the steps: carrying out the discretization of a current driving path of the unmanned vehicle; a target point and attributes of the target point are obtained from the discrete path points by establishing a function relation between the foresight distance and the speed, and the attributes of the target point comprise the direction angle and the position of the target point; calculating a difference value between the course angle of the unmanned vehicle and the direction angle of the target point, and recording the difference value as a control angle; calculating a difference value between the transverse position of the unmanned vehicle and the transverse coordinate of the target point; based on a deep reinforcement learning method, enabling the course angle and the transverse position of the unmanned vehicle to be consistent with the direction angle and the transverse coordinate of the target point respectively, and carrying out target searching on the target point to realize training of a neural network; performing autonomous decision making by using the trained neural network; according to the invention, transverse and longitudinal control of the unmanned vehicle in different driving tasks is realized, and trajectory tracking precision and stability are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a driving control method for an unmanned vehicle. Background Art

[0002] With the development of society and technology, the number of vehicles on the road has increased significantly in recent years. Autonomous driving is seen as an effective solution to alleviate traffic congestion and improve road safety. Due to the complex and nonlinear control system of autonomous vehicles and the changing driving environment, autonomous vehicles are difficult to control and have poor control accuracy and stability.

[0003] The control of autonomous vehicles is mainly divided into lateral control and longitudinal control. Longitudinal movement is the focus of control in common driving tasks such as following a vehicle and passing through an intersection. If the control strategy is not accurate enough or the response is not timely, it may cause the vehicle to collide with obstacles, and the traffic efficiency will also be affected. Lateral movement is also particularly important in the control of autonomous vehicles, especially for driving tasks such as lane changing, turning and parking. Existing technologies can only solve control problems in specific task scenarios, and the control accuracy is low. However, considering the diversity of driving tasks, a general method is needed to meet the needs of various driving tasks. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a driving control method for an unmanned vehicle, which can realize lateral and longitudinal control of the unmanned vehicle in different driving tasks and effectively improve the trajectory tracking accuracy and stability.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A driving control method for an unmanned vehicle comprises the following steps:

[0007] Step 1: discretize the path that the unmanned vehicle is currently traveling on to obtain discrete path points; obtain the discrete path points within a set distance in front of the unmanned vehicle according to a set period;

[0008] Step 2: determine the target point from the discrete path points by establishing a functional relationship between the foresight distance and the speed; obtain the attributes of the target point, the attributes of the target point include the direction angle of the target point and the position of the target point; the position of the target point includes the horizontal coordinate and the vertical coordinate of the target point;

[0009] Step 3: Calculate the difference between the heading angle of the unmanned vehicle and the direction angle of the target point, recorded as the control angle; calculate the difference between the lateral position of the unmanned vehicle and the lateral coordinate of the target point, recorded as the lateral deviation; based on the deep reinforcement learning method, design the action space and observation space, construct the reward function, train the neural network, make the heading angle and lateral position of the unmanned vehicle consistent with the direction angle and lateral coordinate of the target point respectively, search for the target point, and realize the training of the neural network;

[0010] Step 4: Use the trained neural network to make autonomous decisions: input the observation information of the unmanned vehicle into the trained neural network to obtain the optimal action corresponding to the optimal strategy; the unmanned vehicle acts according to the optimal action to achieve target point search; the observation information includes the size and direction of the current control angle of the unmanned vehicle, as well as the size of the lateral deviation.

[0011] Furthermore, in step 2, the obtaining of the attributes of the target point specifically includes: the unmanned vehicle obtains the direction angle and position of the target point based on the on-board radar sensor, camera and inertial navigation system.

[0012] Furthermore, in step 2, determining the target point from the discrete path points by establishing a functional relationship between the foresight distance and the speed specifically includes:

[0013] The functional relationship between the foresight distance and the speed is as follows:

[0014] dist=0.1×v

[0015] Among them, dist is the distance between the target point in front of the unmanned vehicle and the current position of the unmanned vehicle, and v is the current speed of the unmanned vehicle.

[0016] Furthermore, in step 3, the deep reinforcement learning method is used to design the action space and observation space, construct the reward function, and train the neural network so that the heading angle and lateral position of the unmanned vehicle are respectively consistent with the heading angle and lateral coordinate of the target point, and the target point is searched to realize the training of the neural network, which specifically includes:

[0017] During the neural network training process, the unmanned vehicle is used as the vehicle agent, the action space of the vehicle agent is the steering angle, and the observation space of the vehicle agent is the control angle and lateral deviation;

[0018] The reward function r during training is:

[0019] r = r1 + r2 + r3;

[0020] in, represents the angle error penalty term; represents the lateral error penalty term; Represents items used to improve training efficiency;

[0021] The neural network adopts TD3 deep neural network;

[0022] The training process specifically includes:

[0023] First, initialize the TD3 deep neural network;

[0024] Secondly, the driverless vehicle uses the TD3 deep neural network to decide the next action based on the status of the vehicle and surrounding vehicles, and obtains corresponding rewards and penalties; the current state, next state, actions taken, and rewards and penalties obtained by all vehicle agents are stored in the experience pool;

[0025] Finally, during the training process, samples are extracted from the experience pool based on the method of prioritized experience replay to train the evaluation network. The parameters of the evaluation network are updated by minimizing the loss function, and then the parameters of the corresponding policy network are updated according to the gradient descent method. The target policy network and the target evaluation network are updated using soft updates.

[0026] The loss function is:

[0027] Where, L i represents the loss function of the i-th evaluation network, N represents the number of samples extracted, Q represents the action value, θ i represents the parameters of the i-th evaluation network, s represents the state observation value, a represents the action, and y represents the predicted action value:

[0028]

[0029] In the formula, d represents the task completion flag, represents the action value calculated by the i-th target evaluation network, θ i ′ represents the parameters of the i-th target evaluation network, s′ represents the state observation value at the next moment, and a′ represents the action at the next moment.

[0030] Compared with the prior art, the beneficial technical effects of the present invention are:

[0031] The driving control method of the present invention is that in each cycle, the vehicle obtains discrete path points within a certain distance ahead, finds the target point ahead by establishing a functional relationship between the forward-looking distance and the speed, and uses GPS and other devices to obtain the properties of the target point, including the direction angle, lateral coordinates, and longitudinal coordinates. Based on the deep reinforcement learning method, the neural network is trained to adjust the vehicle heading angle so that the vehicle heading angle and lateral position are consistent with the direction angle and lateral position of the target point. The present invention realizes the lateral and longitudinal control of unmanned vehicles in different driving tasks, and effectively improves the trajectory tracking accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of a method for controlling a self-driving vehicle in accordance with an embodiment of the present invention;

[0033] Figure 2 It is a schematic diagram of driving seeking in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] A preferred embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference and illustration only and do not limit the protection scope of the present invention.

[0035] like Figure 1 As shown, this embodiment discloses a driving control method for an unmanned vehicle, including the following steps.

[0036] Step 1: Figure 2 As shown, the path of the vehicle is discretized, the interval between adjacent discrete path points is 0.1m, and the discrete path points within a certain distance ahead are obtained according to a set period. In this embodiment, the set period is 100ms.

[0037] The planned path is discretized into a series of points, and at each moment, a point at a certain distance ahead is taken as the target point, i.e., "the target point". Different driving tasks can be summarized as tracking the target point. The control process is the process of making the angle formed by the vehicle heading and the tangent direction of the trajectory at the target point zero.

[0038] Step 2: Find the target point ahead by establishing a functional relationship between the forward sight distance and the speed, and use GPS and other equipment to obtain the properties of the target point, including the direction angle, horizontal coordinates, and vertical coordinates.

[0039] The functional relationship between foresight distance and speed is as follows:

[0040] dist = 0.1 × v;

[0041] Among them, dist is the distance between the target point and the current position of the unmanned vehicle, and v is the current vehicle speed.

[0042] like Figure 2 As shown, the control angle e formed with the target point is obtained δ , and lateral deviation e y , used for training neural networks.

[0043] Step 3: Based on the deep reinforcement learning method, the action space and observation space are designed, the reward function is constructed, and the neural network is trained to make the heading angle and lateral position of the unmanned vehicle consistent with the heading angle and lateral coordinate of the target point respectively, so as to achieve the purpose of finding the target point, that is, to achieve the purpose of trajectory tracking.

[0044] During the neural network training process, the unmanned vehicle is regarded as a vehicle agent, the action space of the vehicle agent is the steering angle, and the observation space of the vehicle agent is the control angle and lateral deviation.

[0045] The reward function r during training is as follows:

[0046] r = r1 + r2 + r3;

[0047] in,

[0048]

[0049] In this embodiment, the neural network may adopt a TD3 deep neural network.

[0050] The training process specifically includes:

[0051] First, initialize the TD3 deep neural network.

[0052] Secondly, the driverless vehicle uses the TD3 deep neural network to decide the next action based on the status information of the vehicle and surrounding vehicles, and obtains corresponding rewards and penalties. The current state, next state, actions taken, and rewards and penalties obtained by all vehicle agents are stored in the experience pool.

[0053] Finally, during the training process, samples are extracted from the experience pool based on the method of prioritizing experience replay to train the evaluation network. The parameters of the evaluation network are updated by minimizing the loss function, and then the parameters of the corresponding policy network are updated according to the gradient descent method, and the target policy network and target evaluation network are updated using soft updates.

[0054] The method based on priority experience playback stores the current state, next state, actions taken and rewards and penalties obtained of all vehicle agents in the experience pool in the form of tuples. During training, more important experiences are preferentially extracted for training, thereby accelerating convergence.

[0055] The loss function is:

[0056]

[0057] Where, L i represents the loss function of the i-th evaluation network, N represents the number of samples extracted, Q represents the action value, θ i represents the parameters of the i-th evaluation network, s represents the state observation value, a represents the action, and y represents the predicted action value:

[0058]

[0059] In the formula, r represents the reward obtained, d represents the task completion mark, represents the action value calculated by the i-th target evaluation network, θ i ′ represents the parameters of the i-th target evaluation network, s′ represents the state observation value at the next moment, and a′ represents the action at the next moment.

[0060] Step 4: Use the trained neural network to make autonomous decisions. Input the observation information into the trained neural network to get the optimal action corresponding to the optimal strategy. The unmanned vehicle acts according to the action to achieve the target point.

[0061] Specifically, at each time step, the unmanned vehicle observes the size and direction of the current control angle, as well as the size of the lateral deviation. The strategy network outputs the corresponding optimal action, that is, the action with the largest action value, based on the observed information.

[0062] As shown in Table 1, the test results show that the unmanned vehicle seeking driving control method of the present invention can achieve accurate trajectory tracking in the following, double lane change, horizontal parking, 45-degree inclined parking, and vertical parking scenarios.

[0063] Table 1 shows the test results of the homing driving control technology in different driving tasks.

[0064]

[0065] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0066] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A driving control method for an unmanned vehicle, characterized in that: The following steps are involved: Step 1: discretize the path that the unmanned vehicle is currently traveling on to obtain discrete path points; obtain the discrete path points within a set distance in front of the unmanned vehicle according to a set period; Step 2: determine the target point from the discrete path points by establishing a functional relationship between the foresight distance and the speed; obtain the attributes of the target point, the attributes of the target point include the direction angle of the target point and the position of the target point; the position of the target point includes the horizontal coordinate and the vertical coordinate of the target point; Step 3: Calculate the difference between the heading angle of the unmanned vehicle and the direction angle of the target point, recorded as the control angle; calculate the difference between the lateral position of the unmanned vehicle and the lateral coordinate of the target point, recorded as the lateral deviation; based on the deep reinforcement learning method, design the action space and observation space, construct the reward function, train the neural network, make the heading angle and lateral position of the unmanned vehicle consistent with the direction angle and lateral coordinate of the target point respectively, search for the target point, and realize the training of the neural network; Step 4: Use the trained neural network to make autonomous decisions: input the observation information of the unmanned vehicle into the trained neural network to obtain the optimal action corresponding to the optimal strategy; the unmanned vehicle acts according to the optimal action to achieve target point search; the observation information includes the size and direction of the current control angle of the unmanned vehicle, as well as the size of the lateral deviation.

2. The method for controlling the homing of an unmanned vehicle according to claim 1, characterized in that: In step 2, the acquisition of the attributes of the target point specifically includes: the unmanned vehicle acquires the direction angle and position of the target point based on the on-board radar sensor, camera and inertial navigation system.

3. The method for controlling the homing of an unmanned vehicle according to claim 1, characterized in that: In step 2, the target point is determined from the discrete path points by establishing a functional relationship between the forward-looking distance and the speed, which specifically includes: The functional relationship between the foresight distance and the speed is as follows: dist=0.1×v Among them, dist is the distance between the target point in front of the unmanned vehicle and the current position of the unmanned vehicle, and v is the current speed of the unmanned vehicle.

4. The method for controlling the homing of an unmanned vehicle according to claim 1, characterized in that: In step 3, based on the deep reinforcement learning method, the action space and observation space are designed, the reward function is constructed, and the neural network is trained so that the heading angle and lateral position of the unmanned vehicle are respectively consistent with the heading angle and lateral coordinate of the target point, the target point is searched, and the training of the neural network is realized, which specifically includes: During the neural network training process, the unmanned vehicle is used as the vehicle agent, the action space of the vehicle agent is the steering angle, and the observation space of the vehicle agent is the control angle and lateral deviation; The reward function r during training is: r＝r1+r2+r3； in, represents the angle error penalty term; represents the lateral error penalty term; Represents items used to improve training efficiency; The neural network adopts TD3 deep neural network; The training process specifically includes: First, initialize the TD3 deep neural network; Secondly, the driverless vehicle uses the TD3 deep neural network to decide the next action based on the status of the vehicle and surrounding vehicles, and obtains corresponding rewards and penalties; the current state, next state, actions taken, and rewards and penalties obtained by all vehicle agents are stored in the experience pool; Finally, during the training process, samples are extracted from the experience pool based on the method of prioritized experience replay to train the evaluation network. The parameters of the evaluation network are updated by minimizing the loss function, and then the parameters of the corresponding policy network are updated according to the gradient descent method. The target policy network and the target evaluation network are updated using soft updates. The loss function is: Where, L i represents the loss function of the i-th evaluation network, N represents the number of samples extracted, Q represents the action value, θ i represents the parameters of the i-th evaluation network, s represents the state observation value, a represents the action, and y represents the predicted action value: In the formula, d represents the task completion flag, represents the action value calculated by the i-th target evaluation network, θ′ i represents the parameters of the i-th target evaluation network, s′ represents the state observation value at the next moment, and a′ represents the action at the next moment.

Images