Automatic driving vehicle track control method considering intersection conflict vehicle track distribution

Abstract

The invention relates to an automatic driving vehicle trajectory control method considering intersection conflict vehicle trajectory distribution, which comprises the following steps: solving a trajectory control model, obtaining the acceleration and steering wheel rotation angle of an automatic driving vehicle at each moment, expressing the trajectory control model by using a Markov decision process, and taking the maximum reward function as an optimization target; and the value range constraint conditions of the acceleration, the speed and the steering angle of the automatic driving vehicle are considered. The potential trajectory uncertainty of the manual driving vehicle is fully considered, and the trajectory of the automatic driving vehicle is optimized and controlled, so that the traffic safety is guaranteed; a long-short-term memory method is adopted to predict the track distribution range of the manual driving vehicle, and a control algorithm based on reinforcement learning is designed to obtain the acceleration and the steering wheel turning angle of the automatic driving vehicle. The track distribution of conflicting vehicles in the intersection is considered, and the safety of the automatic driving vehicle is greatly improved under the condition that the efficiency is slightly sacrificed.

Description

Technical Field

[0001] The present invention relates to a method for controlling the trajectory of an autonomous driving vehicle, and in particular to a method for controlling the trajectory of an autonomous driving vehicle taking into account the trajectory distribution of conflicting vehicles at an intersection. Background Art

[0002] Autonomous vehicles and manually driven vehicles will coexist for an extended period. The uncertainty of manually driven vehicles' trajectories poses a safety risk when autonomous vehicles navigate intersections. Fully accounting for the uncertainty of manually driven vehicles' trajectories is key to the safe operation of autonomous vehicles in mixed traffic flows.

[0003] Currently, the prediction of the future trajectories of conflicting vehicles primarily relies on a single maximum-probability trajectory. However, intersections often lack clear lane demarcations, leaving human-driven vehicles with an infinite number of possible trajectories. No autonomous vehicle trajectory control methods that consider the distribution of conflicting vehicle trajectories at intersections have been identified, nor have any patents for such methods been found.

[0004] After searching the literature on existing technologies, it was found that there are mainly the following methods for trajectory control of autonomous driving vehicles:

[0005] 1. Game theory-based approach. Specifically, this approach combines human driver risk perception with game theory to simulate the decision-making behavior of manually driven vehicles entering intersections, treating the autonomous vehicle and the manually driven vehicle as two game participants. This approach is applied in the invention patents "A method for planning human-vehicle interaction decisions for autonomous vehicles based on game theory" (application number CN202210486625.8) and "A method for establishing a lane-changing conflict coordination model for autonomous vehicles based on game theory" (application number CN201910603407.6).

[0006] 2. Artificial potential field-based method. Specifically, this method involves constructing virtual attractive and repulsive potential fields for trajectory planning. This method is applied in the invention patents "A Method and System for Autonomous Vehicle Trajectory Planning Based on Edge Computing" (application number CN202411060815.9) and "A Method for Autonomous Control of Free-Flow Vehicles Based on Artificial Potential Fields" (application number CN202210963692.4).

[0007] 3. Optimization-based methods. Specifically, the trajectory planning problem is converted into a mathematical optimization problem, so that the autonomous vehicle seeks the optimal trajectory that minimizes (or maximizes) a specific cost function while satisfying constraints such as dynamics, road conditions, and traffic rules. This method is applied in the invention patent "Learning-based predictive control method for unmanned vehicles based on long-short-term memory networks" (application number CN202410055698.0) and the invention patent "A lateral control method for autonomous vehicles based on DRL-MPC" (application number CN202311517431.0).

[0008] Existing autonomous vehicle trajectory control methods do not consider the future trajectory distribution of conflicting vehicles. Method 1 can only provide fuzzy decision-making and can only control the magnitude of acceleration, not the steering wheel angle. Method 2 is computationally simple and responsive, and can achieve trajectory planning by constructing a virtual potential field that attracts autonomous vehicles and repels obstacles. However, it is prone to falling into local minima, and in complex intersections and when the scope of vehicle conflicts changes, interference between various potential fields may cause unstable trajectory planning results, resulting in limited effectiveness in practical applications. Method 3 can integrate multiple performance indicators (such as safety, traffic efficiency, and comfort), but when solving a problem with many variables, local optimal solutions will exist, and the computational complexity will increase exponentially, resulting in poor adaptability of optimization-based methods in real-time processing of dynamic environments. Summary of the Invention

[0009] Purpose of the invention: In response to the shortcomings of the prior art, the purpose of the present invention is to provide a method for controlling the trajectory of an autonomous driving vehicle that takes into account the trajectory distribution of conflicting vehicles at intersections. This method optimizes the trajectory of the autonomous driving vehicle by considering the potential trajectory uncertainty of manually driven vehicles to ensure traffic safety. The optimization control method uses the long-short-term memory method to predict the trajectory distribution range of manually driven vehicles, and then designs a control algorithm based on reinforcement learning to obtain the acceleration and steering wheel angle of the autonomous driving vehicle, thereby greatly improving the safety of the autonomous driving vehicle.

[0010] Technical Solution: The autonomous vehicle trajectory control method, which considers the trajectory distribution of vehicles in intersection conflicts, obtains the acceleration and steering angle of the autonomous vehicle at each moment by solving the following trajectory control model. The trajectory control model is expressed as a Markov decision process, with the maximum reward function as the optimization objective, and considers the range constraints of the autonomous vehicle's acceleration, speed, and steering angle, where:

[0011] The Markov decision process includes: a state space, an action space, a state transition matrix, a reward function and a discount factor;

[0012] The state space parameters include: x, the horizontal coordinate of the vehicle, m; y, the vertical coordinate of the vehicle, m; θ, the heading angle of the vehicle, rad; v, the speed of the vehicle, m / s;

[0013] The state space parameters are shown as follows Figure 1 As shown;

[0014] The action space parameters include: α, the acceleration of the autonomous driving vehicle, m / s 2 ;κ, vehicle steering wheel angle, rad;

[0015] The state transfer matrix is:

[0016]

[0017] Where: represents the speed of the autonomous vehicle at time step t; represents the heading angle of the autonomous vehicle at time step t; and They represent the coordinate positions of the autonomous vehicle at time step t respectively;

[0018] The reward function is related to the driving efficiency, ride comfort and safety of the autonomous vehicle, as shown in formula (5);

[0019] R t =R done +R step +R safe +R com (5)

[0020] R done =w (6)

[0021] R step =-1 (7)

[0022]

[0023] Rco m =-β2L1-β3L2 (10)

[0024]

[0025] Where: R t is the reward at time step t; R done The reward w for successfully passing the intersection is a maximum value, which is set to 5000 here; R step is the running time reward of the autonomous vehicle; R safe Safety reward for autonomous vehicles; R col collision rewards for autonomous vehicles; Indicates the arrival position of the manually driven vehicle i The probability density of can be calculated by formula (13); R com represents the comfort reward of the autonomous vehicle; β1, β2, and β3 are weights; d represents the diagonal length of the vehicle; L1 and L2 represent the comfort of the autonomous vehicle during lateral and longitudinal motion, respectively;

[0026] The long short-term memory method is used to predict the trajectory distribution range of the manually driven vehicle, and the two-dimensional Gaussian distribution is selected as the future trajectory range distribution model of the manually driven vehicle. The long short-term memory method is used to predict the trajectory distribution of the manually driven vehicle, and the parameters of the two-dimensional Gaussian distribution are output as the mean μ, standard deviation σ, and correlation coefficient ρ. i ,y i ) Distribution probability density f i (x i ,y i ) as described by the following formula:

[0027]

[0028] Where: and are the means of the distributions, respectively, indicating x i and y i The average value of and are the standard deviations of the distribution, indicating x i and y i The degree of dispersion; ρ is the correlation coefficient, indicating x i and y i The degree of linear correlation between

[0029] The discount factor γ∈[0,1] determines the degree of influence of current rewards or future rewards on the autonomous vehicle. The maximum reward function is shown in formula (20);

[0030]

[0031] The constraint condition requires that the vehicle's motion state cannot break through the constraints of real physical conditions, that is, it satisfies the requirements of formula (21) to formula (23);

[0032] α min ≤α≤α max (twenty one)

[0033] -θ max ≤κ≤θ max (twenty two)

[0034] v min ≤v≤v max (twenty three)

[0035] Where: α min and α max The minimum acceleration and maximum acceleration of the autonomous vehicle respectively; θ max represents the maximum steering angle of the autonomous vehicle; v min and v max Represent the lower and upper speed limits of the autonomous vehicle respectively;

[0036] The following considerations should be taken into account during the trajectory control process: (1) From the perspective of traffic efficiency, the faster the autonomous vehicle is, the better; (2) From the perspective of safety, the farther the autonomous vehicle is from high-risk areas, the better, so as to avoid collisions to the greatest extent; (3) From the perspective of comfort, the closer the acceleration and steering wheel angle of the autonomous vehicle are to zero, the better, so as to maximize the comfort experience; this trajectory control method takes these three contradictions into consideration and establishes them in the reward function, so as to fully utilize traffic efficiency and comfort while ensuring traffic safety.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention provides a method for controlling the trajectory of an autonomous driving vehicle that takes into account the trajectory distribution of conflicting vehicles at an intersection. The method of the present invention takes into account the trajectory distribution of manually driven vehicles, thereby ensuring safety, and on this basis, maximizing traffic efficiency and comfort. For the interaction scenario between an autonomous driving vehicle and a manually driven vehicle inside an intersection, the potential trajectory uncertainty of the manually driven vehicle is fully considered, and the trajectory of the autonomous driving vehicle is optimized and controlled to ensure traffic safety; the optimization control method uses the long-short-term memory method to predict the trajectory distribution range of the manually driven vehicle, and then designs a control algorithm based on reinforcement learning to obtain the acceleration and steering wheel angle of the autonomous driving vehicle. This method takes into account the trajectory distribution of conflicting vehicles inside the intersection, and can significantly improve the safety of autonomous driving vehicles at the expense of slightly sacrificing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of state space parameters in the present invention;

[0040] Figure 2 The difference between the rewards of consecutive rounds during the straight-line driving process of the autonomous driving vehicle in Example 1 of the present invention;

[0041] Figure 3 The difference between the rewards in adjacent rounds during the left turn process of the autonomous driving vehicle in Example 1 of the present invention;

[0042] Figure 4 This is a schematic diagram of the trajectory of an autonomous driving vehicle turning left through an intersection in Example 1 of the present invention;

[0043] Figure 5This is a schematic diagram of the trajectory of an autonomous driving vehicle traveling straight through an intersection in Example 1 of the present invention;

[0044] Figure 6 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0045] The following is combined with Figure 2 To the attached Figure 5 The present invention is further illustrated by the following embodiments.

[0046] This example is a Shanghai intersection. The trajectory data of manually driven vehicles is the real trajectory data extracted by drones. A total of 55 manually driven vehicle trajectories are selected, including 42 straight-line and 13 left-turn trajectories. The simulation parameters are: time step, Δt, is 0.1s; initial heading angle, Take 0rad; initial steering angle, κ 0 , take 0rad; initial velocity, Take 6m / s; maximum steering angle, θ max , take π / 5rad; minimum acceleration, α min , take -5m / s 2 ; Maximum acceleration, α max , take 5m / s 2 ; Minimum speed, v min , take 0m / s; maximum speed, v max , take 15m / s; weight parameters, β1, β2 and β3, take 0.10, 0.01 and 0.01.

[0047] The specific process is briefly described as follows:

[0048] Step 1: Substitute the above parameters into the trajectory control model established by the present invention.

[0049] Step 2: The above model can be solved using the Q-learning algorithm in reinforcement learning. Based on the progress of the autonomous vehicle through the intersection, the learning rate and discount factor are dynamically adjusted, as shown in Equations (24) and (25). The iteration stops when the difference in total rewards between adjacent rounds is less than 0.001 or reaches a maximum of 2000 rounds.

[0050] l=l0(1-t / 20) (24)

[0051] γ=γ0(1-t / 20) (25)

[0052] Where: l0 and γ0 represent the initial learning rate and the initial discount factor respectively.

[0053] like Figure 2 and Figure 3As shown in the figure, according to the method of the present invention, the iterative process of the autonomous driving vehicle in going straight and turning left shows obvious convergence after nearly 400 rounds and 1050 rounds respectively, which shows that the algorithm proposed in this article can fully cope with this example scenario.

[0054] like Figure 4 As shown in , when the autonomous vehicle turns left and the human-driven vehicle goes straight, the autonomous vehicle chooses to avoid the human-driven vehicle, prioritizing safety and thus obtaining a higher reward. Figure 5 As shown in the figure, when the autonomous vehicle is going straight and the manually driven vehicle is turning left, since the conflict point is closer to the manually driven vehicle, the autonomous vehicle chooses to avoid the manually driven vehicle, thus achieving a more comfortable avoidance strategy.

[0055] As shown in Table 1, the proposed autonomous vehicle control method significantly outperforms game theory-based, artificial potential field-based, and optimization-based methods in terms of the percentage of post-intrusion times below a threshold (1.5 seconds), both for left turns and going straight. This significantly improves the safety of autonomous vehicles at intersections. The proposed method achieved a post-intrusion time below the threshold of only 0.33% for left turns, a 50% and 80% improvement over the game theory-based and optimization-based methods, respectively. For straight-ahead driving, the percentage was 0.67%, a 33% improvement over the game theory-based, artificial potential field-based, and optimization-based methods, respectively, and a 71% improvement over the game theory-based, artificial potential field-based, and optimization-based methods, respectively.

[0056] As shown in Table 2, while ensuring safety, the method of the present invention achieved an average runtime of 8.84 seconds for both left turns and straight driving of autonomous vehicles. This represents an improvement of 8.58% and 9.89% over the game theory-based and artificial potential field-based methods, respectively. The method of the present invention achieved an average runtime of 8.50 seconds for straight driving, representing improvements of 5.45% and 5.24% over the game theory-based and artificial potential field-based methods, respectively. Due to the significant uncertainty associated with manually driven vehicles passing through intersections, the average runtime of the method of the present invention is slightly inferior to that of the optimization-based method.

[0057] Table 1

[0058]

[0059] Table 2

[0060]

[0061] In summary, the present invention can adapt to various complex conflict scenarios and significantly improve the safety of autonomous driving vehicles at the expense of efficiency.

[0062] The above description of the embodiment is intended to facilitate understanding and application of the present invention by those skilled in the art. It will be apparent that those skilled in the art can readily make various modifications to this embodiment and apply the general principles described herein to other embodiments without requiring inventive effort. Therefore, the present invention is not limited to the embodiments described herein. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A method for controlling the trajectory of an autonomous driving vehicle considering the trajectory distribution of vehicles conflicting at an intersection, characterized in that: By solving the trajectory control model, the acceleration and steering wheel angle of the autonomous vehicle at each moment are obtained. The trajectory control model is expressed as a Markov decision process, with the maximum reward function as the optimization goal, and considering the range constraints of the acceleration, speed and steering angle of the autonomous vehicle. The Markov decision process includes: state space, action space, state transition matrix, reward function and discount factor, and takes into account the future trajectory range distribution of the manually driven vehicle.

2. The automatic driving vehicle trajectory control method considering intersection conflict vehicle trajectory distribution according to claim 1 is characterized in that: The state space parameters include: x, the horizontal coordinate of the vehicle, m; y, the vertical coordinate of the vehicle, m; θ, the heading angle of the vehicle, rad; v, the speed of the vehicle, m / s; The action space parameters include: α, the acceleration of the autonomous driving vehicle, m / s 2 ;κ, vehicle steering wheel angle, rad; The state transfer matrix is: Where: represents the speed of the autonomous vehicle at time step t; represents the heading angle of the autonomous vehicle at time step t; and They represent the coordinate positions of the autonomous vehicle at time step t respectively; The reward function is related to the driving efficiency, ride comfort and safety of the autonomous vehicle, as shown in formula (5); R t =R done +R step +R safe +R com (5) R done =w (6) R step ＝-1 (7) R com =-β2L1-β3L2 (10) Where: R t is the reward at time step t; R done is the reward for successfully passing the intersection; w is a maximum value, which is set to 5000 here; R step is the running time reward of the autonomous vehicle; R safe Safety reward for autonomous vehicles; R col collision rewards for autonomous vehicles; Indicates the arrival position of the manually driven vehicle i The probability density of can be calculated by formula (13); R com represents the comfort reward of the autonomous vehicle; β1, β2 and β3 are weights; d represents the diagonal length of the vehicle; L1 and L2 represent the comfort of the autonomous vehicle during lateral and longitudinal motion, respectively. and are the standard deviations of the distribution, indicating x i and y i The degree of dispersion; ρ is the correlation coefficient, indicating x i and y i The degree of linear correlation between The discount factor γ∈[0,1] determines the degree of influence of current rewards or future rewards on the autonomous vehicle. The maximum reward function is shown in formula (20); The constraint condition requires that the vehicle's motion state cannot break through the constraints of real physical conditions, that is, it satisfies the requirements of formula (21) to formula (23); α min ≤α≤α max (21) -θ max ≤κ≤θ max (22) in min ≤v≤v max (23) Where: α min and α max The minimum acceleration and maximum acceleration of the autonomous vehicle respectively; θ max represents the maximum steering angle of the autonomous vehicle; v min and v max They represent the lower and upper speed limits of the autonomous vehicle respectively.

3. The automatic driving vehicle trajectory control method considering intersection conflict vehicle trajectory distribution according to claim 1 is characterized in that: The long short-term memory method is used to predict the trajectory distribution range of the manually driven vehicle, and the two-dimensional Gaussian distribution is selected as the future trajectory range distribution model of the manually driven vehicle. The long short-term memory method is used to predict the trajectory distribution of the manually driven vehicle, and the parameters of the two-dimensional Gaussian distribution are output as the mean μ, standard deviation σ, and correlation coefficient ρ. i ,y i ) Distribution probability density f i (x i ,y i ) as described by the following formula: Where: and are the means of the distributions, respectively, indicating x i and y i The average value of and are the standard deviations of the distribution, indicating x i and y i The degree of dispersion; ρ is the correlation coefficient, which means x i and y i The degree of linear correlation between The discount factor γ∈[0,1] determines the degree of influence of current rewards or future rewards on the autonomous vehicle. The maximum reward function is shown in formula (20); 4. The automatic driving vehicle trajectory control method considering intersection conflict vehicle trajectory distribution according to claim 1 is characterized in that: The Q-learning algorithm in reinforcement learning is used to solve the problem. Based on the progress of the autonomous vehicle passing through the intersection, the learning rate and discount factor are dynamically adjusted, as shown in Equations (24) and (25). When the difference in total rewards between adjacent rounds is less than 0.001 or reaches a maximum of 2000 rounds, the iteration stops. l＝l0(1-t / 20)(24) γ=γ0(1-t / 20) (25) Where: l0 and γ0 represent the initial learning rate and the initial discount factor respectively.

5. The automatic driving vehicle trajectory control method considering intersection conflict vehicle trajectory distribution according to claim 1 is characterized in that: When optimizing the trajectory of an autonomous vehicle, the trajectory control model comprehensively considers the contradictions among traffic efficiency, safety, and comfort, and establishes them in the reward function.

Images