An interactive trajectory planning method based on a stackelberg game model

Abstract

The application relates to an interactive trajectory planning method based on a Stackelberg game model, which comprises the following steps: step S1, determining a loose-tight coupling relationship between an unmanned vehicle and a traffic participant; step S2, sequentially performing dynamic game on a dynamic obstacle tightly coupled with the unmanned vehicle; step S3, for each game, establishing a Stackelberg game model; and step S4, solving the Stackelberg game model to complete interactive trajectory planning. Compared with the prior art, the application has the advantages of high safety, high traffic efficiency and strong universality.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving trajectory planning technology, and in particular to an interactive trajectory planning method based on the Stackelberg game model. Background Technology

[0002] The trajectory planning module for autonomous driving plays a crucial role in the feasibility, comfort, and efficiency of vehicle operation. Its task is to avoid obstacles and reach the destination in space and time based on the vehicle's current position, surrounding environment, and behavioral decision-making information. With the development of environmental perception and decision-making / planning control modules in the field of autonomous driving, major automakers and universities are deploying autonomous vehicles for testing on urban roads. For a considerable period in the future, autonomous vehicles and human-driven vehicles will share roads. Therefore, how autonomous vehicles can interact with human-driven vehicles in a human-like manner to improve overall traffic efficiency is increasingly becoming a key and challenging issue in the field of autonomous driving trajectory planning.

[0003] Trajectory planning methods include artificial potential field method, sampling method, search method and optimization method.

[0004] A search revealed that Chinese patent application CN114995372A discloses an artificial potential field method. This method uses repulsive and attractive forces to describe the environment and can generate continuous collision-free trajectories. However, the potential field function may fall into local dead zones, leading to planning failure.

[0005] Based on random sampling, the improved fast search random tree (RRT) method proposed in Chinese patent application CN115268456A and the random path map (PRM) method used in Chinese patent application CN113778090A have the defects of poor stability and difficulty in guaranteeing the continuity of the trajectory.

[0006] Deterministic rule-based sampling methods include control state sampling and state-space sampling, which use control state variables and road geometry information as the basis for planning, respectively. Although sampling-based methods perform well in solving structured roads, they may fail to yield solutions when obstacles disrupt the road structure.

[0007] The improved A* algorithm proposed in Chinese patent application CN115268441A can generate smooth trajectories in unstructured roads. However, search-based methods cannot achieve smoothness at connection points because they rely on grid-based point finding. Therefore, many optimization-based algorithms further optimize the curves obtained from the search to satisfy vehicle non-integrity constraints. However, optimization methods suffer from the problem of difficulty in normalizing optimization indices and coefficients in different scenarios.

[0008] Existing technologies struggle to meet the increasingly demanding interactive requirements of autonomous driving's trajectory planning modules. In recent years, game theory has been widely used to solve interactive behaviors in autonomous driving, particularly in scenarios like unprotected intersections and forced lane changes. Chinese patent applications CN115230706A and CN114120664A respectively applied game theory to multi-vehicle lane change decision control and mixed-traffic intersection decision control, achieving cooperative Nash equilibrium solutions with the objective of minimizing the overall cost function of all vehicles. However, for a considerable period in the future, autonomous vehicles will coexist with human-driven vehicles on roads, often requiring the resolution of a non-cooperative game theory problem.

[0009] Currently, decision-making and planning modules are often separated, while game theory is often used in decision-making modules. The reward function in game modeling is based on the results of perception and scenario understanding. This may lead to the result given by the decision not being achievable by the plan, or the result given by the decision not being optimal for the plan. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an interactive trajectory planning method based on the Stackelberg game model that is highly secure, efficient, and universally applicable.

[0011] The objective of this invention can be achieved through the following technical solutions:

[0012] This invention presents an interactive trajectory planning method based on the Stackelberg game model, which includes the following steps:

[0013] Step S1: Determine the loose or tight coupling relationship between the autonomous vehicle and traffic participants;

[0014] Step S2: Perform dynamic game-playing sequentially on the dynamic obstacles tightly coupled with the unmanned vehicle;

[0015] Step S3: For each game, establish a Stackelberg game model;

[0016] Step S4: Solve the Stackelberg game model to complete the interactive trajectory planning.

[0017] Preferably, step S1 includes the following sub-steps:

[0018] Step S1-1: Obtain and integrate the predicted path results of traffic participants with high confidence from upstream tasks;

[0019] Step S1-2: Based on the location of static obstacles in the environment, perform path planning using a hybrid path planner that can switch autonomously;

[0020] Steps S1-3: Perform collision detection on the predicted path and the planned path. Irrelevant dynamic obstacles are classified as loosely coupled, and relevant dynamic obstacles are classified as tightly coupled traffic participants.

[0021] Preferably, the autonomously switchable hybrid path planner in steps S1-2 specifically refers to:

[0022] For structured roads, a planner based on road shape sampling is used. For unstructured roads or when the sampling-based planner has no solution, it automatically switches to a search-based planner. After finding a rough trajectory, it passes through the sampling-based planner and the optimization-based planner in sequence to finally obtain the optimal path. The optimization objective includes a path similarity index and a pre-aiming point safety index.

[0023] Preferably, step S2 includes the following sub-steps:

[0024] Step S2-1: Obtain the area where the autonomous vehicle collides with tightly coupled traffic participants using geometric methods;

[0025] Step S2-2: Determine the order of dynamic game based on the collision areas in the autonomous vehicle's planned path from near to far.

[0026] Preferably, the Stackelberg game model in step S3 includes modeling the strategy choices of the autonomous vehicle and other traffic participants, as well as modeling the determination of the leader and followers.

[0027] Preferably, the modeling of the strategy selection of the autonomous vehicle and other traffic participants specifically involves:

[0028] Based on the issue of who gets priority passage through the conflict zone to become a road, both the autonomous vehicle itself and other traffic participants have only two decisions: priority passage or yielding. The yielding vehicle needs to choose the optimal yielding trajectory based on the trajectory of the passing vehicle.

[0029] Preferably, the modeling for determining leaders and followers specifically includes:

[0030] 1) Analyze the traffic rules that the traffic participants in the game need to meet, their categories, road conditions, and the path curvature for planning and prediction;

[0031] 2) Design a quantitative index, described as the loss function for sequential passage, with the following expression:

[0032]

[0033] In the formula, K type K ob K c K tThe coefficients corresponding to obstacle type, obstacle road distribution, path curvature, and priority loss caused by the time it takes for game participants to reach the conflict area are used to determine the leader, with the one with the smallest loss being selected.

[0034] Preferably, step S4 includes the following sub-steps:

[0035] Step S4-1: Design the reward function for the Stackelberg game model;

[0036] Step S4-2: Solve for the optimal game strategy using backward induction;

[0037] Step S4-3: The speed curve planning result of the unmanned vehicle corresponding to the optimal game strategy is used as the final model output for unmanned driving control.

[0038] Preferably, the reward function in step S4-1 is obtained through the results of exploratory velocity planning under the corresponding strategy. The reward function includes the player's own reward and the opponent's reward, and its expression is:

[0039]

[0040] In the formula, The guilt coefficient is used to define the opponent's reward in a game. For the opponent P n The reward corresponding to strategy j; For players P m The reward for adopting strategy i is determined by a weighted average of the average velocity, average acceleration, and average jerk throughout the entire process of traversing the conflict zone, expressed as:

[0041]

[0042] In the formula, These represent average velocity, average acceleration, and average jerk, respectively, with w1, w2, and w3 being the weighting coefficients for the corresponding terms; v ref For the desired speed on the road, Acc max J is the maximum acceleration of the vehicle. max The maximum acceptable jerk (impact rate).

[0043] Preferably, step S4-3 specifically involves: obtaining a coarse velocity curve using a search method based on the ST graph, then transforming the problem into a quadratic programming problem, and finally using a QP solver to solve for the optimal velocity curve.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] 1) In path planning, the advantages of sampling and search optimization methods are combined, and their disadvantages are avoided in different scenarios by automatically switching, making the path planning results safer and more reliable.

[0046] 2) Combining the results of path planning and prediction to identify traffic participants closely coupled with autonomous vehicles in the environment greatly improves the efficiency of subsequent modeling and solving.

[0047] 3) In speed planning, a Stackelberg game model is built, a quantitative model of the leader-follower model is proposed, and the decision-making and planning modules are organically integrated. The results of the exploratory speed planning help the autonomous vehicle make the optimal game strategy, taking into account the interaction with the surrounding tightly coupled dynamic obstacles.

[0048] 4) The present invention can interact with the environment based on the results of scene understanding, and improves traffic efficiency by about 9.5% when turning left through a scenario without traffic lights. Attached Figure Description

[0049] Figure 1 This is a flowchart of the method of the present invention;

[0050] Figure 2 This is a diagram of the interactive trajectory planning algorithm architecture based on the Stackelberg game model of the present invention.

[0051] Figure 3 This is a scenario of vehicles meeting in the embodiment;

[0052] Figure 4 The game tree for the Stackelberg game model;

[0053] Figure 5 This is a graph showing the speed planning results under all game strategies in the two games in the scenario of vehicles meeting;

[0054] Figure 6 The simulation results are for a scenario where two vehicles meet; among them, Figure 6 (a) to (c) are the line breaks corresponding to 2.4s, 12.6s, and 17.9s, respectively. Figure 6 (d) shows the simulation results of velocity planning;

[0055] Figure 7 The simulation results are for a left-turn scenario without traffic lights; among them, Figure 7 (a) to 7(c) show the oncoming traffic results at 0s, 1.6s, and 4.3s respectively in the scenario of turning left without traffic lights. Figure 7 (d) shows the simulation results of the vehicle speed planning;

[0056] Figure 8 The image shows the simulation results of a game-theoretic approach driven solely by perception in a scenario where left turns are not possible via traffic lights; among them, Figure 8 (a) to (c) show the passing results at 0s, 1.6s, and 4.3s respectively, using a perception-driven game theory approach in a left-turn scenario without traffic lights. Figure 8 (d) shows the simulation results of the vehicle speed planning. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0058] Example 1

[0059] This embodiment presents an interactive trajectory planning method based on the Stackelberg game model, which includes the following steps:

[0060] Step S1: Determine the loose or tight coupling relationship between the autonomous vehicle and traffic participants, specifically including the following sub-steps:

[0061] Step S2: Perform a dynamic game on the dynamic obstacles tightly coupled with the autonomous vehicle, specifically including:

[0062] Step S2-1: Obtain the area where the autonomous vehicle collides with tightly coupled traffic participants using geometric methods;

[0063] Step S2-2: Determine the order of dynamic game based on the collision areas in the autonomous vehicle's planned path, from near to far.

[0064] Step S3: For each game, establish a Stackelberg game model, including modeling the strategy choices of the autonomous vehicle and other traffic participants, as well as modeling the determination of the leader and followers.

[0065] Specifically, the modeling of the strategy choices of autonomous vehicles and other traffic participants is as follows:

[0066] Based on the issue of who gets priority passage through the conflict zone to become a road, both the autonomous vehicle itself and other traffic participants have only two decisions: to pass first or to give way. The vehicle giving way needs to choose the optimal avoidance trajectory based on the trajectory of the passing vehicle.

[0067] The modeling for identifying leaders and followers specifically includes:

[0068] 1) Analyze the traffic rules that the traffic participants in the game need to meet, their categories, road conditions, and the path curvature for planning and prediction;

[0069] 2) Design a quantitative index, described as the loss function for sequential passage, with the following expression:

[0070]

[0071] In the formula, K type K ob K c K t The coefficients corresponding to obstacle type, obstacle road distribution, path curvature, and priority loss caused by the time it takes for game participants to reach the conflict area are used to determine the leader, with the one with the smallest loss being selected.

[0072] Step S4: Solve the Stackelberg game model and complete the interactive trajectory planning, which includes the following sub-steps:

[0073] Step S4-1: Design the reward function for the Stackelberg game model. The reward function is obtained through the trial-and-error velocity planning results under the corresponding strategy. The reward function includes the player's own reward and the opponent's reward, expressed as:

[0074]

[0075] In the formula, The guilt coefficient is used to define the opponent's reward in a game. For the opponent P n The reward corresponding to strategy j; For players P m The reward for adopting strategy i is determined by a weighted average of the average velocity, average acceleration, and average jerk throughout the entire process of traversing the conflict zone, expressed as:

[0076]

[0077] In the formula, These represent average velocity, average acceleration, and average jerk, respectively, with w1, w2, and w3 being the weighting coefficients for the corresponding terms; v ref For the desired speed on the road, Acc max J is the maximum acceleration of the vehicle. max The maximum acceptable jerk.

[0078] Step S4-2: Solve for the optimal game strategy using backward induction;

[0079] Step S4-3: The speed curve planning result of the unmanned vehicle corresponding to the optimal game strategy is used as the final model output for unmanned driving control.

[0080] Example 2

[0081] This embodiment provides an interactive trajectory planning method based on a Stackelberg game model. The method includes: establishing the loose / tight coupling relationship between environmental traffic participants and the autonomous vehicle (RV) based on dynamic obstacle path prediction results obtained by the perception module and the results of an automatically switching hybrid path planner; sequentially conducting dynamic games with tightly coupled dynamic obstacles to the RV, with the game order proceeding from near to far according to predefined conflict zones; establishing a Stackelberg game model for each game and establishing the RV's game position through a quantified leader-follower indicator function; and using the tentative speed planning results under different combinations as the reward function of the game model to allow the RV to make optimal decisions considering interactions with other traffic participants in a complex and highly dynamic environment. In other words, this method uses game theory to model the environment, considering the interaction behavior between the RV and the environment, and planning a trajectory that is most efficient for the RV in terms of overall traffic efficiency.

[0082] In actual operation, see Figure 2 The overall architecture consists of an upper-level path planner and a lower-level speed planner. The hybrid path planner is implemented through the following steps:

[0083] 1) In structured roads, a sampling and optimization-based planner is used. The sampling method involves scattering points along the road laterally. The hard constraints of the optimization method are static obstacle constraints, traffic rule constraints, and vehicle minimum turning radius limits. The optimization objective is a weighted sum of the lateral deviation of the autonomous vehicle from the desired path, curvature, obstacle repulsion considering "grayscale", similarity between adjacent frame paths, and safety of the pre-aimed path.

[0084] 2) On unstructured roads, a pathfinder based on the hybrid A* algorithm is used first, and the search results are taken as the desired path. A smooth path curve is then generated through sampling and optimization methods. The hybrid A* algorithm satisfies vehicle kinematic constraints, and its node search method satisfies the following formula: Where ψ is the heading angle, β is the sideslip angle (considered to be 0), and δ f For the front wheel steering angle, l f For the front overhang length, l r The rear overhang is 0.7 rad, and the maximum steering angle of the vehicle's front wheels is 0.7 rad.

[0085] See Figure 2 The core of this invention—a velocity planner based on the Stackelberg game model—is accomplished through the following steps:

[0086] Step 1: Based on the planning results of the autonomous vehicle's upper-level path planner and the dynamic obstacle prediction results given by the prediction module, determine the tightness of the coupling between traffic participants, obtain the conflict area between the autonomous vehicle and its tightly coupled traffic participants, simulate the way humans solve the game from near to far, establish multiple two-person Stackelberg game models, and change the models in real time as the environment changes.

[0087] Step 2: Model the leader-follower relationship in the Stackelberg game model. The quantitative indicators are determined by the obstacle type, the obstacle distribution on the road, the path curvature, and the time taken to reach the conflict zone. These correspond to traffic rules such as yielding to pedestrians, priority for vehicles without obstacles, and yielding to oncoming traffic when turning, as well as the optimization objective of improving road efficiency. The loss function for the quantitative indicators is shown below: Among them, K type =0.3, K ob =0.2, K c =0.1, K t =0.4, Among them, L type For type loss, L ob Losses due to lane obstruction obstacles. Obstacles occupying lane width, L is the total width of the lane. c For path curvature loss, and L represents the average curvature of the self-planned path and the predicted path of other vehicles, respectively. t Due to the time loss in reaching the collision zone, and This represents the path length between the vehicle and the other vehicle from the collision zone. and These represent the speeds of the vehicle itself and the other vehicle.

[0088] Step 3: After obtaining different combinations of game trees, speed planning will be performed based on the ST graph search method under the corresponding game strategy. It is important to note that in the ST graph, only obstacles that the autonomous vehicle avoids according to the game strategy are projected. When there are multiple games, iterative projection from near to far is required. For example... Figure 3 As shown, this vehicle-to-vehicle scenario involves two games. In the first game with the other vehicle, the quantification formula in step two shows that the autonomous vehicle is in a leader position. In the second game with the human, the autonomous vehicle is in a follower position. When the autonomous vehicle adopts an avoidance strategy in both the first and second games, its speed planning result is as follows. Figure 2 As shown. The velocity planning curves for other strategies in this scenario are as follows. Figure 5 As shown.

[0089] Step 4, for example Figure 4 The game tree extension of the Stackelberg game model shown is used to solve for the optimal strategy in this game through backward induction. First, consider nodes N2 and N3. Empirically, we know that the payoff function when both players choose to proceed (collision) or both choose to avoid (traffic congestion) is much smaller than the payoff when one player chooses to proceed and the other chooses to avoid. That is... Therefore, node N2 will choose strategy o 12 Node N3 will select strategy o 21 Looking at node N1 again, through backward induction, we can deduce that the superiority or inferiority of a game strategy is equivalent to the magnitude of the reward for the leader when choosing that strategy; that is, a comparison... and when In this context, leaders choosing to prioritize passage is superior to choosing an avoidance strategy in terms of overall traffic efficiency.

[0090] Step five, the constructed reward function of the Stackelberg game consists of two parts: This refers to both one's own reward and the opponent's reward, where the opponent's reward is defined by the guilt coefficient. Let be the guilt coefficient, and i, j correspond to P respectively. m and P n The strategy adopted by player P m The reward is determined by the speed planning results throughout the process of traversing the conflict zone, as follows: Where w1 = 0.5, w2 = 0.3, w3 = 0.2, v ref For the desired speed on the road, Acc max J is the maximum acceleration of the vehicle. max This represents the acceptable maximum jerk (impact rate). The search velocity method in the ST diagram satisfies the operational equations. In this process, the node search takes into account the vehicle's acceleration limit, and takes a = [-3, -2, -1, 0, 1, 2] as the candidate acceleration input.

[0091] Table 1 below shows the quantitative indicators of the comparative experiment.

[0092] Table 1

[0093]

[0094]

[0095] Figure 6 The simulation results are for a scenario where two vehicles meet; among them, Figure 6 (a) to (c) are the line breaks corresponding to 2.4s, 12.6s, and 17.9s, respectively. Figure 6 (d) shows the simulation results of velocity planning; Figure 7 The simulation results are for a left-turn scenario without traffic lights; among them, Figure 7 (a) to 7(c) show the oncoming traffic results at 0s, 1.6s, and 4.3s respectively in the scenario of turning left without traffic lights. Figure 7 (d) shows the simulation results of the vehicle speed planning.

[0096] In summary, the test results in this scenario show that, compared to traditional conservative solutions that rely solely on environmental perception for decision-making, the method of selecting game strategies through the reward function provided by the trial-and-error planning in this invention can more closely link the decision-making and planning modules and obtain a better trajectory.

[0097] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An interactive trajectory planning method based on the Stackelberg game model, characterized in that, The method includes the following steps: Step S1: Determine the loose or tight coupling relationship between the autonomous vehicle and traffic participants, including the following sub-steps: Step S1-1: Obtain and integrate the predicted path results of traffic participants with high confidence from upstream tasks; Step S1-2: Based on the location of static obstacles in the environment, a hybrid path planner that can switch autonomously is used for path planning. Specifically, a planner based on road shape sampling is used on structured roads. When the planner based on sampling method has no solution on unstructured roads or when there is no solution on unstructured roads, the planner is automatically switched to a search-based planner. After a rough trajectory is found, the planner is then passed through the sampling-based planner and the optimization-based planner in sequence to finally obtain the optimal path. The path similarity index and the safety index of the pre-aiming point are added to the optimization objective. Steps S1-3: Perform collision detection on the predicted path and the planned path. Irrelevant dynamic obstacles are classified as loosely coupled, and relevant dynamic obstacles are classified as tightly coupled traffic participants. Step S2: Perform dynamic game-playing sequentially on the dynamic obstacles tightly coupled with the unmanned vehicle; Step S3: For each game, establish a Stackelberg game model; Step S4: Solve the Stackelberg game model and complete the interactive trajectory planning, including the following sub-steps: Step S4-1: Design the reward function of the Stackelberg game model. The reward function is obtained through the trial-and-error velocity planning results under the corresponding strategy. The reward function includes the player's own reward and the opponent's reward, and its expression is: In the formula, The guilt coefficient is used to define the opponent's reward in a game. To compete with opponents Take strategy The corresponding reward; For players Take strategy The corresponding reward is determined by a weighted average of the average velocity, average acceleration, and average jerk throughout the entire process of traversing the conflict zone, expressed as: In the formula, , , These are average velocity, average acceleration, and average jerk, respectively. , , These are the weighting coefficients for the corresponding items; For the desired speed of vehicles on the road, This is the vehicle's maximum acceleration. The maximum acceptable jerk; Step S4-2: Solve for the optimal game strategy using backward induction; Step S4-3: The speed curve planning result of the unmanned vehicle corresponding to the optimal game strategy is used as the final model output for unmanned driving control.

2. The interactive trajectory planning method based on the Stackelberg game model according to claim 1, characterized in that, Step S2 includes the following sub-steps: Step S2-1: Obtain the area where the autonomous vehicle collides with tightly coupled traffic participants using geometric methods; Step S2-2: Determine the order of dynamic game based on the collision areas in the autonomous vehicle's planned path from near to far.

3. The interactive trajectory planning method based on the Stackelberg game model according to claim 2, characterized in that, The Stackelberg game model in step S3 includes modeling the strategy choices of the autonomous vehicle and other traffic participants, as well as modeling the determination of the leader and followers.

4. The interactive trajectory planning method based on the Stackelberg game model according to claim 3, characterized in that, The modeling of the strategy selection of the autonomous vehicle and other traffic participants is specifically as follows: Based on the issue of who gets priority passage through the conflict zone to become a road, both the autonomous vehicle itself and other traffic participants have only two decisions: priority passage or yielding. The yielding vehicle needs to choose the optimal yielding trajectory based on the trajectory of the passing vehicle.

5. The interactive trajectory planning method based on the Stackelberg game model according to claim 3, characterized in that, The modeling for determining leaders and followers specifically includes: 1) Analyze the traffic rules that the traffic participants in the game need to meet, their categories, road conditions, and the curvature of the planned and predicted paths; 2) Design a quantitative index, describing the loss function based on the order of passage, with the following expression: In the formula, , , , The coefficients corresponding to obstacle type, obstacle road distribution, path curvature, and priority loss caused by the time it takes for game participants to reach the conflict area are used to determine the leader with the smallest loss. These are type loss, lane occupancy obstacle loss, path curvature loss, and time loss to reach the collision zone.

6. The interactive trajectory planning method based on the Stackelberg game model according to claim 1, characterized in that, Specifically, step S4-3 involves obtaining a coarse velocity curve using a search method based on the ST graph, transforming the problem into a quadratic programming problem, and then using a quadratic programming QP solver to find the optimal velocity curve.

Images