An automatic driving planning method based on autoregressive traffic flow deduction and tree search

Abstract

The application discloses an automatic driving planning method based on autoregressive traffic flow deduction and tree search, and belongs to the technical field of automatic driving decision planning. The application comprises the following steps: constructing and training an autoregressive traffic flow closed-loop deduction model with the condition of a self-vehicle strategy anchor point; obtaining a historical state sequence and a global navigation path of the self-vehicle and traffic participants, and generating a candidate strategy anchor point set according to the historical state sequence and the global navigation path; then, carrying out Monte Carlo tree search, taking the aforementioned model as a forward simulator, taking the strategy anchor point set as an action space, and synchronously constructing a multi-layer sub-tree structure and completing multi-level value backtracking through single-model parallel deduction, so that the search efficiency is improved; and obtaining an optimal strategy sequence and a self-vehicle trajectory covering a planning time domain. The application realizes coupling of prediction and planning, solves the problems of insufficient consideration of interactive behaviors and single decision behaviors in traditional decision planning algorithms, and can generate a safe, efficient and long-term interactive automatic driving vehicle trajectory.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving planning method based on autoregressive traffic flow deduction and tree search. Background Technology

[0002] The planning module of an autonomous vehicle is its intelligent core, responsible for generating safe, efficient, and comfortable driving strategies in complex and dynamic traffic environments. Existing technical solutions mainly follow several paradigms, but all have inherent limitations:

[0003] 1. A hierarchical paradigm decoupling prediction and planning: This paradigm divides the system into independent prediction and planning modules. The prediction module, based on historical observations, provides trajectory predictions of the future behavior of other traffic participants; the planning module, based on these predictions, performs trajectory planning and decision-making based on rule-based or optimization methods. The main limitation of this method is that the prediction process is an open-loop process, failing to consider the feedback effects of the vehicle's planned future actions on the behavior of other vehicles. This results in planning occurring in a static, hypothetical environment that deviates from the actual interaction scenario. This easily leads to two types of problems: first, overly conservative decisions, abandoning reasonable passage opportunities due to the rigidity of prediction results; second, decisions with inherent risks, leading to conflicts due to the failure to anticipate the dynamic responses of other vehicles to the vehicle's actions. This problem is particularly prominent in highly interactive scenarios such as mixed traffic and intersections.

[0004] 2. End-to-End Paradigm Based on Imitation Learning: This type of method uses deep neural networks to directly learn driving strategies from large-scale human driving data, mapping environmental perception information end-to-end into planned trajectories or control commands. The advantage of this method is its ability to learn complex driving experiences from data, avoiding cumbersome rule and scenario design. However, this method is mechanistically a single-step, open-loop decision-making process, lacking explicit forward-looking extrapolation and logical reasoning modeling of the long-term impact of action sequences. Although the model theoretically possesses the ability to implicitly learn some patterns from data, this implicit learning cannot guarantee safe and reasonable decisions when facing scenarios outside the training data distribution. Furthermore, this method typically focuses on fitting driving behavior, lacking mechanisms for explicit safety constraints and verification of the output trajectory, thus posing certain safety risks.

[0005] 3. Sequence Decision-Making Paradigm: This paradigm models the autonomous driving planning problem as a Markov decision process, aiming to maximize long-term cumulative returns by solving for the optimal sequence of decisions. The core of such methods (e.g., Monte Carlo Tree Search) lies in the need for a state transition model capable of simulating environmental dynamics as a forward simulator to evaluate the quality of different decisions. However, this paradigm faces a dual challenge in practical applications: First, there is a contradiction between the realism requirements of the simulator and computational efficiency. To generate reliable decisions, the simulator must be realistic enough to reflect complex traffic interactions; however, high-fidelity physical or rule-based models have enormous computational overhead. Conversely, using simplified models leads to inference distortion, causing the optimal strategy obtained through the search to fail in practical applications. Second, the action space of decisions grows exponentially with the planning dimension, leading to a combinatorial explosion problem. Exhaustive or random searches within a huge action space are extremely inefficient, making it difficult to find high-quality solutions within a limited computational budget. Summary of the Invention

[0006] This invention proposes an autonomous driving planning method based on autoregressive traffic flow deduction and tree search to solve the problems mentioned in the above technical background.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] This invention discloses an autonomous driving planning method based on autoregressive traffic flow deduction and tree search, comprising:

[0009] 1) Construct and train an autoregressive traffic flow model with autonomous vehicle strategy anchor points as conditions. The model encodes the input observation sequence and generates the next observation. The inference process is carried out autoregressively to achieve closed-loop inference of traffic flow.

[0010] 2) Obtain the historical state sequence and global navigation path of the vehicle and traffic participants, and generate a candidate strategy anchor point set based on the historical state sequence and global navigation path;

[0011] 3) Based on the autoregressive traffic flow closed-loop deduction model and the candidate strategy anchor point set, a Monte Carlo tree search is performed. The autoregressive traffic flow deduction model serves as the forward simulator in the Monte Carlo tree search algorithm, and the candidate strategy anchor point set serves as the action space. During the forward simulation of the Monte Carlo tree search, the autoregressive traffic flow deduction model performs parallel deduction on multiple actions, simultaneously constructs a multi-level subtree structure, and completes multi-level value backtracking.

[0012] 4) Based on the Monte Carlo tree search results, obtain the optimal strategy anchor point sequence and vehicle trajectory for the coverage planning time domain.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0014] 1) This invention achieves deep coupling between prediction and planning, enhancing the rationality and safety of decision-making: Traditional hierarchical methods decouple prediction and planning. The prediction module performs open-loop predictions based on fixed historical information, failing to consider the dynamic impact of the vehicle's planned actions on the behavior of other traffic participants. This invention uses an autoregressive model to deduce traffic flow, predicting multi-agent behavior at each step based on the current traffic state (including real-time interactions between the vehicle and other vehicles), forming a dynamic feedback simulation environment. This method can simulate the actual impact of the vehicle's actions on the behavior of other vehicles, thus predicting potential interaction conflicts or cooperation opportunities during the planning stage. This effectively solves the problem of overly conservative or risky decisions caused by fixed predictions in traditional methods, significantly improving the rationality and safety of decision-making in highly interactive scenarios such as intersections and ramp merging.

[0015] 2) Improved decision-making search efficiency and solved the combinatorial explosion problem in complex scenarios: Traditional planning methods based on Monte Carlo tree search face a severe combinatorial explosion problem when the action space is large, resulting in low search efficiency. This invention systematically improves search efficiency through the following technical means: First, it abstracts the underlying continuous control actions into policy anchor points obtained by segmenting the global navigation path, compressing the action space from a high-dimensional continuous space into a finite discrete candidate set, significantly reducing the search breadth; Second, by setting a fixed decision cycle, the complete planning time domain is divided into multiple decision stages, making the depth of the Monte Carlo tree related to the number of decision cycles rather than directly corresponding to the specific number of inference steps, effectively controlling the search depth; Furthermore, a parallel inference mechanism is adopted in the search process to simulate future branches guided by multiple policy anchor points in batches, and simultaneously construct a multi-layer subtree structure and complete multi-node value backtracking, improving the traditional Monte Carlo tree search's "layer-by-layer sequential expansion and backtracking" to "multi-layer parallel expansion and backtracking," thereby improving search efficiency.

[0016] 3) Ensuring the quality of generated trajectories while maintaining realism, safety, and long-term optimality: This invention employs a data-driven approach to train an autoregressive inference model based on deep neural networks. This model learns the interaction patterns of traffic participants from large-scale real-world driving data, generating inference trajectories that conform to real traffic flow characteristics, thus ensuring the realism of the simulation process. Furthermore, by using Monte Carlo tree search to perform long-term forward-looking inference and evaluation of multiple possible future development paths, the long-term cumulative rewards of action sequences can be explicitly considered, thereby avoiding short-sighted decision-making and generating optimized trajectories that balance safety (e.g., collision avoidance), efficiency (e.g., arriving quickly), and comfort (e.g., smooth control). This method overcomes the shortcomings of end-to-end methods lacking explicit safety verification and imitation learning methods lacking long-term planning, achieving the generation of high-quality trajectories. Attached Figure Description

[0017] Figure 1 This is an overall system flowchart of the autonomous driving decision-making and planning method provided in the embodiments of the present invention.

[0018] Figure 2 This is a schematic diagram of the working principle of the intelligent agent coding module provided in the embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram illustrating the working principle of the map element encoding module provided in this embodiment of the invention.

[0020] Figure 4 This is a schematic diagram illustrating the working principle of the strategy anchor point encoding module provided in this embodiment of the invention.

[0021] Figure 5 This is a schematic diagram illustrating the working principle of the intelligent agent-map feature fusion layer provided in this embodiment of the invention.

[0022] Figure 6 This is a schematic diagram illustrating the working principle of the intelligent agent temporal feature fusion layer provided in this embodiment of the invention.

[0023] Figure 7 This is a schematic diagram illustrating the working principle of the strategy anchor scoring module provided in this embodiment of the invention.

[0024] Figure 8 This is a schematic diagram illustrating the working principle of the intelligent agent action prediction module provided in this embodiment of the invention.

[0025] Figure 9 This is the overall architecture diagram of the autoregressive traffic flow closed-loop extrapolation model provided in this embodiment of the invention.

[0026] Figure 10 This is a schematic diagram illustrating the role of the decision cycle in the deduction of the autoregressive traffic flow closed-loop deduction model provided in this embodiment of the invention.

[0027] Figure 11 This is a strategy anchor point and its guidance for generating autonomous vehicle behavior provided in the embodiments of the present invention.

[0028] Figure 12 This is a schematic diagram of the multi-step historical frame reuse mechanism of the autoregressive traffic flow closed-loop extrapolation model provided in this embodiment of the invention.

[0029] Figure 13 This is a schematic diagram of the improved Monte Carlo tree search process provided in an embodiment of the present invention. Detailed Implementation

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. These embodiments are implemented based on the technical solution of the present invention, providing detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0031] The overall architecture of the method described in this invention is as follows: Figure 1 As shown, its core lies in constructing a coupled framework of "learning simulator (autoregressive traffic flow inference model) + efficient searcher (improved Monte Carlo tree search)". This method first trains a dynamic simulation model (autoregressive traffic flow inference model) capable of simulating traffic flow interactions offline; during online execution, this model and the Monte Carlo tree search form a closed-loop system.

[0032] The overall workflow during online operation is as follows:

[0033] 1) Acquiring observation information: The system acquires the historical state sequence (such as position and speed) of the vehicle and surrounding traffic participants, map information, and global navigation path.

[0034] 2) Candidate strategy anchor point generation: Based on the global path, a set of discrete candidate strategy anchor points is generated, and each candidate strategy anchor point represents a driving strategy.

[0035] 3) Interactive Inference and Search: An improved Monte Carlo tree search is conducted using policy anchor points as decision-making actions. The core of the search process is to expand and forward simulate nodes in the Monte Carlo tree search using an autoregressive traffic flow inference model, performing parallel, closed-loop inferences on the development of traffic flow under different policy anchor point-guided vehicle behaviors. This is to evaluate the long-term benefits of the macroscopic driving intentions of vehicles corresponding to candidate policy anchor points.

[0036] 4) Decision output: After the search is completed, select the optimal strategy sequence and output the final vehicle planning trajectory.

[0037] This scheme integrates prediction and planning, and addresses the problem of insufficient interaction consideration in traditional hierarchical methods by considering the dynamic impact of vehicle decisions on other vehicles in traffic flow forward extrapolation. At the same time, it improves the efficiency of search-based algorithms by abstracting the action space and using a parallel extrapolation mechanism.

[0038] In this embodiment, the core function of the autoregressive traffic flow closed-loop extrapolation model is to perform forward extrapolation of traffic flow in an autoregressive manner. Figure 9 For the overall architecture of the model, from Figure 9 As can be seen, the autoregressive traffic flow closed-loop inference model mainly includes an agent coding module, a map element coding module, a policy anchor point coding module, an agent-map feature fusion layer, an agent temporal feature fusion layer, a policy anchor point scoring module, and an agent action prediction module. The working principle of each component module (layer) of the model is described in [link to documentation]. Figures 2 to 8 As shown, the following is a detailed description in conjunction with the text.

[0039] The agent encoding module further includes an agent feature embedding layer and an agent-agent feature fusion layer. This invention encodes all agents within each frame of the observation sequence using the agent encoding module; specifically, as shown... Figure 2 As shown, agent information (state, category, etc.) is first embedded into agent features through an agent feature embedding layer. Then, the features between agents are fused through an agent-agent feature fusion layer based on a self-attention mechanism to model the agent interaction behavior and output the features of each fused agent.

[0040] The map element encoding module further includes a map element feature embedding layer and a map element feature fusion layer. Since the map is a static element and remains unchanged throughout the observation sequence, this invention only encodes all map elements within the last frame of the observation sequence using the map element encoding module, such as... Figure 3 As shown, map element information (such as road marking coordinates, categories, etc.) is embedded into map element features through a map element feature embedding layer. Then, a map element feature fusion layer based on a self-attention mechanism is used to fuse the features between map elements in order to model the topological relationship between map elements. Finally, map element features are output.

[0041] The policy anchor encoding module further includes a policy anchor feature embedding layer and a policy anchor feature fusion encoding layer. This invention encodes all policy anchors in the candidate policy anchor set using the policy anchor encoding module, such as... Figure 4 As shown, the strategy anchor information (anchor coordinates, associated road markings, etc.) is first embedded into strategy anchor features through a strategy anchor feature embedding layer. Then, the features between strategy anchors are fused through a strategy anchor feature fusion encoding layer based on a self-attention mechanism. Finally, the features of all strategy anchors in the candidate strategy anchor set are output.

[0042] Agent-Map Feature Fusion Layer: such as Figure 5 As shown, an agent-map feature fusion layer based on a cross-attention mechanism is used to achieve feature fusion between agents and map elements by using agent features as queries and map element features as keys and values, thereby modeling the dependency and constraint relationships between agents and map elements.

[0043] Agent temporal feature fusion: such as Figure 6 As shown, for each agent within the observation, the agent's intra-frame features are fused across frames in the observation sequence through a self-attention mechanism-based agent temporal feature fusion layer to model the temporal features of each agent in the observation sequence.

[0044] The policy anchor scoring module further includes an agent-policy anchor feature fusion layer and a policy anchor scoring layer. This invention uses the policy anchor scoring module to score policy anchors in the candidate policy anchor set, such as... Figure 7 As shown, firstly, an agent-policy anchor feature fusion layer based on cross-attention mechanism is used to fuse the vehicle features with the policy anchor set features, using the vehicle features as the query and all candidate policy anchor features as the key and value. Then, the matching score between the policy anchor and the current traffic scene is output by dividing the policy anchor into layers.

[0045] The agent action prediction module further includes a vehicle-policy anchor feature fusion layer and an action prediction layer. This invention uses the agent action prediction module to predict the action probability distribution for all traffic participants in the observation, such as... Figure 8 As shown, for the features of the autonomous vehicle, the autonomous vehicle features and the target policy anchor features are fused through the autonomous vehicle-policy anchor feature fusion layer, and then input into the action prediction layer to output the autonomous vehicle action probability distribution that matches the target policy anchor. For other agents, their features are directly input into the action prediction layer to output the action probability distribution.

[0046] Forward update: Sample a specific action for each traffic participant from the action probability distribution of the agent output by the action prediction module, thereby updating the state of all traffic participants forward, generating the inferred state for the next moment, and feeding this state back to the model input as new observation information to form an autoregressive closed-loop inference.

[0047] like Figure 9 As shown, the model realizes the extrapolation of each extrapolation time step in the autoregressive extrapolation process through the above workflow;

[0048] The derivation process of the autoregressive traffic flow model is as follows:

[0049] 1) Initialization: The historical state sequence of the vehicle and traffic participants acquired at the current time is used as the initial observation sequence. Set the planning time domain. and decision-making cycle duration In this example, , .

[0050] 2) Strategy anchor point selection: such as Figure 10 The deduction process shown combines decision-making cycles. For each decision-making cycle (starting from the current moment)... Beginning, with (Step size), at the beginning of the decision cycle, based on the current observation sequence (Initial time) The strategy anchor point for guiding the vehicle during this period is determined using one of the following two methods. .

[0051] A: External specification: Receives the action to be extended when performing a Monte Carlo tree search;

[0052] B: Autonomous Decision-Making: The model evaluates the matching score of each policy anchor in the candidate policy anchor set P through its internal policy anchor scoring module. And select the anchor point with the highest score;

[0053] 3) Circular deduction within the decision-making cycle: This involves performing a basic deduction at each time step within the current decision-making cycle. (In this example, Perform the following operations:

[0054] A. Execute the single-step inference process of the model single-step inference procedure: This involves setting the current observation sequence... Candidate policy anchor set and selected policy anchor (If specified externally) are input into the model together, perform a single-step inference, and obtain the next time step. The agent's inference state To form new observations ;

[0055] B. After each simulation, check whether the decision-making cycle has ended (i.e., whether the simulation has been completed). (Step 3). If not completed, repeat step 3; if completed, proceed to the next decision cycle and repeat step 2).

[0056] 4) Loop Termination: Repeat steps 2) and 3) until the total simulation duration reaches the set planning time domain. The endpoint is then used to complete a closed-loop forward traffic flow simulation that considers multi-cycle strategy guidance in an autoregressive manner.

[0057] Furthermore, to illustrate the specific significance of strategy anchors and their guidance in generating autonomous vehicle behavior, such as... Figure 11 In the demonstrated left lane change scenario, the navigation path is specified as the left lane, and the candidate strategy anchor point set is the division of the center line of the left lane. For each candidate anchor point, the model will infer and generate the trajectory of the vehicle traveling towards that anchor point within the current decision cycle.

[0058] Furthermore, such as Figure 12 As shown, since map elements and candidate policy anchor sets remain unchanged during the inference process and are considered static elements, the static element features encoded by the model during single-step inference will be stored as a static element feature cache. This information can be reused in subsequent inferences. Simultaneously, since the agent's information in the input observation sequence is already fixed, the agent features encoded in the observation sequence during the model's single-step inference are stored as an agent feature cache. The model can be reused when performing the next inference. Only the new agent state needs to be encoded. and encode the agent features Used to update the agent feature cache Based on the aforementioned static element caching and agent feature caching mechanisms, it is not necessary to perform the next inference step on the complete observation sequence. Re-encode to improve efficiency.

[0059] Model training implementation:

[0060] 1) The autoregressive traffic flow model is trained by a teacher-forced strategy, that is, in each single-step training, the input observation sequence is the real observation sequence, and the input target strategy anchor point is the strategy anchor point corresponding to the real trajectory.

[0061] 2) The overall training loss function of the autoregressive traffic flow model. Loss prediction based on action Matching loss with policy anchor Together they constitute the following, and are implemented as follows:

[0062] A. Action prediction loss The goal is to learn accurate predictions of traffic participant behavior by minimizing the cross-entropy loss between the model's predicted action probability distribution and the actual action at the next moment.

[0063] B. Policy Anchor Matching Loss The goal is to supervise the model's selection of policy anchors that better reflect real-world driving behavior by minimizing the cross-entropy loss between the model's predicted scores for policy anchors and the actual matching anchors.

[0064] C. Overall Loss and Training: The overall loss function is... ,in To balance the weights (in this embodiment, we take...) The model is trained on large-scale real-world driving datasets, such as the Waymo Motion OpenDataset.

[0065] like Figure 13 As shown, the specific implementation steps of the Monte Carlo Tree Search (MCTS) of the present invention are as follows:

[0066] 3.1 Initialization:

[0067] 1) Construct the root node The root node state corresponds to the state of the vehicle and traffic participants obtained at the current moment.

[0068] 2) Generate a set of candidate strategy anchor points based on the root node state and the global navigation path. This set of anchor points serves as the action space for MCTS.

[0069] 3) Set the planning time domain and decision-making cycle duration Therefore, the depth of the Monte Carlo tree Determined as (in this example) Each level of the tree corresponds to the selection of the strategy anchor point in a decision-making cycle.

[0070] 3.2 Search Loop:

[0071] Within a defined computational budget (such as a fixed number of iterations or a time limit), repeatedly perform the following selection, expansion and simulation, and backtracking steps:

[0072] 3.2.1) Selection:

[0073] From the root node Start by recursively selecting child nodes until a leaf node that is not fully expanded is reached (i.e., the node still has unvisited candidate strategy anchors).

[0074] 3.2.2) Extension and Simulation:

[0075] 1) When the selection step reaches a leaf node that is not fully expanded. At that time, from the candidate anchor set Select an unexplored anchor point To expand upon.

[0076] 2) Parallel Inference: This involves batch processing multiple unexplored candidate anchor points under a given leaf node. Specifically:

[0077] A. Move the leaf nodes The corresponding complete historical observation sequence from the root node to the leaf node (i.e., the state extrapolated to the start of the current decision cycle) serves as the shared extrapolation input.

[0078] B. All newly expanded candidate anchor points Each of these data is combined with the observed sequence to form a batch of data.

[0079] C. Input this batch of data into the autoregressive traffic flow closed-loop simulation model and initiate a parallel forward simulation covering the entire remaining planning time domain.

[0080] 3) Subtree Instantiation: Parallel forward inference essentially synchronously constructs a subtree based on... The structure is a multi-level subtree rooted at [root]. The state at the end of each decision cycle in the deduction sequence corresponds to a new node in the subtree. These nodes, along with their associated actions (policy anchors) and states, are immediately instantiated and added to the Monte Carlo tree.

[0081] 3.2.3) Backtracking:

[0082] 1) Reward calculation: For each complete sequence obtained through parallel derivation Calculate its cumulative reward ;

[0083] 2) Multi-level value backtracking: Calculate the cumulative rewards... Simultaneously, it backtracks and updates all nodes in the entire multi-level subtree instantiated in this simulation. That is, starting from the last state node of the simulation sequence, it backtracks along the simulation path to the leaf node. The number of visits to each node on the path and cumulative value All have been updated: , .

[0084] The batch backtracking mechanism avoids the inefficiency of traditional MCTS, which only backtracks one path per simulation, and maximizes the utilization of search information.

[0085] When the search loop ends (the computational budget is reached):

[0086] A. From the root node Start by selecting the strategy anchor point corresponding to the child node with the most visits. This is the optimal strategy for the first decision cycle.

[0087] B. Then, by traversing downwards along the path with the most visits, we can obtain a result covering the entire planning time domain. Optimal policy sequence .

[0088] C. Simultaneously, the vehicle trajectory corresponding to this path, generated by the autoregressive model, is the final output planned trajectory. In actual control, typically only the first decision cycle is executed. The trajectory within the planning cycle is then re-perceived in the environment and the entire MCTS process is repeated in the next planning cycle to achieve rolling optimization.

[0089] D. Finally, for each candidate anchor point The model outputs a complete future state-action sequence from the current moment to the end of the planning time domain. And the reward sequence throughout the entire deduction process.

[0090] Furthermore, in the selection step, a hybrid strategy is employed when selecting nodes. For nodes... its child nodes (corresponding action) (Selection score) Calculated in two parts:

[0091] A. Upper confidence threshold The upper confidence bound value is calculated based on the upper confidence bound algorithm of Monte Carlo tree search, and its value combines the historical average reward and the number of visits of child nodes; it is obtained using the PUCT (Polynomial Upper Confidence Trees) algorithm. B. Heuristic evaluation value This value comes directly from the strategy anchor scoring module within the autoregressive traffic flow model. During the search process, when a node is visited for the first time... At that time, the model will be invoked to quickly evaluate the matching scores of all candidate anchor points in that state and normalize them into a probability distribution, which will then serve as the basis for further evaluation. and The foundation.

[0092] Ultimately, the combined score was selected as follows: , To reconcile the weights, the algorithm selects the child node with the highest score for further exploration.

[0093] Furthermore, in the details of the parallel deduction process in the expansion and simulation steps, such as... Figure 13 As shown, in the first decision cycle (i.e. from The next one begins The model strictly follows its assigned extension anchors. The deduction is performed, with the ending state of the first decision cycle serving as the expanded leaf node. The deduction within the first decision cycle can be viewed as a node expansion step in a Monte Carlo tree search; while in subsequent decision cycles (the... Within the planning time domain (and beyond), the model performs autonomous replanning. Based on the real-time traffic state at that moment, it autonomously decides the strategy anchor point to be adopted in subsequent decision-making cycles through its internal strategy anchor point prediction layer, and continues to extrapolate the actions of all traffic participants until the planning time domain is reached. The endpoint, and the subsequent deductions in the decision-making cycle can be regarded as the forward simulation steps in Monte Carlo tree search.

[0094] Furthermore, the cumulative reward in the backtracking step From the current moment To the end of the planned time domain Instant reward The sum of the discounts yields:

[0095]

[0096] in, This is a discount factor used to balance short-term and long-term rewards (in this example, it is taken as...). ).

[0097] The reward function is designed to guide the agent to learn safe, efficient, comfortable, and traffic-compliant driving behaviors. Those skilled in the art should understand that the reward items and their specific forms given below are merely exemplary embodiments and not the sole limitation of the invention. In practical applications, reward items can be added, deleted, combined, or defined using different mathematical forms according to specific scenarios and performance requirements:

[0098] Instantaneous reward function It is a weighted sum of multiple reward items, each targeting a specific performance metric:

[0099]

[0100] in, These are the weighting coefficients for each reward item, used to adjust the priority of different optimization objectives.

[0101] The following are the specific implementation details for each reward item:

[0102] A. Security Bonus Items:

[0103] Collision Penalty Penalty collision:

[0104]

[0105] in, This is an indicator function; its value is 1 if the vehicle collides with a traffic participant or road boundary, and 0 otherwise. The corresponding penalty value is a very large positive value to ensure that the search process absolutely avoids collisions.

[0106] B. Efficiency Bonus Items:

[0107] Progress Rewards This encourages vehicles to move forward along the global reference path:

[0108]

[0109] in, It is a bicycle. The vertical displacement along the global reference path at all times directly rewards speed.

[0110] C. Comfort Bonus Items:

[0111] Comfort penalty Punish aggressive driving maneuvers and encourage smooth driving:

[0112]

[0113] in: It is the longitudinal acceleration of the vehicle. It's accelerometer. It is the curvature of the vehicle's trajectory. , , These are the corresponding weighting coefficients used to balance the impact of different factors on comfort.

[0114] D. Traffic rule incentives:

[0115] Traffic rule violation penalties Punish traffic violations, such as running red lights and speeding.

[0116]

[0117] in: It is an indicator function; if the vehicle violates traffic signal rules, its value is 1, otherwise it is 0. The penalty value in response; It is an indicator function; if the vehicle exceeds the road speed limit, its value is 1, otherwise it is 0. The penalty value in response.

[0118] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. An automatic driving planning method based on autoregressive traffic flow deduction and tree search, characterized in that, Comprising the following steps: 1) Construct and train a self-recursive traffic flow deduction model conditioned on the ego vehicle strategy anchor point, the model encodes the input observation sequence, generates the next step observation, and the deduction process is recursively performed to realize the closed-loop deduction of the traffic flow; The self-recursive traffic flow deduction model is deducted by the following way: 11) At each deduction time step, the model receives the observation sequence from the initial time to the current time, each frame of observation in the observation sequence contains the state of the traffic participants and the map element information; the model performs feature embedding and feature fusion on the agent, map element information, and strategy anchor point information in the candidate strategy anchor point set in the observation sequence to realize the encoding of the observation sequence; The model further outputs the action probability distribution of each traffic participant within a fixed time step in the future, and the action probability distribution of all traffic participants constitutes a joint probability distribution; 12) Sampling from the joint probability distribution to determine a specific action for each traffic participant; 13) Forward update the state of all traffic participants in the observation to obtain the deduction state of the next deduction time step; 14) The updated deduction state is taken as new observation information and incorporated into the observation sequence to obtain a new observation sequence; 15) The model repeats the aforementioned steps 11)~14) to realize the closed-loop deduction of the traffic flow; 2) Obtain the historical state sequence of the ego vehicle and the traffic participants and the global navigation path, and generate a candidate strategy anchor point set based on the historical state sequence and the global navigation path; 3) Based on the self-recursive traffic flow closed-loop deduction model and the candidate strategy anchor point set, a Monte Carlo tree search is carried out, wherein the self-recursive traffic flow deduction model serves as a forward simulator in the Monte Carlo tree search algorithm, and the candidate strategy anchor point set serves as an action space; in the forward simulation process of the Monte Carlo tree search, the self-recursive traffic flow deduction model performs parallel deduction on multiple actions, simultaneously constructs a multi-layer sub-tree structure and completes multi-level value backtracking; 4) Based on the Monte Carlo tree search result, the optimal strategy anchor point sequence covering the planning time domain and the ego vehicle trajectory are obtained.

2. The method of claim 1, wherein, The candidate strategy anchor point set in step 2) includes multiple candidate strategy anchor points, which are generated by the following way: based on the current state of the ego vehicle, its global navigation path is obtained, and the path is divided into continuous segments, each path segment serves as a candidate strategy anchor point.

3. The method of claim 1, wherein, The self-recursive traffic flow deduction model also accepts the candidate strategy anchor point set, calculates the matching score for the candidate strategy anchor point through a strategy anchor point scoring module; and the model performs feature fusion based on the attention mechanism on the features of the ego vehicle in the encoding result of the observation sequence and the features of the selected strategy anchor point to guide the generation of the ego vehicle action probability distribution conforming to the macro intention of the strategy anchor point.

4. The method of claim 1, wherein, The autoregressive traffic flow deduction model has autonomous re-planning capability in deduction: the deduction process covers multiple decision cycles, and the action of the ego vehicle in each decision cycle is generated by the strategy anchor point selected in the cycle; at the moment when the model deducts to the end of a certain decision cycle, a new strategy anchor point is autonomously selected or determined according to external input based on the complete observation sequence formed by the deduction up to that moment; then, the model continues to deduct the traffic flow in the next decision cycle based on the complete observation sequence and in combination with the new strategy anchor point. In this way, the model can flexibly switch the macro intention of the ego vehicle during the deduction process, and at the moment when the decision cycles are connected, it can deduce different branches of traffic flow based on different strategy anchor points, which conforms to the node expansion paradigm of Monte Carlo tree search.

5. The method of claim 1, wherein, The step 3) of expanding the Monte Carlo tree search includes iteratively performing the following steps: 31) selection: starting from the root node of the tree, recursively selecting child nodes according to the tree strategy until reaching an unexpanded leaf node, which represents the state of all traffic participants at a certain moment; 32) expansion and simulation: expanding one or more child nodes for the leaf node, the expansion process being: inputting the observation sequence from the root node to the leaf node into the autoregressive traffic flow deduction model, combining an action in the action space, i.e., the strategy anchor point, to deduce the state of all traffic participants after a decision cycle, forming a child node; then, simulation: inputting the observation sequence from the root node to the child node into the model to continue deduction until the end of the planning time domain, and the strategy anchor point of each decision cycle in the simulation process being autonomously selected by the model; 33) backtracking: calculating the reward based on the deduction result, and updating the statistical information of all nodes on the path from the child node to the root node with the reward; Wherein, the decision cycle length is set to be fixed , the depth L of the tree is determined by the planning horizon T and the decision cycle , satisfying , each layer of the tree corresponds to the selection of a strategy anchor point in a decision cycle.

6. The method of claim 5, wherein, The expansion and simulation processes are performed in parallel on different strategy anchor points, specifically: combining the complete observation sequence from the root node to the leaf node corresponding to the leaf node to be expanded with multiple candidate strategy anchor points to form batch data input into the autoregressive traffic flow deduction model, thereby independently and in parallel deducing the future traffic flow branches guided by different strategy anchor points.

7. The method of claim 5, wherein, The expansion and simulation step further includes: instantiating the multiple complete state-action sequences corresponding to different candidate strategy anchor points generated by the model during parallel deduction in the simulation process and the multiple child nodes expanded by the model during parallel expansion into a multi-layer sub-tree structure in the Monte Carlo tree search.

8. The method of claim 7, wherein, The backtracking step specifically includes: updating the long-term cumulative reward calculated based on the deduction result to all nodes in the multi-layer sub-tree instantiated in this deduction.

9. The method of claim 5, wherein, In the selection step, the selection strategy of the nodes is a hybrid strategy, the value of which is obtained by weighting the upper confidence bound value and the heuristic evaluation value, wherein the upper confidence bound value is calculated according to the upper confidence bound algorithm of the Monte Carlo tree search, and the value integrates the historical average reward and the visit number of the child nodes; the heuristic evaluation value is derived from the autoregressive traffic flow closed-loop deduction model, and is the matching score predicted for the candidate strategy anchor set based on the current observation sequence.

Images