Automatic driving track predicting and planning method based on time sequence autoregression model

Abstract

The invention discloses an automatic driving track prediction and planning method based on a time sequence autoregression model, and the method comprises the steps: obtaining the data of a current vehicle intelligent agent, and carrying out the preprocessing of the data; discretizing the intelligent agent track and constructing a motion mark vocabulary library; inputting the agent trajectory and map information into a time sequence autoregression trajectory generation model based on a Transform architecture, and performing trajectory prediction by using an autoregression mode; finely adjusting the model by using a reinforcement learning method; and generating a future trajectory of the agent. Compared with the prior art, the method has the beneficial effects that the prediction precision and stability can be remarkably improved, the modeling capability of interaction between intelligent agents can be remarkably improved, learning is reinforced, and the behavior strategy of the intelligent agents is dynamically adjusted by utilizing a reward and punishment mechanism through continuous interaction with the environment; the intelligent agent is encouraged to select a safer and more effective path, and potential dangerous behaviors caused by incompleteness of training data are avoided.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and specifically to an autonomous driving trajectory prediction and planning method based on a temporal autoregressive model. Background Art

[0002] With the continuous advancement of deep learning technology, data-driven trajectory prediction and planning methods for autonomous driving have shown great potential in solving planning problems in complex situations. Compared with traditional algorithms, data-driven trajectory prediction and planning methods can generate intelligent decision paths by learning the driving habits and styles of human drivers from massive data, thereby eliminating the step of manual judgment of complex algorithm logic in practical applications. However, existing trajectory prediction and planning models are usually difficult to generate consistent trajectories in multi-agent scenarios, and cannot effectively predict and plan the interactive behaviors between agents. In addition, data-driven models based on supervised learning have the problem of "out of distribution" (hereinafter referred to as OOD), that is, the model may gradually accumulate small errors in the data or neural network output during the iterative planning process, resulting in significant deviations in the planning results, and even eventually causing collision accidents. Existing supervised learning methods cannot effectively solve this problem, which in turn affects the accuracy and reliability of trajectory planning. Summary of the invention

[0003] The purpose of the present invention is to provide an automatic driving trajectory prediction and planning method based on a time series autoregressive model to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions: In a first aspect, the present invention provides an automatic driving trajectory prediction and planning method based on a temporal autoregressive model, the method comprising: Obtain the data of the current vehicle agent and pre-process the data; Discretize the agent’s trajectory and build a motion labeling vocabulary; The agent trajectory and map information are input into the temporal autoregressive trajectory generation model based on the Transformer architecture and the trajectory is predicted using the autoregressive method; Fine-tune the model using reinforcement learning methods; Generate future trajectories for the agent.

[0005] Preferably, the data of the current vehicle agent is obtained at least includes: agent trajectory information in the scene, map lane centerline information, map lane boundary line information and traffic light information.

[0006] Preferably, the discretization of multi-agent trajectories and construction of a motion marker vocabulary library comprises: By normalizing the trajectory of each agent, its position and orientation are standardized relative to the initial moment; Discretize the incremental actions of the trajectory into a series of predefined discrete values, and construct a discretized action vocabulary by uniformly quantizing each coordinate axis; Map each consecutive trajectory increment to a corresponding index, converting the continuous action of the trajectory into a discrete tag sequence; Combine the incremental motion of each coordinate into a single index value as the motion identifier.

[0007] Preferably, the step of inputting the agent trajectory and map information into a temporal autoregressive trajectory generation model based on a Transformer architecture and performing trajectory prediction using an autoregressive method includes: Receive the scene initial state information through the encoder and generate a shared scene embedding; Based on the output of the previous moment and the fixed scene embedding, predict the behavior of all agents in the current time step; The decoder takes as input a set of motion identifiers and a scene embedding at each prediction step and generates a distribution over N output identifiers for the number of agents in the scene.

[0008] Preferably, the decoder generates a distribution of N output identifiers of the number of agents in the scene by inputting a set of motion identifiers and scene embeddings in each prediction step, including: The decoder is divided into multiple layers, each of which applies a self-attention mechanism between input identifiers and a cross-attention mechanism on the scene embedding; Set all N motion identifiers in step t to follow each other and all previous identifiers, where each row represents a query identifier, each column represents a key identifier, and the green block represents the key identifier that the query can follow; In running times After the prediction step, we get , the output motion identifiers form the complete trajectory of N agents.

[0009] Preferably, the fine-tuning of the model using a reinforcement learning method comprises: Model the complete trajectory generation of N agents as a multi-agent Markov decision process; The reward function is designed to be whether each agent collides with the map boundary or other agents at each time step at each moment. The calculation formula is: Where a and ß are the weight coefficients for collision with road boundaries and collision with other agents, respectively. Assume that the model autoregressively unfolds actions in T_{pred} time steps. For step t and each agent i, the reward is: Where is the reward of agent i at step t', γ is the discount factor used to weight future rewards; The REINFORCE algorithm is used to calculate the policy gradient and perform reinforcement learning training to further optimize the decision-making ability of the model.

[0010] In a second aspect, the present invention provides an automatic driving trajectory prediction and planning device, which is used to implement the method described in any of the above embodiments, and the device includes: An acquisition module, which is used to acquire data of the current vehicle agent and pre-process the data; A construction module, the construction module is used to discretize the intelligent body trajectory and build a motion label vocabulary library; A prediction module, which inputs the agent trajectory and map information into a temporal autoregressive trajectory generation model based on the Transformer architecture and uses an autoregressive method to perform trajectory prediction; A fine-tuning module, wherein the fine-tuning module uses a reinforcement learning method to fine-tune the model; A generation module is used to generate the future trajectory of the intelligent agent.

[0011] In a third aspect, the present invention provides an electronic device comprising at least one processor, wherein the processor is communicatively connected to at least one memory, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the methods described in the above embodiments.

[0012] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method described in any one of the above embodiments when executed.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. Multi-agent trajectory tokenization: The present invention innovatively adopts a multi-agent joint trajectory tokenization method to represent the trajectory of each agent as a set of discretized Tokens. This method not only improves the accuracy of trajectory representation by discretizing the continuous movement of the trajectory into a series of quantized actions, but also transforms the trajectory prediction and planning problem into a relatively simple sequence prediction problem, that is, predicting the next Token. This representation method converted into a Token sequence provides a theoretical basis for the application of time series autoregressive models and can effectively capture the interaction relationship between agents. Compared with traditional trajectory prediction methods, the tokenization method of the present invention can use deep learning models to model trajectories more efficiently, especially in complex multi-agent scenarios, which can significantly improve prediction accuracy and stability. Specifically, after representing the trajectory as a combination of Tokens, the model can use the characteristics of time series autoregression to sequentially predict the state of each agent at future moments, thereby achieving more accurate and consistent multi-agent trajectory planning.

[0014] 2. Modeling the interaction process of intelligent agents with the time series autoregression model: The Transformer architecture contained in the time series autoregression model can use the attention mechanism to efficiently capture the interaction process between intelligent agents, thereby displaying the interactive game of modeled intelligent agents. The time series autoregression model can significantly improve the ability to model the interaction between intelligent agents by converting the trajectory prediction problem into a time series prediction problem and using the Transformer's self-attention mechanism. At each time step, the model uses the attention mechanism to dynamically adjust the weights between the agents based on the state of the current agent and the states of other agents, thereby accurately capturing the role and influence of each agent in complex interactions. In this way, the model can not only predict the future trajectory of a single agent, but also accurately simulate the interactive game process between multiple agents.

[0015] 3. Alleviate the OOD problem: This invention effectively alleviates the performance degradation problem of traditional models when facing "out-of-distribution" (OOD) data by introducing reinforcement learning algorithms. In autonomous driving scenarios, models are usually only exposed to limited environmental samples during training, which may lead to prediction failures and decision-making errors when facing unseen environments in practical applications. Reinforcement learning uses a reward and punishment mechanism to dynamically adjust the behavior strategy of the intelligent agent through continuous interaction with the environment, encouraging the intelligent agent to choose a safer and more effective path and avoid potential dangerous behaviors caused by the incompleteness of training data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a method according to an embodiment of the present invention; Figure 2 It is a sub-flow chart of step S200 of the present invention; Figure 3 It is a sub-flow chart of step S300 of the present invention; Figure 4 It is a sub-flow chart of step S330 of the present invention; Figure 5 It is a sub-flow chart of step S400 of the present invention; Figure 6 A device module diagram of an embodiment of the present invention; Figure 7 An example diagram of the operation scenario of the present invention; Figure 8 This is a module diagram of an electronic device according to the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1 The present invention provides a technical solution: a method for predicting and planning an autonomous driving trajectory based on a temporal autoregressive model, comprising: S100: Obtain data of the current vehicle intelligent body and pre-process the data.

[0019] In this embodiment, the intelligent body trajectory information, map lane centerline information, map lane boundary line information, and traffic light information in the scene are collected and analyzed. These raw data are processed, for example, including data cleaning (removing noise, outliers, etc.), data format conversion (converting data from different sensors or data sources into a unified format), coordinate system unification (making the intelligent body trajectory information and map information in the same coordinate system), etc., to ensure the quality and availability of the data. Provide accurate, complete and appropriate input data for subsequent trajectory prediction and planning.

[0020] S200: Discretize the agent trajectory and build a motion tag vocabulary library.

[0021] In this embodiment, the specific operation of trajectory discretization is to divide the continuous trajectory of the agent according to certain rules (such as time interval or spatial distance) and convert it into discrete small segments. For example, if it is divided by time interval, a trajectory point can be taken every certain time (such as 0.5 seconds), thereby obtaining a series of discrete trajectory point sequences.

[0022] The specific operations of constructing a motion tag vocabulary include: defining a series of tags that can describe the motion state of the agent, such as forward, backward, left turn, right turn, acceleration, deceleration, uniform speed, etc. Then, according to the characteristics of the discretized trajectory segment, each segment is represented by a corresponding motion tag. For example, if the coordinates of the agent in a trajectory segment gradually increase in the horizontal direction and the speed is stable, it can be marked as "uniform forward speed". In this way, all possible motion state tags are summarized to form a motion tag vocabulary, in preparation for the subsequent representation of the trajectory as an identifier (Token) and input into the model.

[0023] S300: Input the agent trajectory and map information into the temporal autoregressive trajectory generation model based on the Transformer architecture and use the autoregressive method to predict the trajectory.

[0024] In this embodiment, specifically, it includes data input, encoding and decoding processes, wherein: Data input: The preprocessed map information (such as lane centerline, boundary line, traffic light status, etc.) and agent motion information (discretized trajectory and related status) are input into the temporal autoregressive trajectory generation model based on the Transformer architecture.

[0025] Encoding process: The encoder of the model takes the representation of the initial state of the scene (including the state of traffic lights, map topology information, and historical trajectory data of all traffic participants) as input to generate a scene embedding. This scene embedding can capture the key information in the scene and provide context for subsequent trajectory prediction.

[0026] Autoregressive decoding prediction: In the decoder stage, a Transformer-based autoregressive model is used to iteratively generate the joint trajectory distribution of agents for future time steps that need to be predicted. At each prediction step, the decoder takes a set of motion tokens (from the previous step or initial input) and scene embedding as input, and generates a distribution of all agent tokens in the scene. Through autoregression, that is, the prediction of each time step is based on the output of the previous time step, the model can gradually predict the trajectory state of the agent at each future time step, and can take into account the interaction relationship between agents. For example, when predicting the trajectory of agent A at a certain moment, the previous trajectories and current states of other agents (such as agents B, C, etc.) will be taken into account, so as to more accurately predict agent A.

[0027] S400: Fine-tune the model using reinforcement learning methods.

[0028] In this embodiment, specifically, the problem is first modeled: the trajectory prediction and planning problem in the autonomous driving scenario is modeled as a multi-agent Markov decision (MDP) process, clarifying elements such as action, state, dynamic transfer process and observation.

[0029] Action: The action space of the agent is defined as the incremental action space. Each agent's action is represented by its acceleration in the X and Y directions at the current moment, and these accelerations are discretized into a uniformly quantized grid represented by discrete indices.

[0030] State: The state space includes all information describing the agent and its surroundings, such as map features, joint states of the agents, and traffic light states.

[0031] Dynamic transfer process: Based on the motion laws of the physical model, the agent's position at the next moment is calculated according to its current state (position and velocity) and the action taken (acceleration), thereby defining the rules for the agent to transfer from one state to the next.

[0032] Observation: The observation space contains the historical context (all objects, traffic light states, and map features at historical time steps), the historical actions of all objects, and the identity of the current agent, helping the agent understand the current scene and environment state.

[0033] Furthermore, the reward function is designed according to whether each agent collides with the map boundary or other agents at each moment. By setting an indicator function representing the collision of the agent with the boundary at a moment and a corresponding weight coefficient, the agent is encouraged to choose a safer path to avoid collision.

[0034] Furthermore, the policy gradient method REINFORCE with baseline is used to further optimize the decision-making ability of the model. The REINFORCE algorithm with baseline reduces the variance of policy updates and accelerates the learning process by introducing an advantage function. While training the model, a state value function network is trained to evaluate the value of the current state, and the advantage function is calculated as a baseline to represent the advantage of the agent taking a certain action in this state. Then, based on the calculated gradient information, the policy network is updated to gradually optimize the agent's behavioral decisions, so that the model can learn a better trajectory generation strategy, make more reasonable decisions when facing complex and changing environments, and improve the safety and efficiency of trajectory planning.

[0035] S500: Generate the future trajectory of the intelligent agent.

[0036] In this embodiment, the output of the time series autoregressive trajectory generation model is the probability distribution of the future trajectory of each agent. The future trajectory token of the specified length is obtained from the probability distribution through sampling or search methods. The sampling method can randomly sample according to the probability distribution, and obtain the future trajectory token through iterative generation; the search method selects the appropriate token according to the probability distribution, and also obtains the future trajectory token of the specified length through iterative generation. Finally, the trajectory motion labeling modeling method is used to restore the token to the agent trajectory, and the number of generated scenes can be customized according to the task requirements. In this way, specific future trajectories that can be used by autonomous driving vehicles are obtained, and the vehicle can make driving decisions based on these trajectories, such as selecting the appropriate lane, controlling the speed, planning turns, etc., to achieve safe and efficient autonomous driving.

[0037] In one embodiment of the present invention, the data of the current vehicle agent acquired in step S100 at least includes: agent trajectory information in the scene, map lane centerline information, map lane boundary line information and traffic light information.

[0038] In this embodiment, when autonomous driving trajectory prediction and planning is required, the data needs to be preprocessed first to analyze the input data required by the model. For example, different data can be processed differently: Agent trajectory information: Collect trajectory data of agents (such as vehicles) in the scene. This data records information such as the location of the agent at different time points and is the basis for subsequent processing and prediction.

[0039] Map lane centerline information: Obtain the centerline data of the lane on the map, which helps the model understand the direction and position of the lane, so as to better plan the vehicle's driving trajectory within the lane.

[0040] Map lane boundary information: Clarify the boundary range of the lane so that the model knows the boundary of the area where the vehicle can drive, and avoid planning unreasonable trajectories beyond the lane range.

[0041] Traffic light information: includes the location, status (such as red, green, yellow, etc.) and switching time of traffic lights. This information is crucial for vehicle trajectory planning in traffic scenarios such as intersections. For example, a vehicle needs to decide whether to stop and wait or continue driving based on the status of the traffic light.

[0042] In one embodiment of the present invention, see Figure 2 , step S200 specifically includes the following steps: S210: Normalize the trajectory of each agent so that its position and orientation are standardized relative to the initial moment.

[0043] In this embodiment, it is assumed that in a two-dimensional plane coordinate system, for each agent's trajectory, its initial position (for example, initial coordinates ( )) as the origin, and relatively transform the coordinates of the subsequent trajectory points. At the same time, consider the initial orientation of the intelligent body (which can be expressed as an angle, such as the angle between the initial orientation of the vehicle head and the positive direction of the axis ), the coordinates of the trajectory points are rotated and translated according to this initial orientation, so that the trajectories of all agents seem to start from the same position and orientation. The advantage of this is that when the subsequent trajectory increments are calculated, quantized, and discretized, the trajectory changes between different agents are comparable, and there will be no deviation in data processing due to differences in initial positions and orientations. For example, in a multi-vehicle autonomous driving scenario, different vehicles may start at different positions and directions, but after normalization, their trajectories can be analyzed and processed in a unified manner, providing a consistent basis for accurately constructing a motion marker vocabulary. This step ensures that the trajectories of all agents start from a unified reference point, laying the foundation for subsequent quantization and discretization operations.

[0044] S220: Discretize the incremental actions of the trajectory into a series of predefined discrete values, and construct a discretized action vocabulary by uniformly quantizing each coordinate axis.

[0045] In this embodiment, a certain quantization interval is set for the incremental action of the trajectory on each coordinate axis (such as the x-axis and the y-axis) (i.e., the coordinate difference Δx and Δy between two adjacent trajectory points on the coordinate axis). For example, for the increment in the x-axis direction, if the quantization interval is set to [-5,3), [-3,-1), [-1,1), [1,3), [3,5], then when a trajectory increment Δx=2, it will be mapped to a discrete value 3 (corresponding to the interval [1,3)). Similarly, a similar quantization operation is performed on the y-axis. In this way, the possible continuous incremental values ​​on each coordinate axis are converted into a finite number of discrete values, and then these discrete values ​​are combined to construct a discretized action vocabulary. For example, in the above example, if the quantization result of the y-axis is a discrete value 2, then a combination (3,2) can be used to represent the incremental action of this trajectory point, and this combination becomes an element in the action vocabulary. In this way, the originally continuously changing trajectory increments can be represented by a finite number of discrete combinations (markers), so that the trajectory motion can be described in a more concise and regular way, which facilitates the subsequent trajectory modeling and prediction.

[0046] S230: Map each continuous trajectory increment to a corresponding index, and convert the continuous action of the trajectory into a discrete tag sequence.

[0047] In this embodiment, based on the discretized action vocabulary constructed in the above steps, a unique index is assigned to each possible discrete value combination (label). For example, in the case where (3,2) is used to represent the trajectory increment mentioned above, if this combination is defined as the 5th element in the action vocabulary, then its corresponding index is 5. For the entire trajectory, the incremental action of each trajectory point is mapped to an index in this way, and an index sequence is obtained, which is a discrete label sequence. For example, the trajectory of an agent consists of a series of trajectory points. After the incremental action of each trajectory point is quantized and index mapped, an index sequence such as [2,4,1,3,…] is obtained. This sequence represents the trajectory movement of the agent, and the continuous action of the trajectory is converted into a discrete label sequence that can be processed by the model in a concise and orderly manner. In the subsequent model input, this label sequence can be used as input data, and the model can learn and predict the future trajectory of the agent based on this sequence, because this sequence contains the pattern and regularity information of the trajectory movement of the agent.

[0048] In this embodiment, in order to improve the accuracy of trajectory reconstruction, a greedy search strategy is used to select the quantized action that can minimize the error at each time step. In order to further simplify dynamic modeling, the Verlet step method is used, in which when the action is zero, it means that the current increment is the same as the previous step, which helps to reduce the size of the vocabulary space and maintain the smoothness of speed changes.

[0049] S240: Merge the incremental motion of each coordinate into a single index value as a motion identifier. This reduces the representation complexity of the model. In the previous steps, the incremental motion of the trajectory on each coordinate axis (such as the x-axis and the y-axis) has been discretized and indexed. In step S240, the indexes corresponding to the incremental motion of each coordinate are merged. For example, assuming that for a certain trajectory point, the index after the x-axis incremental motion is mapped is , the index after the y-axis incremental action is mapped to , these two indexes can be merged into a single index value i by using some predetermined merging rule. A simple way to merge is to concatenate the two indexes (such as i=i x× N+i y , where N is the number of discrete values ​​on the y-axis, so that the merged index value is unique), or other encoding methods can be used as long as the information of the two dimensions can be effectively integrated into one value.

[0050] Through this merging operation, the model that originally needed to process two coordinate direction indexes separately now only needs to process a single index value. For the entire trajectory, the incremental actions of all trajectory points are merged into a single index value sequence in this way, which greatly reduces the dimension and complexity of the model input data. For example, in a scene containing multiple agent trajectories, if the merging is not performed, the model needs to process two index sequences of the x-axis and y-axis of each agent trajectory, with a large amount of data and a relatively complex structure; after the merging, the model only needs to process a single index sequence, which is easier and faster to process, can improve the training and prediction speed of the model, and at the same time reduce the demand for storage resources, which helps to optimize the performance of the entire autonomous driving trajectory prediction and planning system.

[0051] In one embodiment of the present invention, see Figure 3 , S300 specifically includes: S310: receiving scene initial state information through an encoder and generating a shared scene embedding; S320: Based on the output of the previous moment and the fixed scene embedding, predict the behavior of all agents at the current time step; S330: The decoder generates a distribution of N output identifiers for the number of agents in the scene by inputting a set of motion identifiers and scene embeddings in each prediction step.

[0052] In one embodiment of the present invention, see Figure 4 , S330 further includes: S3310: Divide the decoder into multiple layers, each layer applies a self-attention mechanism between input identifiers and a cross-attention mechanism to the scene embedding; S3320: Set all N motion identifiers in step t to follow each other and to follow all previous identifiers, where each row represents a query identifier, each column represents a key identifier, and a green block represents a key identifier that the query can follow; S3330: In operation After prediction steps, we get , the output motion identifiers form the complete trajectory of N agents.

[0053] In a specific embodiment of the present invention, the temporal autoregressive trajectory generation model proposed in the present invention is based on the Transformer architecture and uses autoregression to predict trajectories. The encoder of the model receives a set of identifiers (tokens) representing the initial state of the scene and generates a shared scene embedding. These initial states include the state of traffic lights, map topology information, and historical trajectory data of all traffic participants. During the model reasoning process, the decoder predicts the behavior of all agents at the current time step based on the output of the previous moment and the fixed scene embedding in an autoregressive manner. In each prediction step, the decoder generates a distribution of N output identifiers by inputting a set of motion identifiers and scene embeddings.

[0054] At each prediction step, the decoder receives as input a set of motion identifiers and scene embeddings and produces a distribution over N output identifiers. The decoder consists of multiple layers, each of which applies self-attention between the input identifiers and cross-attention to the scene embeddings. All N motion tokens at step t can attend to each other and to all previous tokens, where each row represents a query token and each column represents a key identifier. Green blocks represent key identifiers that the query can attend to. While running After prediction steps, we get , output motion tokens to form the complete trajectory of N agents. This autoregressive approach ensures that each agent’s actions are based on temporal causal relationships with the previous actions of all traffic participants, thereby improving the modeling of interactions between agents within the prediction time horizon.

[0055] Compared with traditional trajectory generation methods, the temporal autoregressive trajectory generation model of the present invention can accurately predict the trajectory of each agent in a scenario where multiple agents interact, and can fully consider the mutual influence between agents, thereby improving the prediction accuracy and robustness of the model. Through the autoregressive decoding process, the model can not only generate trajectories that conform to actual traffic behavior, but also better handle dynamically changing traffic environments, achieving more efficient trajectory planning and prediction.

[0056] In one embodiment of the present invention, see Figure 5 , S400 includes: S410: Model the complete trajectory generation of N agents as a multi-agent Markov decision (MDP) process.

[0057] In one embodiment of the present invention, the components of the multi-agent Markov decision-making include actions, states, dynamic transition processes and observations. Specifically, the functions of each component are as follows: Action: The action space of an agent is defined as the incremental action space. Each agent's action is represented by its acceleration in the X and Y directions at the current moment. In practice, these accelerations are discretized into a uniformly quantized grid, where each action is represented by a discrete index.

[0058] State: The state space is all the information that describes the agent and its surroundings. In this model, the state space includes map features, joint states of the agents, and traffic light states, etc.

[0059] Dynamic transfer process: The dynamic transfer matrix defines the rules for the agent to transfer from one state to the next. In this model, the state transfer of the agent is based on the current state (i.e. the current position and velocity of the agent) and the action taken (acceleration), and calculates the position of the agent at the next moment. This transfer process is based on the motion law of the physical model, and simulates the motion of the agent through discretized acceleration actions.

[0060] Observation: The observation space defines the information that the agent can perceive at each time step. In our method, observations include: historical context, historical actions of all objects, and the identity of the current agent. The historical context includes all objects, traffic light states, and map features of historical time steps, helping the agent understand the current scene and environment state.

[0061] S420: The reward function is designed to be whether each agent collides with the map boundary or other agents at each time step at each moment, and the calculation formula is: Where a and ß are the weight coefficients for collision with road boundaries and collision with other agents, respectively. Assume that the model autoregressively unfolds actions in T_{pred} time steps. For step t and each agent i, the reward is: Where is the reward of agent i at step t', γ is the discount factor used to weight future rewards; S430: Use the REINFORCE algorithm to calculate the policy gradient and perform reinforcement learning training to further optimize the decision-making ability of the model.

[0062] In the embodiment, the REINFORCE algorithm with baseline reduces the variance of policy updates by introducing an advantage function to accelerate the learning process. While training the model, a state value function network is trained to evaluate the value of the current state and used as a baseline to calculate the advantage function, which indicates the advantage of the agent taking a certain action in this state. , and use the advantage function to calculate the advantage of the agent taking a certain action in this state: in The agent is in state Take action The action value of Yes Status On this basis, the REINFORCE algorithm is used to calculate the policy gradient by maximizing the following objective function: in, is the action chosen by agent i at time step t strategy, It is the advantage function, which indicates the advantage of the agent taking a certain action at present.

[0063] Based on the calculated gradient information, the policy network is updated to gradually optimize the agent's behavior decisions and improve the safety and efficiency of trajectory planning.

[0064] By using the reinforcement learning fine-tuning algorithm, our model can fine-tune through reinforcement learning based on the autoregressive generated trajectory, learn how to avoid collisions with boundaries or other intelligent agents, and dynamically adjust trajectory planning according to changes in the environment, thereby enhancing the robustness and adaptability of the model.

[0065] Table 1: Test results of the method of the present invention on the Google Autonomous Driving Open Dataset (Waymo Open Dataset)

[0066] The experimental results in Table 1 show that in the prediction task, the method provided by this application can effectively generate the trajectories of all agents in the scene, and the minimum average distance error of trajectory prediction is 1.39 meters. In the task of simulating human driver behavior, the model provided by this application can simulate the interactive movement of vehicles, which is consistent with the characteristics of human drivers. In the planning task, the model fine-tuned by reinforcement learning has a 60% decrease in the collision rate of agent trajectories compared to the basic model, significantly reducing the risk of agent collision.

[0067] See also Figure 6 , Figure 6 An automatic driving trajectory prediction and planning device provided by the present invention is shown, which is used to implement the method described in any of the above embodiments, and the device includes: An acquisition module 100, which is used to acquire data of the current vehicle agent and pre-process the data; A construction module 200, the construction module is used to discretize the agent trajectory and construct a motion label vocabulary library; Prediction module 300, which inputs the agent trajectory and map information into a temporal autoregressive trajectory generation model based on the Transformer architecture and performs trajectory prediction using an autoregressive method; A fine-tuning module 400, which uses a reinforcement learning method to fine-tune the model; A generation module 500 is used to generate the future trajectory of the intelligent agent.

[0068] After the above modules, the autoregressive prediction results can be generated for the driving scene. Please refer to Figure 7 , Figure 7 It shows that at a complex intersection, an automatic driving trajectory prediction and planning device provided by an embodiment of the present invention uses autoregression to generate the future trajectory of an intelligent agent in time series, wherein the arrows on each line represent the future trajectory points of the intelligent agent with different numbers and the corresponding direction predictions.

[0069] See also Figure 8 , Figure 8 A schematic diagram of an electronic device 20 that can implement an embodiment of the present invention is shown, and the electronic device is intended to represent various forms of control devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0070] The electronic device 20 includes at least one processor 21, and a memory connected to the at least one processor 21, such as a read-only memory (ROM) 22, a random access memory (RAM) 23, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 22 or the computer program loaded from the storage unit 28 to the random access memory (RAM) 13. In the RAM 23, various programs and data required for the operation of the electronic device 20 can also be stored. The processor 21, ROM 22 and RAM 23 are connected to each other through a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0071] A number of components in the electronic device 20 are connected to the I / O interface 25, including: an input unit 26, such as a keyboard, a mouse, etc.; an output unit 27, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 29, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0072] The processor 21 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 21 performs the various methods and processes described above.

[0073] In some embodiments, the method of the above embodiment may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 20 via the ROM 22 and / or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the processor 21, one or more steps in the method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the method of the above embodiment in any other appropriate manner (e.g., by means of firmware).

[0074] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0075] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0076] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0077] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0078] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0079] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client and server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0080] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An autonomous driving trajectory prediction and planning method based on a time series autoregressive model, characterized in that: The method comprises: Obtain the data of the current vehicle agent and pre-process the data; Discretize the agent’s trajectory and build a motion labeling vocabulary; The agent trajectory and map information are input into the temporal autoregressive trajectory generation model based on the Transformer architecture and the trajectory is predicted using the autoregressive method; Fine-tune the model using reinforcement learning methods; Generate future trajectories for the agent.

2. The automatic driving trajectory prediction and planning method based on the time series autoregressive model according to claim 1 is characterized in that: The data of the current vehicle agent is obtained at least including: agent trajectory information in the scene, map lane centerline information, map lane boundary line information and traffic light information.

3. The automatic driving trajectory prediction and planning method based on the time series autoregressive model according to claim 2 is characterized by: Discretizing the multi-agent trajectory and constructing a motion tag vocabulary library includes: By normalizing the trajectory of each agent, its position and orientation are standardized relative to the initial moment; Discretize the incremental actions of the trajectory into a series of predefined discrete values, and construct a discretized action vocabulary by uniformly quantizing each coordinate axis; Map each consecutive trajectory increment to a corresponding index, converting the continuous action of the trajectory into a discrete tag sequence; Combine the incremental motion of each coordinate into a single index value as the motion identifier.

4. The automatic driving trajectory prediction and planning method based on the time series autoregressive model according to claim 3 is characterized by: The step of inputting the intelligent agent trajectory and map information into a temporal autoregressive trajectory generation model based on a Transformer architecture and using an autoregressive method to perform trajectory prediction includes: Receive the scene initial state information through the encoder and generate a shared scene embedding; Based on the output of the previous moment and the fixed scene embedding, predict the behavior of all agents in the current time step; The decoder takes as input a set of motion identifiers and a scene embedding at each prediction step and generates a distribution over N output identifiers for the number of agents in the scene.

5. The automatic driving trajectory prediction and planning method based on the time series autoregressive model according to claim 4 is characterized in that: The decoder generates a distribution of N output identifiers for the number of agents in the scene by inputting a set of motion identifiers and scene embeddings at each prediction step, including: The decoder is divided into multiple layers, each of which applies a self-attention mechanism between input identifiers and a cross-attention mechanism on the scene embedding; Set all N motion identifiers in step t to follow each other and all previous identifiers, where each row represents a query identifier, each column represents a key identifier, and the green block represents the key identifier that the query can follow; In operation After prediction steps, we get , the output motion identifiers form the complete trajectory of N agents.

6. The automatic driving trajectory prediction and planning method based on the time series autoregressive model according to claim 1, characterized in that: The method of fine-tuning the model using the reinforcement learning method includes: Model the complete trajectory generation of N agents as a multi-agent Markov decision process; The reward function is designed to be whether each agent collides with the map boundary or other agents at each time step at each moment. The calculation formula is: Where a and ß are the weight coefficients for collision with road boundaries and collision with other agents, respectively. Assume that the model autoregressively unfolds actions in T_{pred} time steps. For step t and each agent i, the reward is: Where is the reward of agent i at step t', γ is the discount factor used to weight future rewards; Use the REINFORCE algorithm to calculate the policy gradient and perform reinforcement learning training.

7. An automatic driving trajectory prediction and planning device, used to implement the method according to any one of claims 1 to 6, characterized in that: The device comprises: An acquisition module, which is used to acquire data of the current vehicle agent and pre-process the data; A construction module, the construction module is used to discretize the intelligent body trajectory and build a motion label vocabulary library; A prediction module, which inputs the agent trajectory and map information into a temporal autoregressive trajectory generation model based on the Transformer architecture and uses an autoregressive method to perform trajectory prediction; A fine-tuning module, wherein the fine-tuning module uses a reinforcement learning method to fine-tune the model; A generation module is used to generate the future trajectory of the intelligent agent.

8. An electronic device, characterized in that: The method comprises at least one processor, wherein the processor is communicatively connected to at least one memory, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 6 when the computer instructions are executed.

Images