Trajectory prediction method and device based on multi-agent interactive perception
By employing a multi-agent interactive perception method and utilizing importance-weighted features and local map information, trajectory prediction is optimized, addressing the issues of high computational cost and low accuracy in existing methods, and achieving more accurate and efficient trajectory prediction.
Patent Information
- Application Number
- CN202511031123.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-25
- Publication Date
- 2025-11-11
AI Technical Summary
Existing deep learning-based trajectory prediction methods cannot effectively utilize future information, cannot distinguish the importance of historical trajectories, have high computational requirements, and are difficult to quickly reflect changes in interactions among traffic participants, resulting in low accuracy in trajectory prediction.
A multi-agent interactive perception method is adopted. By encoding the historical trajectory information of each agent, importance-weighted features are obtained. Local map features are encoded by combining lane and pedestrian crossing information. Interaction features between agents and the local map are extracted. RetNet and Transformer are used for feature stitching and decoding to optimize trajectory prediction.
It improves the accuracy and efficiency of trajectory prediction, can quickly respond to sudden changes in traffic participants, predict potential obstacles, adapt to complex and ever-changing dynamic scenarios, and provide reliable trajectory prediction results.
Smart Images
Figure CN120922169A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a trajectory prediction method and device based on multi-agent interactive perception. Background Technology
[0002] Because autonomous driving technology has long-term advantages in solving traffic problems such as safety, congestion, and energy conservation, it is attracting increasing attention from academia and industry. To further improve safety, the key to autonomous driving technology lies in enabling autonomous vehicles to predict the future state of their surroundings in real time, just like human drivers. However, due to the significant differences in the mobility of traffic participants (i.e., intelligent agents, such as vehicles and pedestrians), the complex interactions between traffic participants and their surroundings, the uncertainty of perception information, and the high computational demands and time requirements, accurately predicting the future trajectories of traffic participants (i.e., intelligent agents) has become the most critical issue in improving the safety of autonomous driving.
[0003] Currently, the most commonly used trajectory prediction method is the deep learning-based trajectory prediction method. However, the existing deep learning-based trajectory prediction methods mainly have the following problems: (1) They can only use the information of the current moment and the previous moment of the vehicle to update the current state. They cannot use the information that appears later in the sequence in advance, and the feature representation of the agent is not comprehensive enough, resulting in low accuracy; (2) They cannot distinguish the importance of historical trajectories from the large amount of information generated by the frequent interaction between traffic participants during the driving process, which makes it impossible for them to react quickly to sudden changes in direction or unexpected obstacles; (3) They use the Transformer-based multi-head self-attention mechanism to predict the future trajectory of the agent. The Transformer has a complex structure and a large model scale, which requires a large amount of computation. However, the computing resources of autonomous vehicles are limited, which is not conducive to the computation of autonomous vehicles. Summary of the Invention
[0004] The purpose of this invention is to provide a trajectory prediction method and apparatus based on multi-agent interactive perception, which can predict potential obstacles or dangers and improve the accuracy of trajectory prediction.
[0005] In a first aspect, the present invention provides a trajectory prediction method based on multi-agent interactive perception, comprising:
[0006] The historical trajectory information of each agent is encoded to obtain the importance-weighted features of each agent.
[0007] The lane information and pedestrian crossing information around each intelligent agent are encoded to obtain local map features around each intelligent agent.
[0008] Based on the importance-weighted features of each agent and the local map features surrounding each agent, the interaction features R between agents are extracted. A2A The interaction features between the agent and the local map R A2M .
[0009] The interaction characteristics between agents R A2A The interaction features between the agent and the local map R A2M The features are concatenated to obtain the final feature vector.
[0010] The final feature vector is decoded to obtain the future trajectory of each agent (i.e., trajectory prediction result).
[0011] Preferred methods for obtaining the importance-weighted features of each agent include:
[0012] BiGRU is used to read the historical trajectory information of each agent in the input forward and backward directions, and then outputs the hidden state h that connects the forward and backward directions. t The hidden state h t The data is passed to a fully connected layer to obtain the agent's features.
[0013] Using the formula: The attention weight ω is calculated.
[0014] The importance vector I is calculated using the formula: I = softmax(ω).
[0015] Using the formula: The importance-weighted features of each agent were calculated.
[0016] Using BiGRU, historical trajectory information of agents can be read bidirectionally and fused to construct a more comprehensive and continuous state representation, which is more conducive to improving the accuracy of trajectory prediction. First, attention weights are calculated, then importance vectors (representing the degree of importance) are calculated, and finally, the importance-weighted features of each agent are calculated. This process classifies different information by importance and distinguishes the importance of historical trajectories.
[0017] Preferred methods for obtaining local map features around each agent include:
[0018] The lane information around each agent is encoded using MLP to obtain the lane features L around each agent.
[0019] The pedestrian crossing information around each agent is encoded using MLP to obtain the pedestrian crossing features C around each agent.
[0020] The lane features L and pedestrian crossing features C around each agent are concatenated to form local map features around each agent.
[0021] Encoding lane and pedestrian crossing information separately using MLP allows for the independent capture of features from both types of information. This independent processing avoids mutual interference and better adapts to the characteristics and patterns of each data point, resulting in more accurate feature extraction for each type of information. Concatenating lane features (L) and pedestrian crossing features (C) forms a local map feature. These features reflect the environmental information surrounding the agent from the perspectives of road structure and pedestrian crossing facilities, respectively. They complement each other, providing a more comprehensive description of the local map environment in which the agent is located. This helps to more accurately understand the agent's motion constraints and possible behavioral patterns during prediction, improving the accuracy of trajectory prediction. Furthermore, using MLP to encode lane and pedestrian crossing information is relatively simple in structure, computationally efficient, and easy to implement and optimize.
[0022] Lane information around a given agent includes traffic light type, lane type, stop sign, lane position, lane direction, and speed limit.
[0023] Preferred methods for obtaining lane features L around a given agent include:
[0024] The Embedding layer is used to encode the traffic light type, lane type, and stop sign to obtain the discrete feature vector of the lane.
[0025] The lane position, lane direction, and speed limit are encoded using an MLP to obtain a continuous feature vector of the lane.
[0026] The discrete feature vectors and continuous feature vectors of the lanes are concatenated and then encoded based on MLP to obtain the lane features L around the agent.
[0027] It comprehensively considers discrete information such as traffic light type, lane type, and stop signs, as well as continuous information such as lane position, direction, and speed limit. By encoding different types of information through an embedding layer and an MLP (Multi-Level Processing), it can comprehensively capture various key features of the lane, providing rich foundational information for subsequent feature fusion. First, discrete and continuous feature vectors are extracted separately, then concatenated, and finally encoded using an MLP. This hierarchical fusion method effectively integrates different types of information to form a more comprehensive feature representation. Subsequent MLP encoding further explores the potential relationships between these concatenated features, providing stronger support for trajectory prediction.
[0028] Preferably, the interaction features R between agents are extracted. A2A The interaction features between the agent and the local map RA2M One method includes:
[0029] The relationships between the agents are set as a fully connected graph G = (V, E); where G represents the interaction relationships between the agents, V represents the set of agents, and E represents the mutual influence relationships between the agents. E = 1 when there is mutual influence between agents, and E = 0 when there is no mutual influence between agents. The features of each agent are the importance-weighted features. Specifically, RetNet is used to set the relationships between the agents as a fully connected graph G = (V, E).
[0030] Processing the fully connected graph yields the interaction features R between agents. A2A Specifically, the Transformer is used to process the fully connected graph to obtain the interaction features R between agents. A2A .
[0031] The interaction characteristics between agents R A2A The interaction features R between the agents and the local map around each agent are processed to obtain the interaction features R between the agents and the local map. A2M Specifically, it involves using RetNet to analyze the interaction features between agents, R... A2A The interaction features R between the agents and the local map around each agent are processed to obtain the interaction features R between the agents and the local map. A2M .
[0032] By setting the relationships between agents as a fully connected graph, the interaction relationships between agents are clearly represented. The features of each agent employ the aforementioned importance-weighted features, which fully consider the importance and characteristics of each agent. Applying this feature to the construction of the fully connected graph makes the interaction features R between agents more explicit. A2A This approach can more accurately reflect the mutual influence between agents based on differences in importance. Processing fully connected graphs allows for deeper exploration of the complex dynamic relationships between agents, providing crucial information for predicting behavioral patterns. This is particularly relevant in extracting the interaction features R between agents. A2A Based on this, further combining the local map features around each agent to obtain R A2M This approach fully considers that agents are not only influenced by other agents, but also constrained and guided by the local map. This information is crucial for accurately predicting the agent's future trajectory.
[0033] In addition, RetNet (with its multi-scale preservation mechanism) is mainly used for interaction feature extraction and concatenation (only when the interaction features between agents, R, are obtained). A2AThe algorithm uses Transformer to make it more lightweight, shorten prediction time, improve prediction efficiency, and ensure prediction accuracy.
[0034] Preferably, the interaction features R between agents are extracted. A2A The interaction features between the agent and the local map R A2M Another method includes:
[0035] The relationships between the agents are set as a fully connected graph G = (V, E); where G represents the interaction relationships between the agents, V represents the set of agents, and E represents the mutual influence relationships between the agents. E = 1 when there is mutual influence between agents, and E = 0 when there is no mutual influence between agents. The features of each agent are the importance-weighted features. Specifically, RetNet is used to set the relationships between the agents as a fully connected graph G = (V, E).
[0036] Processing the fully connected graph yields the interaction features R between agents. A2A Specifically, the Transformer is used to process the fully connected graph to obtain the interaction features R between agents. A2A .
[0037] The interaction characteristics between agents R A2A The lane feature L is concatenated with the lane feature L and then encoded to obtain the interaction features R between each agent and the lane. A2L Specifically, this involves using RetNet to analyze the interaction features between agents, R... A2A The lane feature L is concatenated with the lane feature L and then encoded to obtain the interaction features R between each agent and the lane. A2L .
[0038] The interaction characteristics between agents R A2A The features C of the pedestrian crossing are concatenated with the features C of the pedestrian crossing and then encoded to obtain the interaction features R between each agent and the pedestrian crossing. A2C Specifically, this involves using RetNet to analyze the interaction features between agents, R... A2A The features C of the pedestrian crossing are concatenated with the features C of the pedestrian crossing and then encoded to obtain the interaction features R between each agent and the pedestrian crossing. A2C .
[0039] The interaction features R between each agent and the lane A2L The interaction features between each intelligent agent and the pedestrian crossing R A2C The data is then concatenated and encoded to obtain the interaction features R between the agent and the local map. A2M Specifically, RetNet is used to analyze the interaction features R between each agent and the lane. A2LThe interaction features between each intelligent agent and the pedestrian crossing R A2C The data is then concatenated and encoded to obtain the interaction features R between the agent and the local map. A2M .
[0040] Extracting the interaction features between agents R A2A Based on this, R will be respectively A2A The interaction features R between each agent and the lane are concatenated with and encoded from the lane feature L and the pedestrian crossing feature C. A2L The interaction features between each intelligent agent and the pedestrian crossing R A2C Finally, the fusion yields R. A2M This layered approach meticulously models the interactions between the agent and different types of map elements (i.e., lanes, pedestrian crossings), more accurately and comprehensively reflecting the complex interactive relationships between the agent and the local map.
[0041] Furthermore, using RetNet for feature concatenation and encoding can effectively process long sequence feature data, maintain the integrity and stability of information during transmission, prevent overfitting, and improve the generalization ability and robustness of feature representations, thus enhancing R... A2L R A2C and R A2M It more reliably reflects the interaction between the agent and map elements.
[0042] Preferably, RetNet is used to analyze the interaction features between agents, R... A2A The interaction features between the agent and the local map R A2M The features are concatenated to obtain the final feature vector. The MLP is then used to decode the final feature vector to obtain the future trajectories of each agent (i.e., trajectory prediction results).
[0043] Secondly, the present invention provides a trajectory prediction device based on multi-agent interactive perception, comprising:
[0044] The agent encoder is used to encode the historical trajectory information of each agent to obtain the importance-weighted features of each agent.
[0045] The map encoder is used to encode the lane information and pedestrian crossing information around each agent to obtain local map features around each agent.
[0046] The interaction module is used to extract interaction features R between agents based on the importance-weighted features of each agent and the local map features around each agent. A2A The interaction features between the agent and the local map R A2M ; and the interaction features R between agents A2A The interaction features between the agent and the local map R A2MThe features are concatenated to obtain the final feature vector.
[0047] The decoder is used to decode the final feature vector to obtain the future trajectory (i.e., trajectory prediction result) of each agent.
[0048] The present invention has the following effects:
[0049] (1) By encoding the historical trajectory information of each agent, the importance weighted features are obtained. This fully explores the key characteristics of each agent in the past movement process. Encoding these key characteristics and assigning importance weights can accurately reflect the movement habits and behavior patterns of the agents. Then, based on the importance weighted features, subsequent processing is carried out, which improves the accuracy of subsequent trajectory prediction.
[0050] (2) The importance weighted features of each agent reflect the importance division of different information during encoding, distinguishing the importance of historical trajectories, so that the agent can cope with sudden changes in direction, predict potential obstacles or dangers, and make rapid responses.
[0051] (3) Encoding lane and pedestrian crossing information around each agent yields local map features, reflecting a thorough consideration of the agent's environment. Lane information determines the agent's possible driving path and direction selection. Pedestrian crossing information is particularly important for scenarios involving interactions between various types of agents, such as pedestrians and vehicles. By encoding local map features, different road layouts and traffic rules can be adapted during trajectory prediction, improving the accuracy and rationality of the prediction.
[0052] (4) In complex scenarios, the interaction between agents (the interaction features between agents) is one of the key factors affecting the trajectory. The interaction features between the agent and the local map take into account how the agent adjusts its trajectory based on the map information. By concatenating these two interaction features, the final feature vector is obtained, which comprehensively considers the influence of various interaction factors on trajectory prediction, thereby improving the accuracy of trajectory prediction.
[0053] (5) The final feature vector not only considers the agent's own historical behavior, but also combines environmental factors and complex interaction relationships. This comprehensive processing method can better adapt to complex and ever-changing dynamic scenarios, accurately predict the future motion trajectory of each agent, and provide a reliable basis for subsequent decision-making (such as path planning, collision avoidance, etc.). Attached Figure Description
[0054] Figure 1 This is an architecture diagram of a trajectory prediction device based on multi-agent interactive perception in an embodiment of the present invention.
[0055] Figure 2This is a flowchart of the trajectory prediction method based on multi-agent interactive perception in an embodiment of the present invention.
[0056] Figure 3 This is a flowchart illustrating the method for obtaining the importance-weighted features of each agent in an embodiment of the present invention.
[0057] Figure 4 This is a flowchart of a method for obtaining local map features around each agent in an embodiment of the present invention.
[0058] Figure 5 Extracting R in the embodiments of the present invention A2A and R A2M A flowchart of one method.
[0059] Figure 6 Extracting R in the embodiments of the present invention A2A and R A2M Another method flowchart. Detailed Implementation
[0060] To gain a more detailed understanding of the features and technical content of the embodiments of the present invention, the implementation of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of the present invention.
[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.
[0062] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0063] like Figure 1 As shown, the trajectory prediction device based on multi-agent interactive perception in this embodiment of the invention includes:
[0064] The agent encoder is used to encode the historical trajectory information of each agent to obtain the importance-weighted features of each agent.
[0065] A map encoder is used to encode lane information and pedestrian crossing information around each agent to obtain local map features around each agent. As an example, the map encoder includes a lane encoder (MLP-L) for processing lane information and a pedestrian crossing encoder (MLP-C) for processing pedestrian crossing information.
[0066] The interaction module is used to extract interaction features R between agents based on the importance-weighted features of each agent and the local map features around each agent. A2A The interaction features between the agent and the local map R A2M ; and the interaction features R between agents A2A The interaction features between the agent and the local map R A2M The features are concatenated to obtain the final feature vector.
[0067] The decoder is used to decode the final feature vector to obtain the future trajectory (i.e., trajectory prediction result) of each agent.
[0068] like Figure 2 As shown, the trajectory prediction method based on multi-agent interactive perception in this embodiment of the invention includes:
[0069] S1. Encode the historical trajectory information of each agent to obtain the importance-weighted features of each agent.
[0070] For each agent, the historical trajectory at time t is s. t s t ={x t ,y t ,θ t ,v t}. Among them, (x t ,y t ) represents the 2D position, θ t Let v be the heading angle. t For velocity. As an example, consider the 10 nearest agents around the vehicle (11 agents in total including the vehicle), whose observation data is stored in a fixed-size tensor, with missing agents padded with 0.
[0071] like Figure 3 As shown, in some embodiments, the method for obtaining the importance-weighted features of each agent includes:
[0072] S11. Use BiGRU to read the historical trajectory information of each agent in the forward and backward directions, and then output the hidden state h connecting the forward and backward directions. t The hidden state h t The data is passed to a fully connected layer to obtain the agent's features.
[0073] The agent encoder employs a bidirectional gated recurrent unit (BiGRU) to encode the historical trajectory information of each agent. Specifically, the BiGRU consists of two GRU networks, capable of reading input in both forward and backward directions. The forward GRU processes information from front to back, and its hidden state can be represented as... Then, the hidden state of the information processed by the GRU from back to front can be represented as follows: The output of BiGRU is obtained by connecting the hidden states of the forward and backward passes. This indicates that the hidden state h t The data is passed to a fully connected layer to obtain the agent's features.
[0074] S12. Using the formula: The attention weight ω is calculated. Here, attentionWeight() represents the attention weight calculation function.
[0075] S13. Using the formula: I = softmax(ω), calculate the importance vector I. Here, softmax() represents the normalized exponential function.
[0076] S14. Use formula: The importance-weighted features of each agent were calculated.
[0077] BiGRU uses an attention mechanism to distinguish the importance of an agent's historical trajectories. When making predictions, its task is to direct the model's attention to the most important parts of the input sequence, rather than considering the entire sequence. The attention mechanism automatically determines the importance of each part, assigning higher weights to more important parts and lower weights to less important parts.
[0078] S2. Encode the lane information and pedestrian crossing information around each intelligent agent to obtain the local map features around each intelligent agent.
[0079] like Figure 4 As shown, in some embodiments, methods for obtaining local map features around each agent include:
[0080] S21. Use MLP to encode the lane information around each agent to obtain the lane features L around each agent.
[0081] Lane information around a given agent includes traffic light type, lane type, stop sign, lane position, lane direction, and speed limit.
[0082] In some embodiments, a method for obtaining lane features L around an agent includes:
[0083] First, the Embedding layer is used to encode the traffic light type, lane type, and stop sign to obtain the discrete feature vector of the lane.
[0084] Then, the lane position, lane direction and speed limit are encoded using MLP to obtain the continuous feature vector of the lane.
[0085] Finally, the discrete and continuous feature vectors of the lane are concatenated and then encoded based on the MLP to obtain the lane features L around the agent. In other words, the discrete and continuous feature vectors of the lane are concatenated as input to the lane encoder (MLP-L), and the lane encoder (MLP-L) outputs the lane features L.
[0086] S22. Encode the pedestrian crossing information around each agent using an MLP to obtain the pedestrian crossing features C around each agent. That is, input the pedestrian crossing information into the pedestrian crossing encoder (MLP-C), and the pedestrian crossing encoder (MLP-C) outputs the pedestrian crossing features C.
[0087] S23. Connect the lane features L and pedestrian crossing features C around each intelligent agent to form local map features around each intelligent agent.
[0088] For each agent, a local map is built, including the lanes it may travel in and the surrounding pedestrian crossings. Lane (waypoint) features L include the position and orientation of the lane center and lane boundaries, as well as speed limits and traffic signals (such as traffic light status and stop signs). Pedestrian crossing (waypoint) features C include the positions of points surrounding the pedestrian crossing area. All agent and lane / pedestrian crossing position attributes are transformed into the vehicle's local coordinate system.
[0089] S3. Based on the importance-weighted features of each agent and the local map features around each agent, extract the interaction features between agents, R. A2A The interaction features between the agent and the local map R A2M .
[0090] like Figure 5 As shown, in some embodiments, the interaction features R between agents are extracted. A2A The interaction features between the agent and the local map R A2M One method includes:
[0091] S311. Using RetNet, the relationships between the agents are represented as a fully connected graph G = (V, E). Here, G is the graph, representing the interaction relationships between the agents; V is the set of vertices, representing the set of agents; and E is the set of edges, representing the mutual influence relationships between agents. E = 1 when agents influence each other, and E = 0 when agents do not influence each other. The features of each agent are the aforementioned importance-weighted features.
[0092] S312. Use Transformer to process the fully connected graph to obtain the interaction features R between agents. A2A .
[0093] S313. Utilizing RetNet to analyze the interaction features between agents. A2A The interaction features R between the agents and the local map around each agent are processed to obtain the interaction features R between the agents and the local map. A2M .
[0094] like Figure 6 As shown, in some embodiments, the interaction features R between agents are extracted. A2A The interaction features between the agent and the local map R A2M Another method includes:
[0095] S321. Using RetNet, the relationships between the agents are represented as a fully connected graph G = (V, E). Here, G is the graph, representing the interaction relationships between the agents; V is the set of vertices, representing the set of agents; and E is the set of edges, representing the mutual influence relationships between agents. E = 1 when agents influence each other, and E = 0 when agents do not influence each other. The features of each agent are the aforementioned importance-weighted features.
[0096] S322. Use Transformer to process the fully connected graph to obtain the interaction features R between agents. A2A Specifically: R A2A =Transformer(G).
[0097] S323. Utilize RetNet to analyze the interaction features between agents. A2A The lane feature L is concatenated with the lane feature L and then encoded to obtain the interaction features R between each agent and the lane. A2L Specifically: R A2L =RetNet(concat[R A2A ,L]). Among them, concat[,] means concatenation.
[0098] S324. Utilize RetNet to analyze the interaction features between agents. A2A The features C of the pedestrian crossing are concatenated with the features C of the pedestrian crossing and then encoded to obtain the interaction features R between each agent and the pedestrian crossing. A2C Specifically: R A2C =RetNet(concat[R A2A ,C]).
[0099] S325. Using RetNet to analyze the interaction features between each agent and the lane. A2L The interaction features between each intelligent agent and the pedestrian crossing R A2CThe data is then concatenated and encoded to obtain the interaction features R between the agent and the local map. A2M Specifically:
[0100] R A2M =RetNet(concat[R A2L ,R A2C ])
[0101] S4. The interaction features between agents R A2A The interaction features between the agent and the local map R A2M The features are concatenated to obtain the final feature vector.
[0102] Specifically, this involves using RetNet to analyze the interaction features between agents, R... A2A The interaction features between the agent and the local map R A2M Concatenate the vectors to obtain the final feature vector R = concat[R]. A2A ,R A2M ].
[0103] S5. Decode the final feature vector to obtain the future trajectory of each agent (i.e., trajectory prediction result).
[0104] Specifically, the MLP is used to decode the final feature vector R to obtain the future trajectories (i.e., trajectory prediction results) of each agent.
[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A trajectory prediction method based on multi-agent interactive perception, characterized in that, include: Encode the historical trajectory information of each agent to obtain the importance-weighted features of each agent; Encode the lane information and pedestrian crossing information around each intelligent agent to obtain local map features around each intelligent agent; Based on the importance-weighted features of each agent and the local map features surrounding each agent, the interaction features R between agents are extracted. A2A The interaction features between the agent and the local map R A2M ; The interaction characteristics between agents R A2A The interaction features between the agent and the local map R A2M Concatenate the vectors to obtain the final feature vector. The final feature vector is decoded to obtain the future trajectory of each agent.
2. The trajectory prediction method based on multi-agent interactive perception according to claim 1, characterized in that, Methods for obtaining importance-weighted features of each agent include: BiGRU is used to read the historical trajectory information of each agent in the input forward and backward directions, and then outputs the hidden state h that connects the forward and backward directions. t The hidden state h t The data is passed to a fully connected layer to obtain the agent's features. Using the formula: The attention weight ω is calculated. The importance vector I can be calculated using the formula: I = softmax(ω); Using the formula: The importance-weighted features of each agent were calculated.
3. The trajectory prediction method based on multi-agent interactive perception according to claim 1, characterized in that, Methods for obtaining local map features around each agent include: The lane information around each agent is encoded using MLP to obtain the lane features L around each agent; The pedestrian crossing information around each agent is encoded using MLP to obtain the pedestrian crossing features C around each agent; The lane features L and pedestrian crossing features C around each agent are concatenated to form local map features around each agent.
4. The trajectory prediction method based on multi-agent interactive perception according to claim 3, characterized in that: Lane information around a given agent includes traffic light type, lane type, stop sign, lane position, lane direction, and speed limit; Methods for obtaining lane features L around a given agent include: The Embedding layer is used to encode the type of traffic lights, lane types, and stop signs to obtain the discrete feature vector of the lanes; The lane position, lane direction, and speed limit are encoded using an MLP to obtain a continuous feature vector of the lane; The discrete feature vectors and continuous feature vectors of the lanes are concatenated and then encoded based on MLP to obtain the lane features L around the agent.
5. The trajectory prediction method based on multi-agent interactive perception according to claim 1, characterized in that, Extracting the interaction features between agents R A2A The interaction features between the agent and the local map R A2M The methods include: The relationships between the agents are set as a fully connected graph G = (V, E); where G represents the interaction relationship between the agents, V represents the set of agents, E represents the mutual influence between agents, E = 1 when there is mutual influence between agents, and E = 0 when there is no mutual influence between agents. The feature of each agent is the importance-weighted feature. Processing the fully connected graph yields the interaction features R between agents. A2A ; The interaction characteristics between agents R A2A The interaction features R between the agents and the local map around each agent are processed to obtain the interaction features R between the agents and the local map. A2M .
6. The trajectory prediction method based on multi-agent interactive perception according to claim 5, characterized in that: RetNet is used to set the relationships between the agents as a fully connected graph G = (V, E); By using Transformer to process the fully connected graph, we can obtain the interaction features R between agents. A2A ; Using RetNet to analyze the interaction features between agents R A2A The interaction features R between the agents and the local map around each agent are processed to obtain the interaction features R between the agents and the local map. A2M .
7. The trajectory prediction method based on multi-agent interactive perception according to claim 3, characterized in that, Extracting the interaction features between agents R A2A The interaction features between the agent and the local map R A2M The methods include: The relationships between the agents are set as a fully connected graph G = (V, E); where G represents the interaction relationship between the agents, V represents the set of agents, E represents the mutual influence between agents, E = 1 when there is mutual influence between agents, and E = 0 when there is no mutual influence between agents. The feature of each agent is the importance-weighted feature. Processing the fully connected graph yields the interaction features R between agents. A2A ; The interaction characteristics between agents R A2A The lane feature L is concatenated with the lane feature L and then encoded to obtain the interaction features R between each agent and the lane. A2L ; The interaction characteristics between agents R A2A The features C of the pedestrian crossing are concatenated with the features C of the pedestrian crossing and then encoded to obtain the interaction features R between each agent and the pedestrian crossing. A2C ; The interaction features R between each agent and the lane A2L The interaction features between each intelligent agent and the pedestrian crossing R A2C The data is then concatenated and encoded to obtain the interaction features R between the agent and the local map. A2M .
8. The trajectory prediction method based on multi-agent interactive perception according to claim 7, characterized in that: RetNet is used to set the relationships between the agents as a fully connected graph G = (V, E); By using Transformer to process the fully connected graph, we can obtain the interaction features R between agents. A2A ; Using RetNet to analyze the interaction features between agents R A2A The lane feature L is concatenated with the lane feature L and then encoded to obtain the interaction features R between each agent and the lane. A2L ; Using RetNet to analyze the interaction features between agents R A2A The features C of the pedestrian crossing are concatenated with the features C of the pedestrian crossing and then encoded to obtain the interaction features R between each agent and the pedestrian crossing. A2C ; Using RetNet to analyze the interaction features between each agent and the lane A2L The interaction features between each intelligent agent and the pedestrian crossing R A2C The data is then concatenated and encoded to obtain the interaction features R between the agent and the local map. A2M .
9. The trajectory prediction method based on multi-agent interactive perception according to any one of claims 1 to 8, characterized in that: Using RetNet to analyze the interaction features between agents R A2A The interaction features between the agent and the local map R A2M Concatenate the vectors to obtain the final feature vector. The MLP is used to decode the final feature vector to obtain the future trajectory of each agent.
10. A trajectory prediction device based on multi-agent interactive perception, characterized in that, include: The agent encoder is used to encode the historical trajectory information of each agent to obtain the importance-weighted features of each agent. The map encoder is used to encode the lane information and pedestrian crossing information around each intelligent agent to obtain local map features around each intelligent agent. The interaction module is used to extract interaction features R between agents based on the importance-weighted features of each agent and the local map features around each agent. A2A The interaction features between the agent and the local map R A2M ; and the interaction features R between agents A2A The interaction features between the agent and the local map R A2M Concatenate the vectors to obtain the final feature vector. The decoder is used to decode the final feature vector to obtain the future trajectory of each agent.