Intelligent agent trajectory prediction method

Abstract

The invention discloses an agent trajectory prediction method, which comprises the following steps of: according to a cross attention mechanism, coding each scene element in a scene map to obtain node embedding and edge embedding of each scene element and initial mode query embedding of an agent, and obtaining space embedding and time sequence embedding; furthermore, priori embedding and reverse learning embedding are obtained; further obtaining multi-feature mode query embedding; obtaining time embedding according to the constructed lane-time interaction diagram and the time step interaction diagram; processing the space-time embedding according to a double-current subtraction Mi-Transform, and executing an attention decomposition mechanism to obtain a proposal trajectory; and generating an anchor point according to the proposal trajectory, and processing the proposal trajectory according to the anchor point to obtain a final prediction trajectory. The method has good intention recognition accuracy in a complex environment, the uncertainty and collision risk of the prediction trajectory are low, and the prediction precision and robustness are high.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer simulation, and in particular relates to a method for predicting intelligent body trajectories. Background Art

[0002] With the rapid development of intelligent driving technology, predicting the interactive behavior of multiple agents (such as vehicles, pedestrians, and cyclists) in complex traffic scenarios has become one of the core key technologies for achieving high-level autonomous driving. In dynamic and open traffic environments, vehicles need to perceive the intentions of surrounding traffic participants in real time and predict their future motion trajectories to support path planning, collision avoidance decisions, and multi-vehicle coordinated control. However, due to the high uncertainty of actual traffic scenarios, the randomness of the interactive agent's behavior, and the strong coupling of spatiotemporal information, existing trajectory prediction methods still face technical bottlenecks in complex interactive scenarios. Summary of the Invention

[0003] In order to solve the above problems existing in the prior art, the present invention provides an intelligent agent trajectory prediction method.

[0004] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] In a first aspect, the present invention provides a method for predicting an intelligent agent trajectory, the method comprising:

[0006] Encoding each scene element in the scene map to obtain a node embedding and an edge embedding of each scene element; wherein the scene elements include lane segments and intelligent agents;

[0007] Obtaining spatial embedding and temporal embedding according to the cross-attention mechanism, the node embedding and edge embedding of each scene element, and the initial pattern query embedding of the agent; wherein the initial pattern query embedding represents the future motion pattern of the agent;

[0008] Obtaining a priori embedding based on the spatial embedding and the temporal embedding, and obtaining a reverse learning embedding based on the prior embedding; wherein the prior embedding is used to predict the behavior of the intelligent agent, and the reverse learning embedding is used to enhance the prediction performance;

[0009] Obtaining a multi-feature pattern query embedding according to the spatial embedding, the temporal embedding, and the reverse learning embedding;

[0010] Obtaining a temporal embedding based on a constructed lane-time interaction graph and a time-step interaction graph; wherein the lane-time interaction graph is constructed based on the node embedding and edge embedding of each scene element, and the time-step interaction graph is constructed based on the relative positions of the agents at different time steps;

[0011] Processing the spatiotemporal embedding according to a two-stream subtractive Mi-Transformer and performing a decomposed attention mechanism to obtain a proposed trajectory; wherein the spatiotemporal embedding is obtained from the multi-feature pattern query embedding and the temporal embedding;

[0012] Anchor points are generated according to the proposed trajectory, and the proposed trajectory is processed according to the anchor points to obtain a final predicted trajectory.

[0013] Optionally, encoding each scene element in the scene map to obtain a node embedding and an edge embedding of each scene element further includes:

[0014] Constructing a first node feature based on the lane segments and corresponding polylines; wherein each lane segment comprises a plurality of vectorized polylines;

[0015] constructing a first edge feature according to a mapping relationship between the lane segment and the polyline;

[0016] Obtaining a first node embedding and a first edge embedding according to a multilayer perceptron, the first node feature, and the first edge feature;

[0017] Obtaining a local representation of the lane segment according to the first node feature and the corresponding first edge feature;

[0018] Each lane segment is regarded as the second node, and the mapping relationship between each lane segment is regarded as the second edge;

[0019] Obtaining a second node feature and a second edge feature and encoding them according to the multi-layer perceptron to obtain a second node embedding and a second edge embedding;

[0020] A scene map embedding is obtained according to the graph attention network, the second node embedding, and the second edge embedding.

[0021] Optionally, encoding each scene element in the scene map to obtain a node embedding and an edge embedding of each scene element further includes:

[0022] Constructing a third node feature of the agent according to the speed, speed direction, shape, and category of the agent, and constructing a third edge feature according to the mapping relationship between the agents;

[0023] The third node embedding and the third edge embedding are obtained according to the multilayer perceptron, the third node feature, and the third edge feature.

[0024] Optionally, the calculation method of the prior embedding is expressed as follows:

[0025]

[0026] Among them, qfa represents the prior embedding, MHGCA(q,k,v) represents multi-head graph cross attention, q is the query, k is the key, v is the value, q ta Denotes the temporal embedding, E a represents the third node embedding, Indicates a third edge embedding.

[0027] Optionally, the calculation method of the reverse learning embedding is expressed as follows:

[0028]

[0029] Among them, q ia represents the reversely learned embedding.

[0030] Optionally, the time embedding is calculated as follows:

[0031] q t =q lt +q tt ;

[0032] Among them, q t represents the time embedding, q lt represents the dynamic interaction embedding between the scene map and the agent’s temporal state, q tt Represents the dynamic interaction embeddings between the temporal states of each agent.

[0033] Optionally, the dynamic interaction embedding between the scene map and the agent's temporal state is calculated as follows:

[0034]

[0035] Among them, q sa represents the spatial embedding, E m represents the second node embedding, Represents the edge embedding corresponding to the mapping relationship between the scene map and the agent.

[0036] Optionally, the calculation method of the dynamic interaction embedding between the temporal states of each agent is expressed as follows:

[0037]

[0038] in, Represents the edge embeddings corresponding to the mapping relationship between each agent.

[0039] Optionally, generating an anchor point according to the proposed trajectory, and processing the proposed trajectory according to the anchor point to obtain a final predicted trajectory includes:

[0040] Get the historical trajectory of the agent;

[0041] Encode the historical trajectory and the proposed trajectory respectively according to a multi-layer perceptron to obtain a historical trajectory embedding and a proposed trajectory embedding;

[0042] Extracting local correlation features of the historical trajectory embedding and the proposed trajectory embedding respectively through 2D-CNN to obtain historical trajectory features and proposed trajectory features;

[0043] Aligning the historical trajectory features and the proposed trajectory features and performing one-dimensional splicing to obtain multi-dimensional features;

[0044] Obtaining the anchor point according to the multidimensional feature;

[0045] Converting the proposed trajectory into a polar coordinate representation based on the anchor point to obtain a converted feature;

[0046] After encoding the converted features through the multi-layer perceptron, processing them according to the graph attention mechanism and the decomposition attention mechanism to obtain the trajectory offset;

[0047] The final predicted trajectory is obtained according to the trajectory offset and the proposed trajectory.

[0048] The technical solutions provided by the embodiments of the present invention may have the following beneficial effects:

[0049] In the above technical solution, the present invention combines the graph attention network with the multi-head cross-attention mechanism to effectively capture the complex relationship between agents and maps, and between agents and agents; through reverse learning embedding, historical intentions are inferred from a future perspective, significantly improving the accuracy of intention recognition; the dual-stream subtraction Mi-Transformer architecture is adopted to optimize the efficiency and stability of long sequence modeling, and pattern interaction modeling is realized in multiple dimensions through the decomposition attention mechanism; finally, the anchor point and trajectory optimization are combined to reduce the uncertainty and collision risk of the predicted trajectory, thereby achieving high-precision and high-robustness trajectory prediction, providing reliable behavior prediction support for autonomous driving and intelligent transportation systems.

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of an intelligent agent trajectory prediction method provided by an embodiment of the present invention;

[0052] Figure 2 This is an algorithm flow chart provided by an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of a predicted trajectory of a model without introducing time query coding provided by an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of a prediction trajectory of a model that introduces time query coding provided by an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of a prediction trajectory of a model without introducing reverse learning attention provided by an embodiment of the present invention;

[0056] Figure 6 Schematic diagram of a prediction trajectory of a model introducing reverse learning attention provided by an embodiment of the present invention;

[0057] Figure 7 This is a schematic diagram of a predicted trajectory of a model without dynamic anchor point selection provided by an embodiment of the present invention;

[0058] Figure 8 This is a schematic diagram of a predicted trajectory of a model that introduces dynamic anchor point selection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to facilitate the understanding of the present invention, the related art and the inventive concept of the present invention are briefly described first.

[0060] Existing trajectory prediction technologies mainly include:

[0061] (1) Trajectory prediction method based on interaction modeling: Modeling the interaction relationship between intelligent agents or between intelligent agents and the environment through mechanisms such as self-attention, cross-attention, or recurrent neural network (RNN). Representative models: Social-LSTM (Alahi A, Goel K, Ramanathan V, et al. Social lstm: Human trajectory prediction in crowded spaces [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 961-971.): Introducing a social pooling layer to capture the spatiotemporal dependencies between agents, but ignoring road topology information; AgentFormer (Yuan Y, Weng X, Ou Y, et al. Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2021: 9813-9823.): Separating the temporal and spatial dimensions through spatiotemporal encoders and agent-aware attention, but not explicitly modeling the spatiotemporal linkage of interactions; QCNet (Zhou, Zikang, et al. "Query-centric trajectory prediction." Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition.2023): proposes a query-centric encoding paradigm that has high prediction accuracy but still relies on static interaction modeling.

[0062] (2) Two-stage trajectory prediction method: The prediction process is divided into two stages: trajectory proposal and trajectory refinement, and multimodal performance is improved through iterative optimization. Representative models: HPNet (Tang X, Kan M, Shan S, et al. HPnet: Dynamic trajectory forecasting with historical prediction attention [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2024: 15261-15270.): Generate multimodal trajectories based on historical prediction attention and refine them through fixed-radius anchor points; SmartRefine (Zhou Y, Shao H, Wang L, et al. Smartrefine: A scenario-adaptive refinement framework for efficient motion prediction [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2024: 15281-15290.): Adaptively select anchor points and iteratively correct trajectories.

[0063] Existing trajectory prediction methods mainly rely on interaction modeling based on the attention mechanism, and their core problems are: (1) Insufficient static interaction modeling: Existing methods use the spatiotemporal attention mechanism to model the temporal continuity and global spatial association of trajectories separately, but lack explicit encoding of the dynamic spatiotemporal linkage between interacting entities. For example, in vehicle lane changes or intersection game scenarios, changes in the intentions of neighboring vehicles will trigger chain reactions, and traditional methods have difficulty capturing such dynamic interaction patterns. (2) Limitations of fixed anchor point strategies: Existing two-stage prediction frameworks select anchor points through fixed radius or statistical features to refine trajectories, but in complex interaction scenarios (such as multi-vehicle racing and pedestrian crossing), fixed anchor points are prone to lose key motion trends, resulting in insufficient trajectory correction capabilities. (3) Existing vectorized encoding methods fail to effectively integrate lane geometry, topological relationships and temporal dynamics, resulting in insufficient correlation between the agent's historical information and future behavior. Therefore, the present invention proposes an agent trajectory prediction method to solve this technical problem.

[0064] Figure 1 is a flow chart of an intelligent agent trajectory prediction method provided by an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0065] S101. Encode each scene element in the scene map to obtain a node embedding and an edge embedding of each scene element; wherein the scene elements include lane segments and intelligent agents.

[0066] Optionally, S101 may include:

[0067] Constructing a first node feature according to the lane segments and corresponding polylines; wherein each lane segment comprises a plurality of vectorized polylines;

[0068] Construct the first edge feature based on the mapping relationship between lane segments and polylines;

[0069] Obtaining a first node embedding and a first edge embedding according to the multilayer perceptron, the first node feature, and the first edge feature;

[0070] Obtaining a local representation of the lane segment according to the first node feature and the corresponding first edge feature;

[0071] Each lane segment is regarded as the second node, and the mapping relationship between each lane segment is regarded as the second edge;

[0072] Obtain the second node feature and the second edge feature and encode them according to the multi-layer perceptron to obtain the second node embedding and the second edge embedding;

[0073] The scene map embedding is obtained based on the graph attention network, the second node embedding and the second edge embedding.

[0074] For example, the scene map may contain G lane segments {l 1 ,…,l g} g∈[1,G] , each lane segment consists of O vectorized polylines {p 1 ,…,p o} o∈[1,O] The spatial state of each lane segment is represented by the position and direction of the midpoint of its centerline. The lane segment and the polyline can be used as the first node, and the first node feature is represented by the length embedding. The mapping relationship between the polyline and the lane segment is represented by the edge, and its first edge feature includes {d pl ,φ pl , γ pl}, where d pl represents the distance from the polyline to the lane segment, φ pl represents the direction of the edge in the local coordinate system of the lane segment, represents the relative direction between the polyline and the lane segment, γ plRepresents the geometric properties of the polyline. Then, a multi-layer perceptron (MLP) is used to obtain the first node embedding E pl =MLP(l G ,p O ) and the first edge embedding The first node feature and the first edge feature are fused through the graph cross attention network to obtain the local representation of the lane segment l m The lane segment is taken as the second node, and the second node feature is l m , the second edge features include {d m ,φ m , γ m}, where γ m Represent the attributes of the lane segment, including topological connectivity (such as predecessor, successor, and adjacency) and the number of multi-hops, and encode the second edge features to obtain the second edge embedding And apply the graph self-attention network to capture the topological relationship between lanes and the interaction between long-distance lanes. Finally, the scene map embedding is generated. in, Represents a real number field of G×D dimensions, where D represents the feature dimension.

[0075] Optionally, S101 may further include:

[0076] Construct the third node feature of the agent based on the speed, speed direction, shape and category of the agent, and construct the third edge feature based on the mapping relationship between the agents;

[0077] The third node embedding and the third edge embedding are obtained according to the multi-layer perceptron, the third node feature and the third edge feature.

[0078] For example, a local coordinate system is created for each agent at each timestamp, referring to its current position and orientation. The non-position information of the agent is used as the third node feature a t ={v a ,s a ,c a},in is the speed and direction of the agent, s a =(l a ,w a ) is the shape of the agent, including the length l a and width w a , c a is the agent category (such as vehicles and pedestrians). The third node is embedded as E a =MLP(v a ,s a ,c a ),in, The third side features include {da ,φ a , λ a}, where d a is the length of the side, φ a is the direction of the edge, is the relative direction between agents. a Only used for agent interactions involving time spans, representing timestamp intervals, using MLP to generate third-edge embeddings in, Y represents the number of edges.

[0079] S102. Based on the cross-attention mechanism, the node embedding and edge embedding of each scene element and the initial pattern query embedding of the agent, spatial embedding and temporal embedding are obtained; wherein the initial pattern query embedding represents the future motion pattern of the agent.

[0080] It is understandable that Figure 2 This is an algorithm flow chart provided by an embodiment of the present invention, refer to Figure 2 ,Step S102 designs the agent-map attention, the purpose of which is to allow the agent to extract key environmental constraint information. First, initialize the pattern query embedding for each agent Each embedding represents a possible future motion pattern. The initial pattern query embedding captures surrounding information based on the motion state of the corresponding agent and continuously updates itself during different interactions. The initial pattern query embedding and lane segments are modeled as target nodes and source nodes, respectively. At each time stamp, each target node will be moved to a fixed spatial radius R. m The source nodes around it are queried within the range and the contextual information of the lane segment is embedded through the cross-attention mechanism. The process is described as follows:

[0081]

[0082] Among them, q sa is a spatial embedding, MHGCA(q,k,v) represents multi-head graph cross attention, q is the query, k is the key, and v is the value. It is the edge embedding corresponding to the mapping relationship between the scene map and the agent in the agent-map attention.

[0083] The inverse learning attention is decomposed into three layers: ego-agent temporal attention, agent future attention, and agent historical attention.

[0084] In ego-agent temporal attention, for each agent at timestamp t, attention can be performed on the states of its historical timestamp interval [0, t-1]. First, the global position of the historical timestamps is converted to relative positions to obtain edge features: the length and direction of the edge, the relative direction between nodes, and the time interval. Then, each initial pattern query performs ego-agent temporal attention on all source nodes at each timestamp:

[0085]

[0086] Among them, q ta is temporal embedding, The edge embeddings representing the mapping relationships between agents in the ego-car agent’s temporal attention. This attention ensures that the model captures key historical information that is closely related to the current timestamp state.

[0087] S103. Obtain a priori embedding based on the spatial embedding and the temporal embedding, and obtain a reverse learning embedding based on the prior embedding; wherein the prior embedding is used to predict the behavior of the intelligent agent, and the reverse learning embedding is used to enhance the prediction performance.

[0088] It is worth mentioning that the reference Figure 2 , the agent's future attention first focuses on the future timestamp state of the interacting agent, models the interaction between the ego vehicle at the current timestamp and the interacting agent at the future timestamp, and thus obtains the prior embedding. The edge index is constructed based on the connection relationship between the ego vehicle at timestamp t∈[0,T-1] and the interacting agent at timestamp t+1. These connection relationships can be reused in the subsequent agent's historical attention. Edge features include the length and direction of the edge, and the relative direction between nodes. For each time series embedding q at each timestamp ta , in the search radius R f Query the source node in the visible future spacetime, and then perform agent future attention to obtain the prior embedding q fa :

[0089]

[0090] Among them, q fa represents the prior embedding, MHGCA(q,k,v) represents the multi-head graph cross attention, q is the query, k is the key, v is the value, q ta represents temporal embedding, E a represents the third node embedding, Represents the edge embeddings corresponding to the mapping relationship between agents in future attention.

[0091] The time span of the agent's historical attention is set from timestamp t-1 to t+1. On the one hand, this setting expands the window of reverse learning in the time dimension without affecting the subsequent agent interactions at the same moment, but combines them together to maintain the spatiotemporal consistency in intent understanding. On the other hand, this setting allows edges to be constructed by reversing the message passing direction of the agent's future attention. Then, each prior embedding q at timestamp t fa As a query, in radius R h Perform agent history attention on the interacting agent at timestamp t-1 to obtain the reverse learning embedding q representing the proposed intention ia :

[0092]

[0093] Among them, q ia represents the reverse learning embedding, Represents the edge embeddings corresponding to the mapping relationship between agents in historical attention.

[0094] This reverse learning mechanism enables the model to infer past behavior from the perspective of the future, thereby establishing closer dynamic connections between historical trajectories. This approach leads to better predictive performance than using only forward reasoning to acquire proposed intent.

[0095] S104. Obtain multi-feature pattern query embedding based on spatial embedding, temporal embedding, and reverse learning embedding.

[0096] For example, finally the spatial embedding, temporal embedding and reverse learning embedding are added together to obtain the multi-feature pattern query embedding q m :

[0097] q m =q sa +q ta +q ia .

[0098] S105. Obtain a time embedding based on the constructed lane-time interaction graph and time-step interaction graph; wherein the lane-time interaction graph is constructed based on the node embedding and edge embedding of each scene element, and the time-step interaction graph is constructed based on the relative positions of the agents at different time steps.

[0099] Understandably, the reference Figure 2The method combines spatiotemporal state interactions with temporal self-attention using temporal query encoding, enhancing the modeling of spatiotemporal historical dependencies. The lane-time interaction graph is constructed by computing valid edge masks between agents and lanes, filtering valid relationships using Euclidean distance, and learning the spatiotemporal dependencies between agents and lanes by combining the agent's position, orientation, and visibility mask. Features are extracted through local coordinate transformation to more accurately capture spatiotemporal information.

[0100] Time-step interactions are used to model the relative positions and behavioral dependencies of agents at different time steps, construct a time-step interaction graph, and filter invalid edges by distance and visibility.

[0101] The interaction radius between agents is defined as the dynamic radius. This dynamic radius is adaptively adjusted based on the average speed of the agents in the scene. First, the global average speed is calculated as follows:

[0102]

[0103] in, represents the global average speed of the agent. Then calculate the speed difference of each agent:

[0104]

[0105] in, represents the velocity difference of the i-th agent at time step t. The dynamic radius is calculated as follows:

[0106]

[0107] Among them, r i represents the dynamic radius of the i-th agent, α represents the speed factor weight, r min and r max Represent the minimum and maximum radius thresholds respectively; r ij represents the joint dynamic radius of agents i and j.

[0108] Edge features are calculated based on relative position and orientation differences, and then converted into temporal features through an embedding layer. Finally, the graph attention mechanism is used to fuse spatiotemporal information to generate high-quality spatiotemporal embeddings, effectively enhancing the model's temporal prediction capabilities. The calculation process is as follows:

[0109]

[0110] q t =q lt +q tt ;

[0111] Among them, q sa represents the spatial embedding, Em represents the second node embedding, Represents the edge embedding corresponding to the mapping relationship between the scene map and the agent in the lane-time interaction graph, represents the edge embedding corresponding to the mapping relationship between each agent in the time step interaction, q t represents the time embedding, q lt represents the dynamic interaction embedding between the scene map and the agent’s temporal state, q tt Represents the dynamic interaction embeddings between the temporal states of each agent.

[0112] S106. Process the spatiotemporal embedding according to the two-stream subtraction Mi-Transformer and perform a decomposition attention mechanism to obtain a proposed trajectory; wherein the spatiotemporal embedding is obtained based on the multi-feature pattern query embedding and the time embedding.

[0113] For example, we can get spatiotemporal embeddings based on multi-feature pattern query embeddings and time embeddings:

[0114] E q =q m +q t .

[0115] Understandably, the reference Figure 2 Traditional Transformer models suffer from high computational complexity and information loss when processing long sequences, making it difficult to effectively capture long-range dependencies. A two-stream subtractive Mi-Transformer algorithm can be introduced to learn residuals in each layer and add auxiliary output branches to each module, gradually achieving a decomposition of input and output. This approach aims to improve the bidirectional spatiotemporal information flow and enhance the interaction between future predictions and historical trajectories.

[0116] The input format of Mi-Transformer is [B,T h ,X'], the output format is [B,T f ,X'], where B represents the batch size, X' represents the number of variables, and the definition of T is the same as above. h Indicates the number of historical steps, T f Indicates the number of future steps. Since Transformer can only accept fixed-length input variables, in order to align with the input dimension of Mi-Transformer, the spatiotemporal embedding E q As input variable X. In order to capture the pattern interaction between different historical moments, the variable dimension is defined as K represents the number of modalities, and the number of agents N is taken as batch input.

[0117] Mi-Transformer performs inverse encoding on the input. Given an input The expression is as follows:

[0118]

[0119] in, Represents the values ​​of all variables at the same time step. The transposed variables are

[0120]

[0121] in, Each line Represents the value of a variable at all time steps. By replacing the "addition" in traditional residual connections with "subtraction", a new dynamic residual mechanism is constructed to alleviate the overfitting problem in time series forecasting and gradually optimize the model's learning effect on the target signal.

[0122] The backbone network of Mi-Transformer adopts a bifurcated structure. The variable in the main branch is defined as X i , the variables in the residual branch are defined as R i , whose layers are represented as follows:

[0123]

[0124] in, represents the features after attention, i represents the output of the lth layer, Attention(·) represents the self-attention calculation, and δ represents the Dirac function, which suppresses the influence of attention when the output is useless and only retains the key information.

[0125] Multiple layers of Mi-Transformer modules are stacked together, extracting features from both the main branch and the residual branch in each layer. Through this design, the algorithm focuses on learning the residual layer by layer. The final output of each Mi-Transformer layer is:

[0126]

[0127] The final output is sequentially subjected to a decomposed attention mechanism, including agent attention, history prediction attention, and pattern attention, to model interactions between pattern queries in different dimensions. First, agent attention is used to extract interactions between agents at the same timestamp. Subsequently, history prediction attention is applied to the ego vehicle's history dimension under the same pattern, and pattern attention is applied to the pattern dimension at the same timestamp.

[0128]

[0129] Among them, MHGA(·) represents the multi-head graph self-attention mechanism, and finally the proposed trajectory is generated through MLP decoding

[0130] S107: Generate anchor points based on the proposed trajectory, and process the proposed trajectory based on the anchor points to obtain the final predicted trajectory.

[0131] Optionally, S107 may include:

[0132] Get the historical trajectory of the agent;

[0133] The historical trajectory and the proposed trajectory are encoded separately according to the multi-layer perceptron to obtain the historical trajectory embedding and the proposed trajectory embedding;

[0134] The local correlation features of historical trajectory embedding and proposed trajectory embedding are extracted through 2D-CNN respectively to obtain historical trajectory features and proposed trajectory features;

[0135] Align the historical trajectory features and the proposed trajectory features and perform one-dimensional splicing to obtain multi-dimensional features;

[0136] Get anchor points based on multi-dimensional features;

[0137] Convert the proposed trajectory into an anchor-based polar coordinate representation to obtain the converted features;

[0138] After encoding the converted features through a multi-layer perceptron, they are processed according to the graph attention mechanism and the decomposition attention mechanism to obtain the trajectory offset;

[0139] The final predicted trajectory is obtained based on the trajectory offset and the proposed trajectory.

[0140] Understandably, the reference Figure 2 , the historical trajectory and proposed trajectory Convert to the scene center representation in polar coordinates. Obtain trajectory embedding through different MLP layers and proposed embedding Then, Expand in the last dimension and Add them together to get E p,i Therefore, E p,i Each element in contains the spatial features of the future timestamp and the corresponding historical timestamp. Next, a 2D-CNN block is introduced to extract local correlation features of the future trajectory of the vehicle in different dimensions. The process is shown as follows:

[0141]

[0142] Among them, u represents the input feature in Conv2D(u,v,w), v is the number of convolution channels, and w is the convolution kernel size. Represent the number of channels, length, and width of features respectively, and use multi-channel convolution to independently learn trajectories of different dimensions to improve the overall representation ability of the model. Contains the local relevant features of the future trajectory at each historical timestamp, and Contains the local relevant features of the future trajectory in each mode. and After alignment, the last dimension is concatenated to form a multidimensional feature X i Decode X using an MLP with a Sigmoid activation function i , the output dimension is one. Scale the result according to the optional trajectory point range to generate anchor points

[0143] The proposed trajectory is converted into a polar coordinate representation based on the anchor point, including relative distance and direction. These features are encoded by MLP to obtain a new pattern query. Then the agent-map attention and decomposition attention mechanism are performed on the pattern query again. Finally, the MLP outputs the trajectory offset and probability score π i The larger the probability score, the more the model believes that this trajectory is more consistent with the true value. The final trajectory after optimization Expressed as:

[0144]

[0145] To verify the effectiveness of the proposed algorithm, validation and comparative experiments were conducted. The experiments used the INTERACTION dataset, a high-quality multimodal traffic interaction dataset for autonomous driving and intelligent transportation system research. This dataset aims to promote research on modeling interactions between intelligent agents in complex traffic scenarios, and is particularly suitable for tasks such as trajectory prediction, behavior planning, and risk assessment.

[0146] Evaluation metrics include:

[0147] minJointADE: Calculates the average Euclidean distance between the predicted trajectories and the true trajectories of all agents at all future time steps. This is used to measure the overall performance of the trajectory closest to the true trajectory among multiple predicted trajectories in terms of average displacement error.

[0148] minJointFDE: Calculates the Euclidean distance between the last time step of each agent's predicted trajectory and the actual trajectory. This reflects the overall performance of the optimal endpoint prediction error for each trajectory in a multi-agent scenario. The results of the comparative experiment are shown in the table below.

[0149] Table 1

[0150] Model Type minJointADE minJointFDE Baseline Model 0.25 0.83 Model with inverse learning attention 0.24 0.78 Add dynamic anchor selection model 0.24 0.77 Model with temporal state encoding 0.22 0.73

[0151] As can be seen from Table 1, the proposed algorithms all achieve significant performance improvements, verifying the effectiveness of the methods. To verify the effectiveness and superiority of the proposed algorithms, a qualitative evaluation experiment was set up, that is, a visual comparison was performed between the baseline method and the proposed method. Figure 3 This is a schematic diagram of a predicted trajectory of a model without introducing time query coding provided by an embodiment of the present invention. Figure 4 This is a schematic diagram of a prediction trajectory of a model that introduces time query coding provided by an embodiment of the present invention. Figure 3 and Figure 4 In the figure, the end point of the predicted trajectory is indicated by an orange five-pointed star, and the predicted trajectory is indicated by an orange line; the true trajectory is shown using a blue-orange gradient line, and the true end point is indicated by a purple five-pointed star. Figure 5 is a schematic diagram of a prediction trajectory of a model without introducing reverse learning attention provided by an embodiment of the present invention. Figure 6 This is a schematic diagram of the prediction trajectory of a model that introduces reverse learning attention provided by an embodiment of the present invention. Figure 5 and Figure 6 In the figure, the predicted trajectory is represented by a red line, and each prediction result is output at the last timestamp of the historical trajectory. The true trajectory is represented by green and blue lines. Figure 7 is a schematic diagram of a predicted trajectory of a model without introducing dynamic anchor point selection provided by an embodiment of the present invention. Figure 8 This is a schematic diagram of a predicted trajectory of a model that introduces dynamic anchor point selection provided by an embodiment of the present invention. Figure 7 and Figure 8 In the figure, the yellow line represents the proposed trajectory, the red line represents the final predicted trajectory, and the blue stars represent the anchor points marked in the predicted trajectory.

[0152] In the above technical solution, the present invention combines the graph attention network with the multi-head cross-attention mechanism to effectively capture the complex relationship between agents and maps, and between agents and agents; through reverse learning embedding, historical intentions are inferred from a future perspective, significantly improving the accuracy of intention recognition; the dual-stream subtraction Mi-Transformer architecture is adopted to optimize the efficiency and stability of long sequence modeling, and pattern interaction modeling is realized in multiple dimensions through the decomposition attention mechanism; finally, the anchor point and trajectory optimization are combined to reduce the uncertainty and collision risk of the predicted trajectory, thereby achieving high-precision and high-robustness trajectory prediction, providing reliable behavior prediction support for autonomous driving and intelligent transportation systems.

[0153] It should be noted that the terms "first," "second," and the like are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention.

[0154] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0155] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by viewing the drawings and the disclosed content. In the description of the present invention, the word "comprising" does not exclude other components or steps, "one" or "a" does not exclude multiple situations, and "multiple" means two or more, unless otherwise clearly and specifically defined. In addition, certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0156] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for predicting an intelligent agent trajectory, characterized in that: The method comprises: Encoding each scene element in the scene map to obtain a node embedding and an edge embedding of each scene element; wherein the scene elements include lane segments and intelligent agents; Obtaining spatial embedding and temporal embedding according to the cross-attention mechanism, the node embedding and edge embedding of each scene element, and the initial pattern query embedding of the agent; wherein the initial pattern query embedding represents the future motion pattern of the agent; Obtaining a priori embedding based on the spatial embedding and the temporal embedding, and obtaining a reverse learning embedding based on the prior embedding; wherein the prior embedding is used to predict the behavior of the intelligent agent, and the reverse learning embedding is used to enhance the prediction performance; Obtaining a multi-feature pattern query embedding according to the spatial embedding, the temporal embedding, and the reverse learning embedding; Obtaining a temporal embedding based on a constructed lane-time interaction graph and a time-step interaction graph; wherein the lane-time interaction graph is constructed based on the node embedding and edge embedding of each scene element, and the time-step interaction graph is constructed based on the relative positions of the agents at different time steps; Processing the spatiotemporal embedding according to a two-stream subtractive Mi-Transformer and performing a decomposed attention mechanism to obtain a proposed trajectory; wherein the spatiotemporal embedding is obtained from the multi-feature pattern query embedding and the temporal embedding; Anchor points are generated according to the proposed trajectory, and the proposed trajectory is processed according to the anchor points to obtain a final predicted trajectory.

2. The method for predicting the trajectory of an intelligent agent according to claim 1, wherein: The encoding of each scene element in the scene map to obtain a node embedding and an edge embedding of each scene element includes: Constructing a first node feature based on the lane segments and corresponding polylines; wherein each lane segment comprises a plurality of vectorized polylines; constructing a first edge feature according to a mapping relationship between the lane segment and the polyline; Obtaining a first node embedding and a first edge embedding according to a multilayer perceptron, the first node feature, and the first edge feature; Obtaining a local representation of the lane segment according to the first node feature and the corresponding first edge feature; Each lane segment is regarded as the second node, and the mapping relationship between each lane segment is regarded as the second edge; Obtaining a second node feature and a second edge feature and encoding them according to the multi-layer perceptron to obtain a second node embedding and a second edge embedding; A scene map embedding is obtained according to the graph attention network, the second node embedding, and the second edge embedding.

3. The method for predicting the trajectory of an intelligent agent according to claim 2, wherein: The encoding of each scene element in the scene map to obtain a node embedding and an edge embedding of each scene element further includes: Constructing a third node feature of the agent according to the speed, speed direction, shape, and category of the agent, and constructing a third edge feature according to the mapping relationship between the agents; The third node embedding and the third edge embedding are obtained according to the multilayer perceptron, the third node feature, and the third edge feature.

4. The method for predicting the trajectory of an intelligent agent according to claim 1, wherein: The calculation method of the prior embedding is as follows: Among them, q fa represents the prior embedding, MHGCA(q,k,v) represents multi-head graph cross attention, q is the query, k is the key, v is the value, q ta Denotes the temporal embedding, E a represents the third node embedding, Represents the edge embeddings corresponding to the mapping relationship between agents in future attention.

5. The method for predicting the trajectory of an intelligent agent according to claim 4, wherein: The calculation method of the reverse learning embedding is expressed as follows: Among them, q ia represents the reverse learning embedding, Represents the edge embeddings corresponding to the mapping relationships between agents in historical attention.

6. The method for predicting the trajectory of an intelligent agent according to claim 5, wherein: The calculation method of the time embedding is as follows: q t =q lt +q tt ; Among them, q t represents the time embedding, q lt represents the dynamic interaction embedding between the scene map and the agent’s temporal state, q tt Represents the dynamic interaction embeddings between the temporal states of each agent.

7. The method for predicting the trajectory of an intelligent agent according to claim 6, wherein: The dynamic interaction embedding between the scene map and the agent's temporal state is calculated as follows: Among them, q sa represents the spatial embedding, E m represents the second node embedding, Represents the edge embeddings corresponding to the mapping relationship between the scene map and the agent in the lane-time interaction graph.

8. The method for predicting the trajectory of an intelligent agent according to claim 7, wherein: The calculation method of the dynamic interaction embedding between the temporal states of each agent is expressed as follows: in, Represents the edge embeddings corresponding to the mapping relationship between each agent in the time step interaction.

9. The method for predicting the trajectory of an intelligent agent according to claim 1, wherein: Generating an anchor point according to the proposed trajectory, and processing the proposed trajectory according to the anchor point to obtain a final predicted trajectory, including: Get the historical trajectory of the agent; Encode the historical trajectory and the proposed trajectory respectively according to a multi-layer perceptron to obtain a historical trajectory embedding and a proposed trajectory embedding; Extracting local correlation features of the historical trajectory embedding and the proposed trajectory embedding respectively through 2D-CNN to obtain historical trajectory features and proposed trajectory features; Aligning the historical trajectory features and the proposed trajectory features and performing one-dimensional splicing to obtain multi-dimensional features; Obtaining the anchor point according to the multidimensional feature; Converting the proposed trajectory into a polar coordinate representation based on the anchor point to obtain a converted feature; After encoding the converted features through the multi-layer perceptron, processing them according to the graph attention mechanism and the decomposition attention mechanism to obtain the trajectory offset; The final predicted trajectory is obtained according to the trajectory offset and the proposed trajectory.

Images