Track prediction method based on adaptive interaction and dynamic intention

Abstract

The invention relates to the technical field related to automatic driving, in particular to a trajectory prediction method based on adaptive interaction and dynamic intention, which comprises the following steps: firstly, constructing a heterogeneous interaction map, dividing a traffic scene into a vehicle grid, an environment grid and a non-driving area grid, and embedding multi-dimensional dynamic features; then dynamically adjusting a region of interest based on the behavior intention of the target vehicle, and extracting a high-correlation interaction subnet; modeling an interaction relationship by adopting a heterogeneous graph convolutional network, and processing the motion characteristics of the target vehicle and the neighbor vehicle through a sub-channel coding strategy; further realizing dynamic intention perception through a double-branch parallel attention architecture, and fusing macroscopic intention and dynamic intention information; and finally, iteratively generating a future trajectory prediction result based on a decoding architecture of a message passing mechanism. The method can effectively improve the long-term prediction performance in a lane changing scene, adaptively captures a dynamic interaction relationship, and improves the adaptability of a prediction system to the behavior intention change of a driver.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, and in particular to a trajectory prediction method based on adaptive interaction and dynamic intention. BACKGROUND

[0002] As one of the core technologies of automatic driving system, the main goal of trajectory prediction is to analyze historical trajectories and traffic environment information to predict the future driving trajectory of a vehicle, providing an important basis for decision planning. However, existing methods perform well in simple straight-line scenarios, but face the bottleneck of insufficient performance in sudden intention trajectory and long-term prediction (3-5 seconds). Traditional models rely on fixed grid feature maps and convolutional neural networks (CNN) to extract interaction features, but static grid processing defaults equal importance of different regions, cannot distinguish the actual contribution of different regions according to actual conditions, and may also introduce redundant noise, significantly increasing the computational burden and leading to decreased prediction accuracy and efficiency. In addition, current single-intention and multi-intention models rely on static intention modeling, i.e., assuming that the intention of the target vehicle remains unchanged throughout the prediction period. This method is suitable for simple straight-line trajectory scenarios, but it is difficult to cope with sudden intention adjustments or complex interactive game situations in real traffic (such scenarios are the cause of high accident rates), resulting in insufficient prediction reliability in dynamic environments. SUMMARY

[0003] The purpose of the present application is to provide a method for deblurring based on event spatio-temporal motion perception in real dynamic scenarios, to solve the problem of prediction uncertainty such as sudden change of driving intention that the existing technology cannot cope with.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical solution: a trajectory prediction method based on adaptive interaction and dynamic intention, comprising the following steps:

[0005] S1, heterogeneous interaction map construction: dividing the grid in the traffic scene into vehicle grid, environment grid and non-drivable area grid, and embedding multi-dimensional dynamic features respectively;

[0006] S2, adaptive regional grid positioning: based on the behavior intention of the target vehicle and the real-time changes of the traffic environment, dynamically adjusting the size and position of the focus area, obtaining the key focus area highly related to the future trajectory prediction of the target vehicle;

[0007] S3, interaction relationship extraction: using a heterogeneous graph convolutional network to model the node relationship in the interaction subnetwork and extract interaction features;

[0008] S4, introducing a motion and interaction encoding module: through a split-channel encoding strategy, independently encoding the motion and interaction features of the target vehicle and the motion state of the key neighbor vehicles;

[0009] S5, introduce frame-by-frame dynamic intention perception module: distinguish dynamic intention through double-branch parallel attention architecture, and fuse with macro intention;

[0010] S6, decoding architecture based on message passing mechanism, iteratively generate future trajectory prediction results of target vehicle.

[0011] Preferably, the heterogeneous interaction map construction process in S1 comprises:

[0012] According to different grid categories, a grid feature tensor describing the dynamic interaction information of the target vehicle and its surroundings at time t is constructed The definition of the mth row and nth column grid is:

[0013]

[0014] Where x, y, v x ,v y ,a x ,a y are the coordinates, speed and acceleration of the vehicle in x and y directions, Δx and Δy are the relative positions of the environment grid center point and the target vehicle grid center point in x and y directions; N front ,N back ,N left ,N right respectively represent the number of vehicles in the front, rear, left and right directions of the environment grid, if there is a vehicle in this direction, the value is 1; if there is no vehicle, the value is 0.

[0015] Preferably, the adaptive region grid positioning process in S2 comprises:

[0016] S201, calculate the positioning offset coordinates of the adaptive interaction sub-grid in the complete grid feature map, and determine the position of the sub-grid:

[0017] g x ,g y ,g w ,g h =Sigmoid(genc(M t ))

[0018] Where g x ,g y ,g w ,g h respectively represent the mapping of the horizontal coordinate, vertical coordinate, width and height of the interaction sub-grid in the range of [0, 1]; genc() is a multilayer perception (MLP) composed of two linear layers and batch normalization layers, and Sigmoid() is an activation function;

[0019] S202, specific coordinate calculation method of sub-grid:

[0020] grid x = floor(P, g x )

[0021] grid y = floor(Q, g y )

[0022] grid w = ceil((P-grid x ), g w )

[0023] gird h = ceil((Q-grid y ), g h )

[0024] where gird x , gird y , gird w , gird h represent the horizontal coordinate, vertical coordinate, width and height of the interaction subnetwork respectively; floor is the floor operation, ceil is the ceiling operation, P and Q are the total grid in the horizontal and vertical direction respectively.

[0025] Preferably, the interaction relationship extraction process in S3 comprises:

[0026] S301, constructing a heterogeneous graph to represent the dynamic dependency structure in the interaction area, the nodes Definition:

[0027]

[0028] wherein, is a flag, which is 1 when it is a vehicle node, -1 when it is an environment node, and 0 when it is an untraversable area;

[0029] S302, the adjacency matrix of the heterogeneous graph G is represented as where L represents the time step, N is the number of nodes in the graph, based on the constructed heterogeneous graph, an interaction feature is extracted using a three-layer graph convolution network (GCN), and the core calculation process is:

[0030]

[0031] wherein H (l) represents the node feature matrix of the lth layer, is the result of adding self-connection to the adjacency matrix A, I is the unit matrix, is the degree matrix of , whose diagonal elements are W (l)is the weight parameter learned by GCN; σ() is the activation function, and the final module outputs the interaction features R of all nodes t :

[0032]

[0033] wherein, represents the interaction feature vector of the i-th node in the graph, represents the interaction feature vector of the target vehicle.

[0034] Preferably, the motion and interaction encoding module in S4 specifically comprises:

[0035] S401, using the Transformer encoder to model the time sequence of the motion and interaction features of the target vehicle, and taking the target vehicle interaction feature t from R and its motion state feature to perform feature-level fusion to generate a complete interaction feature representation δ t as the input of the encoder:

[0036]

[0037] wherein, represents the motion state feature vector of the target vehicle at time step t, containing position coordinates, velocity, acceleration and other kinetic parameters, and Concat() is a feature concatenation operation. The feature δ t is mapped to a high-dimensional space representation by an embedding layer (Embedding Layer)

[0038]

[0039] wherein, W d is a learnable embedding weight matrix, and the embedding feature at each time step t is superimposed with a position encoding vector P t with time sequence perception to generate the interaction embedding feature E t of the target vehicle:

[0040]

[0041] wherein, the d-dimensional components p t of P t,d are generated by different frequency sine and cosine functions, and the calculation process is as follows:

[0042]

[0043] Wherein, D is the position coding mapping dimension, the encoder is composed of six identical coding layers stacked, each layer contains multi-head self-attention mechanism and feed-forward fully connected network, and is respectively provided with residual connection and layer normalization;

[0044] S402, the motion characteristics of the neighbor vehicle are encoded by using a gated recurrent unit network (GRU), and the motion state of the ith key neighbor vehicle at time r can be expressed as The six-dimensional motion characteristics including position, velocity, acceleration, etc. are included, and the continuous motion characteristic sequence of the vehicle within the history T=15 frames are sequentially input into the GRU model for time series modeling, and the update process of the GRU is as follows:

[0045]

[0046] Wherein, z t and r t respectively represent the update gate and the reset gate, is the hidden state, is the candidate hidden state, and is the element-wise multiplication. Q z , W r , and W h are learnable weight matrices, and after the recursive operation of the above GRU unit, the final encoding representation of the ith neighbor vehicle is obtained and is used as the decoding input of the key neighbor vehicle.

[0047] Preferably, the frame-by-frame dynamic intention perception module in S5 specifically comprises:

[0048] S501, macroscopic intention branch, assuming that the input of the decoder at the prediction time t+k (k∈[1, 25]) is the query vector processed by the embedding layer and the self-attention layer The historical trajectory feature sequence output by the encoder is encoded into a key matrix and a value matrix Wherein, N is the number of historical frames, d is the model mapping dimension, and the cross-attention mechanism first calculates the distribution of attention weights, and then calculates the relevance between the decoder query Q t+k and the encoder key value K through scaled dot-product attention (Scaled Dot-Product Attention) to generate a weight matrix as follows:

[0049]

[0050] Wherein, d h represents the dimension of the key vector or the query vector in each attention head;

[0051] S502, in the t+k (k∈[1, 25]) time of trajectory prediction, the target vehicle decoder first inputs the current time through the embedding layer and the self-attention layer to generate a query vector At the same time, based on the motion state sequence of the M key neighbor vehicles extracted from the adaptive interaction map, the GRU encoding is input into the decoder to obtain the neighbor hidden state of the i-th neighbor vehicle at t+k Subsequently, it is further mapped into a key-value pair matrix through a multi-layer perception (MLP):

[0052]

[0053] Wherein, W K , W V and b K , b V are learnable weight matrix and bias term respectively, and σ() represents the activation function, and the key-value vectors of all M neighbor vehicles constitute the key-value pair matrix in the attention module, on this basis, the frame-level neighbor-aware frame-wise attention (NFA) calculation process:

[0054]

[0055] S503, the macroscopic intention and the dynamic intention are fused through the gating fusion mechanism, the macroscopic intention and the dynamic intention branch are expanded into a multi-head attention structure, and at the prediction t+k frame, the macroscopic intention and the frame-by-frame dynamic intention context are represented as:

[0056]

[0057] The gating fusion mechanism is adopted to calculate the fusion weight:

[0058]

[0059] Wherein, || represents the vector splicing operation, W α and b α are learnable parameters, and σ() represents the Sigmoid activation function, and the fusion coefficient α t+k ∈[0, 1], finally, the two types of intention vectors are linearly weighted and fused using the gating weight to obtain the final fused feature representation:

[0060]

[0061] Preferably, the specific implementation of the message passing decoding architecture in S6 comprises:

[0062] S601, is sent into a feedforward neural network for nonlinear transformation, and the transformation process is as follows:

[0063] FFN(x) = W2ReLU(W1x + b1) + b2

[0064] Wherein, W1, W2 and b1, b2 are weight matrix and bias term of linear transformation respectively, through residual connection and layer normalization to stabilize the training process, and output the predicted coordinates at future t+k time

[0065] S602, introduce the message passing mechanism between the target vehicle and the neighbor vehicles, and there are M key neighbor vehicles, and the coordinate information Firstly, the target vehicle message is mapped through a nonlinear layer:

[0066]

[0067] Wherein, W e is a feature projection matrix, b e is a bias term, and sigma() represents a GELU activation function, and the target vehicle message will be spliced with the decoding hidden state vector of each key neighbor vehicle m in {1, 2,..., M} to form the GRU unit input of the neighbor vehicle at the next prediction time t+k+1, so as to update the hidden state, and the calculation process is:

[0068]

[0069] The updated hidden state is used as the input of the next prediction unit to generate the trajectory prediction result at the next time:

[0070]

[0071] Wherein, W o is a linear transformation weight matrix, b o is a bias term, and next, the trajectory prediction result of the neighbor vehicle is generated through a nonlinear layer to generate a neighbor vehicle message, and is used as a key-value matrix K m and V m are input into the dynamic intention branch of the target vehicle to perform weight calculation:

[0072]

[0073] Wherein, the query vector Q ego comes from the current decoding state of the target vehicle, and the above process is repeated until the predicted trajectories at all future times t+k are generated.

[0074] Compared with the prior art, the beneficial effects of the present application are:

[0075] 1. A dynamic adaptive region-based interactive feature extraction method is proposed to solve the problem of fixed grid feature map that cannot distinguish the importance of regions, is prone to introduce noise and has low computational efficiency.

[0076] 2. A trajectory prediction model based on frame-by-frame dynamic intention perception is proposed. This model can capture the macro trend and instantaneous change of driving intention in real time, and combines the channel coding strategy and message passing decoding mechanism to ensure computational efficiency while significantly improving the accuracy of long-term trajectory prediction in complex traffic scenarios (especially when the intention changes suddenly). It solves the prediction deviation problem caused by the fixed intention assumption of traditional methods. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1 The overall architecture diagram of the "trajectory prediction method based on adaptive interaction and dynamic intention" of the present application;

[0078] Figure 2 The overall framework diagram of the dynamic adaptive interactive feature extraction method of the present application;

[0079] Figure 3 The heterogeneous interaction map of the present application;

[0080] Figure 4 The process diagram of the heterogeneous graph convolutional neural network for extracting interaction relationship of the present application;

[0081] Figure 5 The double-branch parallel attention architecture of the present application. DETAILED DESCRIPTION

[0082] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0083] Please refer to Figures 1-5 The present application provides a technical solution: a trajectory prediction method based on adaptive interaction and dynamic intention, comprising the following steps:

[0084] Step 1: dynamic adaptive region interaction extraction, Figure 2The overall framework of the dynamic adaptive interaction feature extraction method is demonstrated. The method dynamically adjusts the attention area by constructing a heterogeneous interaction map and combining the behavior intention of the target vehicle, and uses a graph convolution network to extract key interaction features to provide structured support for the trajectory prediction task. First, a heterogeneous interaction map is constructed to comprehensively represent the spatial relationship between the target vehicle, surrounding vehicles and the environment. The map is composed of three types of nodes: vehicle nodes, environment nodes and non-drivable area nodes. The vehicle node contains dynamic information such as its position, speed and acceleration; the environment node encodes the relative position and adjacent vehicle distribution; and the non-drivable area node is used to mark non-passable areas such as green belts and emergency lanes, and its feature is set to a zero vector. Second, based on the intention and behavior characteristics of the target vehicle, the interaction subgraph is dynamically extracted, the perception range is flexibly adjusted, and the areas highly related to the future trajectory are preferentially focused on. Finally, a heterogeneous graph convolution network is used to aggregate the features of the extracted interaction subgraph. This method fully integrates the dynamic interaction information between the target vehicle and the surrounding environment, providing strong support for high-precision trajectory prediction.

[0085] Further, step 1 specifically includes the following sub-steps:

[0086] Step 11: Heterogeneous interaction map construction. First, this method designs a 13x3 grid feature map, where the width of each grid is consistent with the lane width, and the length is equal to the length of a standard vehicle, Figure 3 The heterogeneous interaction map constructed in this step is shown. The map divides the grid into three types: vehicle grid (large solid node), environment grid (small solid node), and non-drivable area grid (such as green belt, emergency lane, etc., white hollow node). Different types of grids will be embedded with different feature vectors.

[0087] According to different grid categories, a grid feature tensor is constructed to describe the dynamic interaction information of the target vehicle and its surroundings at time t The definition of the mth row and nth column grid is as follows:

[0088]

[0089] where β mn is a flag. If it is a vehicle grid (target vehicle and surrounding vehicles), β mn = 1; if it is an environment grid, β mn = -1; if it is a non-drivable area grid (such as green belt, emergency lane, etc.), β mn = 0, where X mn is a six-dimensional feature vector, and the embedding method is defined as follows according to the grid type:

[0090] (1) Vehicle grid: The feature vector is the motion feature of the target vehicle or surrounding vehicles, defined as follows:

[0091] X car = [x, y, v x , v y , a x , a y ] (2)

[0092] where x, y, v x , v y , a x , a y are the coordinates, velocity and acceleration of the vehicle in x and y directions, respectively;

[0093] (2) Environment grid: The feature vector is to describe the relative position of the grid to the target vehicle and the complexity of the surrounding environment, which is defined as follows:

[0094] X env = [Δx, Δy, N front , N back , N left , N right ] (3)

[0095] where Δx and Δy are the relative positions of the center point of the environment grid and the center point of the target vehicle grid in x and y directions; N front , N back , N left , N right represent the number of vehicles in front, back, left and right directions of the environment grid, respectively. If there is a vehicle in this direction, the value is 1; if there is no vehicle, the value is 0. This way of avoiding embedding all zero vectors can effectively reflect the interaction density and complexity around the environment grid, helping the model to evaluate whether the target vehicle needs to focus on this area;

[0096] (3) Non-drivable area grid: The feature vector is all zeros, indicating that the target vehicle does not need to pay attention to the unreachable area in the current road state, which is defined as follows:

[0097] X no = [0, 0, 0, 0, 0, 0] (4)

[0098] Combine the above flag β mn and the feature vector X mn to finally build a complete heterogeneous interaction map grid tensor Specifically, it is as follows:

[0099] (5)

[0101] With the above improvements, this differentiated feature embedding strategy not only retains the effective capture of vehicle interaction information by traditional models, but also significantly enhances the understanding of environmental features in complex traffic scenarios, providing richer contextual information for trajectory prediction.

[0102] Step 12: Adaptive regional grid positioning. By combining the full grid feature map with the motion state of the target vehicle and surrounding vehicles, the positioning offset coordinates of the adaptive interaction sub-grid in the full grid feature map are calculated to determine the position of the sub-grid. The calculation formula is as follows:

[0103] g x ,g y ,g w ,g h = Sigmoid(genc(M t )) (6)

[0104] where g x ,g y ,g w ,g h represent the mapping of the horizontal coordinate, vertical coordinate, width and height of the interaction sub-grid in the range [0, 1] respectively; genc() is a multilayer perception composed of two linear layers and batch normalization layers; Sigmoid() is an activation function used to constrain the output value in the range [0, 1]. The specific coordinates of the sub-grid are as follows:

[0105] gird x = floor(P,g x ) (7)

[0106] gird y = floor(Q,g y ) (8)

[0107] gird w = ceil((P-gird x ),g w ) (9)

[0108] gird h = ceil((Q-gird y ),g h ) (10)

[0109] where gird x , gird y , gird w , gird h represent the horizontal coordinate, vertical coordinate, width and height of the interaction sub-grid respectively; floor is the floor operation, ceil is the ceil operation, and P and Q are the total grids in the horizontal and vertical directions respectively.

[0110] Compared with the traditional fixed full grid feature extraction method, the adaptive interactive sub-grid can significantly reduce the redundant information, and by selectively focusing on the key area, the calculation of irrelevant areas is reduced, and this dynamic adjustment strategy can not only better capture the key interactive features of the target vehicle and the surrounding environment, but also improve the accuracy and robustness of the prediction.

[0111] Step 13: Interaction relationship extraction, each grid is regarded as a node in the graph convolution network, and the interaction relationship between the nodes is described by constructing a heterogeneous graph convolution network, in this way, the GCN can effectively model the dynamic topology of the interactive sub-grid, so as to extract the environmental information in the adaptive range, Figure 4 The process of extracting the interaction relationship by the heterogeneous graph GCN is shown, in order to effectively depict the dynamic interaction relationship between the target vehicle and the surrounding environment, the application constructs a heterogeneous graph G to represent the dynamic dependence structure in the interactive area, which is defined as follows:

[0112] G={G 1 ,G 2 ,G 3 ...G b} (11)

[0113] Among them, represents the heterogeneous graph at time t, represents all nodes at time t, and N≤39, the node includes three types of nodes: vehicle nodes, environment nodes and non-drivable area nodes, which are defined as follows:

[0114]

[0115] Among them, is a flag, which is 1 when it is a vehicle node, -1 when it is an environment node, and 0 when it is a non-drivable area;

[0116] The adjacency matrix of the heterogeneous graph G is represented as Where L represents the time step, N is the number of nodes in the graph, based on the constructed heterogeneous graph, the application adopts a three-layer graph convolution network (GCN) to extract the interaction features, and the core calculation process is as follows:

[0117]

[0118] Among them, H (l) represents the node feature matrix of the lth layer, is the result of adding self-connection to the adjacency matrix A, and I is the unit matrix, is the degree matrix of , whose diagonal elements are W(l) is the weight parameter learned by GCN; σ() is the activation function, by stacking three layers of GCN, the model can gradually aggregate neighbor information, so as to more comprehensively capture the complex interaction relationship between nodes, and the output of the final module is the interaction feature of all nodes, as follows:

[0119]

[0120] wherein represents the interaction feature vector of the i-th node in the graph, represents the interaction feature vector of the target vehicle.

[0121] Compared with the traditional CNN-based method, GCN can more flexibly adapt to different scales and shapes of interaction regions by directly acting on the graph structure, and at the same time, GCN models the interaction through the adjacency relationship between nodes, so that it has higher adaptability in capturing complex spatial relationships and dynamic environmental influences.

[0122] Step 2: Introduce motion and interaction encoding module, encode the motion and interaction features of the target vehicle and the motion state of the key neighbor vehicles respectively through the split-channel encoding strategy.

[0123] Step 2 specifically includes the following sub-steps:

[0124] Step 21: Time series encoding of the motion and interaction features of the target vehicle

[0125] The motion and interaction features of the target vehicle are time series modeled through the encoding structure of the Transformer, first, through the dynamic adaptive regional interaction extraction framework of step one, the interaction information R t at time t is obtained, which contains the multi-dimensional interaction features of the target vehicle and the surrounding environment, then, the target vehicle interaction feature t is taken out from R and its motion state feature is fused to generate a complete interaction feature representation δ t , which is taken as the input of the encoder, and the fusion process is as follows:

[0126]

[0127] wherein, represents the motion state feature vector of the target vehicle at time step t, including position coordinates, velocity and acceleration, etc. Concat() is a feature concatenation operation, in order to further improve the feature representation ability, the feature δ t is first mapped to a high-dimensional space representation by an embedding layer (Embedding Layer), and the embedding process is as follows:

[0128]

[0129] where W d is the learnable embedding weight matrix, and since the Transformer lacks explicit modeling ability for time series information, it is necessary to retain the time series characteristics of the motion state through the sinusoidal position encoding mechanism, specifically, the embedding feature of each time step t is superimposed with the position encoding vector P t that has time awareness, to generate the interaction embedding feature E t of the target vehicle, and the position encoding process is as follows (5.3):

[0130]

[0131] where P t is the position encoding vector, and each d-dimensional component p t,d is generated using sine and cosine functions with different frequencies, and the calculation process is as follows:

[0132]

[0133] where D is the position encoding mapping dimension. This encoding scheme ensures the uniqueness of position information for sequences up to 10000 steps through exponentially decaying frequency intervals (from 2π to 2000π), effectively expanding the model's perception ability for time steps.

[0134] The encoder module is stacked by six encoding layers with identical structure, each of which mainly consists of two parts: multi-head self-attention mechanism (Multi-Head Self-Attention) and feed-forward fully connected network (Feed-Forward Network), and after each sub-module, residual connection (Add) and layer normalization (Layer Normalization) are introduced to alleviate the gradient vanishing problem and speed up the convergence speed.

[0135] The multi-head self-attention mechanism is the core of the Transformer architecture, aiming to model the dependency between time steps in parallel through multiple different projection spaces, this mechanism first maps the target vehicle embedding feature E t into query (Query), key (Key), and value (Value) matrices through three learnable linear mappings, and the mapping process is as follows (5.6):

[0136] Q = E (t) W Q , K = E (t) W K , V = E (t) W V (20)

[0137] where Q W , W K , W V is a trainable weight matrix, then the attention weights are obtained by calculating the similarity of Q and K, and the single-head attention output is obtained by weighted summation of Value, the calculation process is as follows:

[0138]

[0139] where d h represents the dimension of the key vector or the query vector in each attention head. Multi-head self-attention mechanism calculates h different attention heads in parallel, so that each head can learn different attention representations independently, and finally the outputs of all heads are spliced and projected back to the original dimension, the calculation process is as follows:

[0140] MultiHead(Q,K,V)=Concat(head1,…,head h )W o (22)

[0141]

[0142] where, is a learnable weight matrix, head i is the i-th attention head, through the multi-head attention mechanism, the modeling ability of the model to different scales and different position dependencies is enhanced, so that the encoder can efficiently capture the complex dependencies between the target trajectory and its historical behavior, and provide a stable semantic representation basis for trajectory prediction.

[0143] Step 22: motion feature time series encoding of neighbor vehicles

[0144] The present application proposes the concept of "key neighbor vehicles", which is defined as the surrounding vehicles contained in the adaptive interaction map of the last frame of the historical trajectory. The vehicle distribution reflected by this frame can best represent the traffic scene and driving intention at the current time, and has a direct impact on the subsequent decision and behavior change of the target vehicle. Therefore, reasonable encoding of the historical motion state of these neighbor vehicles is crucial for predicting their future behavior. Considering that the present application has used Transformer to encode and decode the motion and interaction features of the target vehicle, the prediction accuracy of the surrounding vehicles is not the main optimization goal of this research. Therefore, a more lightweight gated recurrent unit network (GRU) is used to encode the historical trajectory and motion state of the neighbor vehicles, which effectively controls the parameter quantity while ensuring the performance of the model.

[0145] The GRU structure has better long-term dependence modeling capability than the traditional RNN, and is more efficient in calculation. Specifically, the motion state of the i-th key neighbor vehicle at time t can be expressed as The six-dimensional motion features including position, velocity, acceleration, etc. of the vehicle in the continuous motion feature sequence of the vehicle in the history T = 15 frames are sequentially input into the GRU model for time series modeling. The update process of the GRU is as follows:

[0146]

[0147] wherein z t and r t represent the update gate and the reset gate respectively, is the hidden state, is the candidate hidden state, σ() is the Sigmoid activation function, and is the element-wise multiplication. W z , W r , and W h are learnable weight matrices. After the recursive operation of the above GRU unit, the final encoding representation of the i-th neighbor vehicle is obtained and is used as the decoding input of the key neighbor vehicle.

[0148] Step 3: Introducing a frame-by-frame dynamic intention perception module, which realizes the continuous update of the intention by establishing a frame-level dynamic interaction mechanism between the target vehicle and the key neighbor vehicle. Specifically, the improved Transformer decoder adopts a double-branch parallel attention architecture, Figure 5 which shows the double-branch parallel attention architecture. The first branch is the macro intention branch, which retains the standard cross-attention mechanism, captures the spatial dependency of the input sequence through global context modeling, and provides a stable intention benchmark for trajectory prediction. The second branch is the dynamic intention branch, which dynamically infers the current intention frame by frame through the introduction of the frame-by-frame intention perception module, and realizes the fine correction of the macro intention. Finally, through the feature fusion layer, the multi-granularity collaborative optimization of the macro intention and the dynamic intention is realized, which effectively improves the adaptability of the model to complex scenes.

[0149] Step 3 specifically includes the following sub-steps:

[0150] Step 31: Macroscopic intention branch, in the Transformer-based trajectory prediction model, the Cross-Attention module is the core component connecting the encoder and the decoder, which dynamically aggregates the historical trajectory features output by the encoder to macroscopically guide the decoder to generate future trajectories. Compared with the self-attention mechanism, Cross-Attention focuses on cross-sequence information interaction, enabling the decoder to adaptively extract the most relevant information from the historical trajectory representation of the encoder at the current prediction time.

[0151] Specifically, let the input of the decoder at the prediction time t+k (k ∈ [1, 25]) be the query vector Q after the embedding layer and the self-attention layer processing The historical trajectory feature sequence output by the encoder is encoded into the key matrix K and the value matrix V where N is the number of historical frames, and d is the model mapping dimension. The Cross-Attention mechanism first calculates the distribution of attention weights, and then calculates the relevance between the decoder query Q t+k and the encoder key value K to generate the weight matrix as follows (5.14):

[0152]

[0153] where d h represents the dimension of the key vector or the query vector in each attention head. Similar to the encoder, the multi-head attention mechanism can be used to enhance feature diversity, and Q t+k , K, and V are mapped into h subspaces for parallel calculation, and the mapping process is as follows:

[0154] MultiHead(Q t+k ,K,V)=Concat(head1,…,head h )W 0 (29)

[0155]

[0156] where is a learnable weight matrix.

[0157] The Cross-Attention mechanism enables the decoder to adaptively focus on the most representative position in the input trajectory at each prediction time step based on the current generation state, effectively capturing macroscopic motion trends and semantic context information, and guiding future trajectory prediction. In the vehicle trajectory prediction scenario, this cross-sequence information interaction is particularly crucial for modeling long-term dependencies and understanding overall intent.

[0158] Step 32: dynamic intention branch, in the trajectory prediction task, the macro intention dimension is usually taken as a baseline reference for prediction, guiding the future trajectory, however, the intention in the actual driving scene will present dynamic change characteristics within the prediction time domain, which requires the model to have more fine intention recognition ability, for this purpose, the application introduces a frame-by-frame intention perception module in the decoder, which realizes the accurate identification and prediction of frame-level dynamic intention by analyzing the motion state changes of the key neighbor vehicles within the prediction time domain in real time.

[0159] Specifically, at the t+k (k∈[1,25]) time of trajectory prediction, the target vehicle decoder first processes the current time input through the embedding layer and the self-attention layer to generate the query vector to represent the decoding state of the target vehicle at the current time, at the same time, the motion state sequence of the M key neighbor vehicles extracted based on the adaptive interaction map is input into the decoder after GRU encoding to obtain the neighbor hidden state of the i-th neighbor vehicle at t+k To maintain dimensional consistency, further map it to a key-value pair matrix through a multi-layer perception (MLP), the calculation process is as follows:

[0160]

[0161] Where, W K , W V and b K , b V are the learnable weight matrix and bias term respectively, and σ() represents the activation function, the key-value vectors of all M neighbor vehicles constitute the key-value pair matrix in the attention module, as follows (5.19) and (5.20):

[0162]

[0163] On this basis, the application proposes a frame-level neighbor-aware attention mechanism NFA (Neighbor-aware Frame-wise Attention), which dynamically infers the intention of the target vehicle at each prediction frame by modeling the future trajectory and interaction relationship of the key neighbor vehicles, the calculation process of this mechanism is as follows:

[0164]

[0165] Where, reflects the attention degree of the target vehicle to the future trajectory of the neighbor vehicle, and the frame-level features of the neighbor vehicles are weighted and fused after softmax normalization, finally generating the dynamic intention representation d hdenotes the dimension of Key vector or Query vector in each attention head, to further enhance the model expression ability, NFA can also be extended to multi-head attention structure as follows:

[0166]

[0167] wherein, is a learnable weight matrix, h denotes the number of attention heads, this module realizes the frame-by-frame dynamic intention reasoning of future trajectory, so that the decoder can adaptively perceive the potential influence of neighbor vehicles on the current state at each prediction time, thereby accurately identifying the intention mutation and improving the prediction accuracy.

[0168] Step 33: Branch fusion, to effectively combine the above two intentions, the application proposes an intention fusion method based on a gating mechanism, which is used to fuse static macro intention and frame-by-frame dynamic intention in the Transformer decoder at the same time, to realize the complementation and collaborative guidance of different intentions, assuming that at the prediction time of the t+k frame, the macro intention and frame-by-frame dynamic intention context are represented as:

[0169]

[0170] To fuse the two types of intention information, a gating fusion mechanism is used to calculate the fusion weight, and the calculation process is as follows:

[0171]

[0172] wherein || denotes the vector splicing operation, W α and b α are learnable parameters, and sigma() represents the Sigmoid activation function, the fusion coefficient alpha t+k ∈[0, 1], represents the attention degree to the dynamic intention, finally, the two types of intention vectors are linearly weighted and fused using the gating weight, and the final fused feature representation is as follows:

[0173]

[0174] The intention vector after fusion has both macro and dynamic intention information, not only retains the guiding ability of the model in long-term behavior planning, but also has the sensitivity to sudden changes in the behavior of neighbor vehicles, effectively solving the prediction uncertainty problem.

[0175] Step 4: Decoding architecture based on message passing mechanism, iteratively generating future trajectory prediction results of the target vehicle, first, the final hidden state representation of the target vehicle historical trajectory after encoding is denoted as H tAnd pass it as initial input into the Transformer decoder, the decoder structure is consistent with the encoder, which is composed of six layers of decoder layers (DecoderLayer), firstly, the input features are modeled by the multi-head self-attention mechanism, then the residual connection and layer normalization operation are used to alleviate the gradient vanishing problem and accelerate the convergence, then the feature vectors are input into the double-branch parallel attention module proposed in the application, which captures the macro intention and frame-by-frame dynamic intention respectively, and obtains the final intention information in the feature fusion layer Subsequently, Will be sent into the feedforward neural network for nonlinear transformation, the transformation process is as follows (5.28):

[0176] FFN(x)=W2·ReLU(W1x+b1)+b2 (42)

[0177] Wherein, W1, W2 and b1, b2 are weight matrix and bias term of linear transformation respectively, then the residual connection and layer normalization are used again to stabilize the training process, and finally the predicted coordinates of future t+k time are output

[0178] After the Transformer module outputs the future trajectory coordinates of the target vehicle at the current time, the application further introduces a message passing mechanism between the target vehicle and the neighbor vehicles to enhance the timeliness and response ability of multi-vehicle interaction modeling, which is used to bidirectionally pass the trajectory information of the target vehicle at each prediction time to the GRU decoding units of multiple key neighbor vehicles, so as to realize cross-agent dynamic intention perception, there are M key neighbor vehicles, and the trajectory coordinates of the target vehicle at the prediction time t+k are represented as The coordinate information is first mapped into the target vehicle message through a nonlinear layer, and the mapping process is as follows:

[0179]

[0180] Wherein, W e is a feature projection matrix, b e is a bias term, and σ(·) represents a GELU activation function, the target vehicle message will be concatenated with the decoding hidden state vector Of each key neighbor vehicle m∈{1,2,...,M} to form the GRU unit input of the neighbor vehicle at the next prediction time t+k+1, so as to update the hidden state, and the calculation process is as follows:

[0181]

[0182] The updated hidden state contains the joint information of the current neighbor vehicle's own historical state and the target vehicle's state, which is used as the input of the next prediction unit to generate the trajectory prediction result at the next time as follows:

[0183]

[0184] where W o is a linear transformation weight matrix, b o is a bias term. Next, the trajectory prediction results of the neighbor vehicles go through a nonlinear layer to generate neighbor vehicle messages, which are taken as a key-value matrix K m and V m are input into the dynamic intention branch of the target vehicle, and the weight calculation method is as follows:

[0185]

[0186] where the query vector Q ego comes from the current decoding state of the target vehicle. The above process is repeated until the prediction trajectories of all time instances t+k in the future are generated.

[0187] Through the above message passing mechanism, the model can realize the bidirectional perception of the target vehicle and the key neighbor vehicles at each prediction time step. If the intention of the neighbor vehicle changes dramatically, this mechanism can provide real-time feedback and guide the target vehicle to correct the trajectory generation result, thereby improving the robustness and accuracy of the overall prediction system in the intention mutation scenario.

[0188] Embodiment one, Figure 1 shows the overall architecture diagram of the trajectory prediction method based on adaptive interaction and dynamic intention. First, a split-channel encoding strategy is adopted, and the motion and interaction features of the target vehicle are processed through the Transformer encoder respectively. Meanwhile, a lightweight gated recurrent unit (GRU) network is used to independently encode the selected key neighbor vehicle trajectories. Second, a frame-by-frame intention perception module is designed, which models the macro intention and frame-level dynamic intention through a double-branch parallel attention mechanism, and realizes multi-scale intention fusion through a feature fusion layer. Finally, a message passing mechanism is introduced to model the bidirectional dynamic relationship between the target vehicle and the key neighbor vehicles in the decoding stage. Through the synergistic effect of hierarchical feature encoding, multi-granularity intention perception, and interactive message passing, this method significantly improves the prediction performance in complex scenarios.

[0189] Embodiment two, in order to comprehensively evaluate the performance of the present application in the trajectory prediction task, the root mean square error (RMSE) and the average displacement error (ADE) are used as the main evaluation indicators:

[0190] (1) Root Mean Square Error (RMSE): This index is obtained by calculating the square root of the mean of the square of the deviation between the predicted value and the true value, mainly reflecting the spatial deviation between the predicted trajectory and the true trajectory, and is used to measure the spatial position prediction accuracy of the model for the motion path, and the calculation process is as follows:

[0191]

[0192] (2) Average Displacement Error (ADE): This index calculates the arithmetic mean of the position error of each time step of the predicted trajectory, which has the advantage of being able to reflect the prediction accuracy of the model at each time point evenly, and the calculation process is as follows:

[0193]

[0194] The standard trajectory data set NGSIM and HighD are selected to comprehensively evaluate the trajectory prediction ability of the present application under various traffic situations. In all experimental settings, 15 frames (3 seconds) of historical trajectory data are used as input sequences to predict the position coordinates of the next 25 frames (5 seconds) of future trajectory data. The model architecture uses the FI-Trans model proposed in the present application, all multi-head attention mechanisms use 4 heads, and the decoder is trained by autoregressive method to better simulate the actual reasoning process. In the training parameter configuration, the Adam optimizer is used for parameter update, the initial learning rate is set to 10 -4 , and the exponential decay strategy is used with a decay rate of 0.1 to prevent oscillation or falling into local optimum in the later training period. The batch size is set to 256, and all experiments are implemented based on the PyTorch deep learning framework. To ensure the reliability of the evaluation, each data set is divided into training set and test set in the ratio of 7:3, and the test set is composed of unseen samples only, which is used to objectively evaluate the generalization ability of the model.

[0195] A comparative experiment scheme is designed, and six representative models are selected for comparative analysis on the HighD data set, the NGSIM data set and the special lane changing data set (LCX). The experiment compares the prediction error index under different prediction time domains (1-5s) to quantitatively analyze the performance difference of each model in short-term and long-term trajectory prediction tasks.

[0196] The experimental results are shown in Tables I and II. In the HighD dataset, the EA-Net leads in short-term prediction (1-2s), while the FI-Trans proposed by the present application performs outstandingly in long-term prediction (3-5s), especially when the prediction time domain is 5s, the RMSE error is reduced to 0.78, reaching a new low, which is reduced by 77.1% compared with the traditional S-LSTM model, and by 40.9% compared with the suboptimal model EA-Net. The experimental results of the NGSIM dataset further verify the above conclusion. Although the RMSE errors of all models generally increase due to the increase in scene complexity, the EA-Net and the FI-Trans still maintain a relative advantage. In particular, the RMSE error of the FI-Trans in 5s long-term prediction is reduced by 41.8% compared with the S-LSTM. In terms of the ADE index, the FI-Trans achieves the best in both datasets, indicating that the FI-Trans has a significant advantage in the smoothness of trajectory prediction.

[0197] Table I: Comparison of indicators of each test model in the HighD dataset

[0198]

[0199] Table II: Comparison of indicators of each test model in the NGSIM dataset

[0200]

[0201] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A trajectory prediction method based on adaptive interaction and dynamic intent, characterized in that, Includes the following steps: S1. Construction of heterogeneous interactive map: Divide the grid in the traffic scene into vehicle grid, environment grid and non-drivable area grid, and embed multi-dimensional dynamic features into each. S2. Adaptive regional grid positioning: Based on the target vehicle's behavioral intentions and real-time changes in the traffic environment, dynamically adjust the size and position of the area of ​​interest to obtain key areas of interest that are highly correlated with the prediction of the target vehicle's future trajectory. S3. Interaction Relationship Extraction: A heterogeneous graph convolutional network is used to model the node relationships in the interaction subnet and extract interaction features; S4. Introduce a motion and interaction coding module: Through a channel-based coding strategy, the motion and interaction characteristics of the target vehicle and the motion state of key neighboring vehicles are independently coded. S5. Introducing a frame-by-frame dynamic intent perception module: Dynamic intents are identified through a dual-branch parallel attention architecture and fused with macro-intents; S6. A decoding architecture based on a message passing mechanism iteratively generates the future trajectory prediction results of the target vehicle.

2. The trajectory prediction method based on adaptive interaction and dynamic intent according to claim 1, characterized in that, The heterogeneous interaction map construction process in S1 includes: Based on different grid categories, a grid feature tensor describing the dynamic interaction information of the target vehicle and its surroundings at time t is constructed. The definition of the grid in the m-th row and n-th column is: Where x, y, v x ,v y ,a x ,a y N represents the vehicle's coordinates, velocity, and acceleration in the x and y directions, respectively; Δx and Δy represent the relative positions of the environmental grid center point and the target vehicle grid center point in the x and y directions, respectively. front N back N left N right These represent the number of vehicles in the four directions of the environmental grid: front, back, left, and right. If there is a vehicle in that direction, the value is 1; otherwise, the value is 0.

3. The trajectory prediction method based on adaptive interaction and dynamic intent according to claim 1, characterized in that, The adaptive region grid localization process in S2 includes: S201. Calculate the location offset coordinates of the adaptive interactive subnet in the complete grid feature map to determine the position of the subnet: g x ,g y ,g w ,g h =Sigmoid(genc(Mt)) Among them, g x ,g y ,g w ,g h These represent the mapping of the horizontal coordinate, vertical coordinate, width, and height of the interactive subnet within the range [0,1]; genc() is a multilayer perceptron (MLP), consisting of two linear layers and a batch normalization layer; and Sigmoid() is the activation function. S202, Specific coordinate calculation method for subnets: gird x =floor(P,g x ) gird y =floor(Q,g y ) grid w =ceil((P-gird x ),g w ) gird h =ceil((Q-gird y ),g h ) grid x gird y gird w gird h These represent the x-coordinate, y-coordinate, width, and height of the interactive subnet, respectively; floor is the floor function, ceil is the floor function, and P and Q are the total grid in the horizontal and vertical directions, respectively.

4. The trajectory prediction method based on adaptive interaction and dynamic intent according to claim 1, characterized in that, The interaction relationship extraction process in S3 includes: S301. Construct a heterogeneous graph to represent the dynamic dependency structure within the interaction region, nodes. definition: in, This is a flag; its value is 1 when it is a vehicle node, -1 when it is an environment node, and 0 when it is a non-drivable area. S302, the adjacency matrix of the heterogeneous graph G is represented as: Where L represents the time step and N is the number of nodes in the graph, a three-layer graph convolutional network (GCN) is used for interactive feature extraction based on the constructed heterogeneous graph. The core calculation process is as follows: Among them, H (l) This represents the feature matrix of the nodes in the l-th layer. It is the result of adding self-joins to the adjacency matrix A, where I is the identity matrix. yes The degree matrix, whose diagonal elements are W (l) These are the weight parameters learned by GCN; σ() is the activation function, and the final module outputs the interaction features R of all nodes. t : in, This represents the interaction feature vector of the i-th node in the graph. This represents the interaction feature vector of the target vehicle.

5. The trajectory prediction method based on adaptive interaction and dynamic intent according to claim 1, characterized in that, The motion and interaction coding module in S4 specifically includes: S401. A Transformer encoder is used to perform temporal modeling of the motion and interaction characteristics of the target vehicle in R. t Extracting the target vehicle's interaction features Its motion state Perform feature concatenation and fusion to generate a complete interactive feature representation δ t As input to the encoder: in, This represents the motion state feature vector of the target vehicle at time step t, containing dynamic parameters such as position coordinates, velocity, and acceleration. `Concat()` is a feature concatenation operation, where feature δ... t Mapped to a high-dimensional space after embedding layer Among them, W d For a learnable embedding weight matrix, the embedding features at each time step t Superimposed with time-aware position encoding vector P t Generate the interactive embedding features E of the target vehicle. t : Among them, P t p of each d-dimensional component t,d Generated using sine and cosine functions of different frequencies; calculation process: Where D is the positional encoding mapping dimension, the encoder consists of six identical encoding layers stacked together, each layer contains a multi-head self-attention mechanism and a feedforward fully connected network, and each is equipped with residual connections and layer normalization; S402. A gated recurrent unit (GRU) network is used to encode the motion characteristics of neighboring vehicles. Let the motion state of the i-th key neighboring vehicle at time t be represented as: Including six-dimensional motion features such as position, velocity, and acceleration, the continuous motion feature sequence of the vehicle within a historical T=15 frames. The data is sequentially input into the GRU model for time series modeling. The GRU update process is as follows: Among them, z t and r t These represent updating the door and resetting the door, respectively. In hidden state, This represents a candidate hidden state, where ⊙ indicates element-wise multiplication. W z W r W h Given a learnable weight matrix, the final encoded representation of the i-th neighboring vehicle is obtained after the recursive operation of the GRU unit described above. It also serves as the decoding input for key neighboring vehicles.

6. The trajectory prediction method based on adaptive interaction and dynamic intent according to claim 1, characterized in that, The frame-by-frame dynamic intent perception module in S5 specifically includes: S501, Macro-intention Branch: Suppose that the input of the decoder at prediction time t+k (k∈[1,25]) is the query vector processed by the embedding layer and the self-attention layer. The historical trajectory feature sequence output by the encoder is encoded into a key matrix. Sum matrix Where N is the number of historical frames, d is the model mapping dimension, and the cross-attention mechanism first calculates the allocation of attention weights, and then calculates the decoder query Q through scaled dot-product attention. t+k The weight matrix generated based on the correlation with the encoder key value K is as follows: Where, d h This represents the dimension of the key vector or query vector in each attention head; S502, Dynamic Intent Branch: At time t+k (k∈[1,25]) of trajectory prediction, the target vehicle decoder first processes the current input through the embedding layer and the self-attention layer to generate a query vector. Meanwhile, the motion state sequences of M key neighbor vehicles extracted from the adaptive interactive map are encoded using GRU and then input into the decoder to obtain the hidden neighbor state of the i-th neighbor vehicle at time t+k. Subsequently, it is further mapped into a key-value pair matrix using a multilayer perceptron (MLP): Among them, W K W V and b K b V Here, σ represents the learnable weight matrix and bias term, respectively, and σ() represents the activation function. The key-value vectors of all M neighboring vehicles constitute the key-value pair matrix in the attention module. Based on this, the calculation process of the frame-aware neighbor-wise attention mechanism NFA is as follows: S503. The macro-intention and dynamic intention are fused through a gating fusion mechanism, and the macro-intention and dynamic intention branches are extended into a multi-head attention structure. At the prediction time of frame t+k, the macro-intention and frame-by-frame dynamic intention contexts are respectively represented as: A gating fusion mechanism is used to calculate the fusion weights: Where || denotes the vector concatenation operation, W α With b α The parameters are learnable, σ() represents the Sigmoid activation function, and the fusion coefficient α is... t+k ∈[0,1], and finally, gating weights are used to linearly weight and fuse the two types of intent vectors to obtain the final fused feature representation: 。 7. The trajectory prediction method based on adaptive interaction and dynamic intent according to claim 1, characterized in that, The specific implementation of the message passing decoding architecture in S6 includes: S601, The signal is fed into a feedforward neural network for nonlinear transformation. The transformation process is as follows: FFN(x)=W2ReLU(W1x+b1)+b2 Where W1, W2 and b1, b2 are the weight matrices and bias terms of the linear transformation, respectively. Through residual connections and layer normalization to stabilize the training process, the predicted coordinates at the future time t+k are output. S602. Introduce a message passing mechanism between the target vehicle and its neighboring vehicles. There are M key neighboring vehicles with coordinate information... First, the target vehicle message is mapped using a non-linear layer: Among them, W e Let b be the characteristic projection matrix. e As the bias term, σ() represents the GELU activation function. The target vehicle message will be compared with the decoded hidden state vector of each key neighbor vehicle m∈{1,2,…,M}. The data is concatenated to form the GRU cell input for the neighboring vehicle at the next prediction time t+k+1, in order to update the hidden state. The calculation process is as follows: The updated hidden state is used as input to the next prediction unit to generate the trajectory prediction result for the next time step: Among them, W o Let b be the linear transformation weight matrix. o As the bias term, the next step is to predict the trajectories of neighboring vehicles. Neighbor vehicle messages are generated through a nonlinear layer and used as the key matrix K. m With V m The input is fed into the dynamic intent branch of the target vehicle for weight calculation: Wherein, the query vector Q ego Based on the current decoding state of the target vehicle, repeat the above process until the predicted trajectories for all future time points t+k are generated.

Images