A spatio-temporal graph fusion trajectory prediction method across time steps interaction

Abstract

The present application relates to a kind of space-time graph fusion trajectory prediction methods of cross time step interaction, belong to trajectory prediction technical field, solve the problem of insufficient cross time step interaction feature extraction of conventional model.The technical scheme is: first, construct cross time step 3D space-time relationship graph to represent the information interaction of each target in space domain and time domain in scene;Then adopt position feedforward network and graph attention network to extract cross time step interaction information;At the same time, by constructing single-step space-time relationship graph and introducing graph convolution network and time convolution network, the spatial features of current time and the time dependence of each target are extracted, then the graph features of the above two models are fused using feature fusion network;Finally, the predicted trajectory of each target is output by ConvGRU encoder-decoder.The present application compared with other methods in recent period has obvious reduction in average ADE and FDE indicators, improves the accuracy of trajectory prediction.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of trajectory prediction of moving objects, and particularly relates to a spatio-temporal graph fusion trajectory prediction method of cross-time step interaction. BACKGROUND

[0002] The trajectory prediction technology mainly refers to predicting the future motion trend of a traffic participant according to the historical trajectory and high-precision map of the traffic participant. Precise and effective trajectory prediction can guide subsequent decision planning and improve the safety and reliability of a self vehicle in a driving process.

[0003] Currently, trajectory prediction research mainly includes two kinds: 1) modeling according to the dynamics or kinematics constraints of a moving object to predict the future motion state of the moving object. However, the use of this kind of method is based on the premise that the vehicle state and trajectory will not change dramatically in a short time, so it has certain limitations when facing complex traffic scenes. 2) a trajectory prediction method based on deep learning. With the continuous improvement of data volume and computer computing power, the trajectory prediction method based on deep learning is driven by data, combines the map perception information and historical trajectory features around the vehicle, and enables the model to better adapt to complex road conditions, gradually becoming the industry mainstream.

[0004] Among the many deep learning trajectory prediction methods, the prediction algorithm with LSTM and GRU network in recurrent neural network as the core architecture is widely used. Patent CN116872963A gives a vehicle trajectory prediction model based on LSTM, which includes an encoder and a decoder constructed based on LSTM. Patent CN113408588A proposes a bidirectional GRU trajectory prediction based on attention mechanism, which uses attention mechanism to mine important features and reduce the loss of key information, thereby effectively improving the accuracy of the prediction model. In addition, the WOA algorithm is combined to optimize the weights in the forward and backward propagation processes of the bidirectional GRU neural network. However, they only focus on the motion information of the target itself, and when facing complex traffic scenes such as multiple targets existing at the same time, they cannot obtain more accurate results. Therefore, it is necessary to extract the interaction feature information in the scene before performing trajectory prediction. The graph neural network (GNN) represents the interaction between targets in the scene by constructing a graph structure, providing more rich spatiotemporal features for subsequent trajectory prediction. Patent CN115147790A gives a multi-modal trajectory prediction algorithm based on graph neural network (GNN), which inputs the road graph and time sequence of the vehicle to be predicted into a deep learning model to output the predicted trajectory. Patent CN113505878A inputs the detected track sequence dataset into a graph neural network model to obtain the track prediction trajectory. Patent CN113869170A designs a pedestrian trajectory prediction model applied to complex scenes, which is based on graph partition convolutional neural network GP-CNN and uses a combination of two channels to extract the interaction features of the scene, and then obtains the predicted trajectory through multiple CNN layers.

[0005] The above graph-based research mostly considers the surrounding traffic participants equally, which cannot effectively capture the most important information interaction between targets and targets. Therefore, many scholars at home and abroad use attention-based methods to capture important interactions between targets based on graph networks, such as patent CN113888638A which uses a spatio-temporal graph convolutional neural network to extract the spatio-temporal features of the pedestrian trajectory node graph, and uses a graph channel attention mechanism to filter out important spatio-temporal features and input them into the predictor to obtain the future trajectory. However, the existing methods mostly separate the extraction of time and space information in the scene, ignoring the spatial interaction between moving objects across time steps. SUMMARY

[0006] In view of the above deficiencies in the prior art, the purpose of the present application is to provide a spatio-temporal graph fusion trajectory prediction method across time steps, which constructs a 3D spatio-temporal relationship graph across time steps, simultaneously focuses on spatial correlation and temporal correlation in the feature extraction module, and focuses on adjacent targets that have a greater impact on moving targets through an attention mechanism. The specific steps are as follows:

[0007] Step 1: Construct a cross-time-step 3D space-time relation graph. The graph is composed of two parts of nodes and edges. One target represents a node, and the edges are divided into two parts of space edges and time edges. The space edges are used to represent the interaction between two targets. The application distinguishes the relationship between each target and the surrounding targets by setting an adjacency distance threshold. If the distance between two nodes is less than the set adjacency distance, a space edge is created between the two nodes at each timestamp. The time edge represents the historical information frame by frame. Here, when constructing the time edge, not only the connection between the targets at the previous moment and the targets at the next moment is considered, but also the interaction between different targets across time steps is additionally introduced. The known target information is constructed into a cross-time-step 3D space-time relation graph by the above method. In the experiment, the graph structure is constructed by setting different thresholds for many times, and the trajectory prediction effect is verified by calculating the root mean square error (RMSE) of 6 future frames.

[0008] Step 2: Construct an adjacency matrix as the input of the neural network according to the space-time relation graph in step 1, where A0 is a unit matrix representing the self-connection in time, A1 is a space connection adjacency matrix, and E is an edge set. If an edge relationship is established between v it and v jt , the element at this position in the adjacency matrix is 1, otherwise it is 0, as follows:

[0009]

[0010] Step 3: Use the position feedforward network and the graph attention network (GAT) based on the multi-head attention mechanism to model the spatial interaction features between the motion targets across time steps. Through the attention mechanism, the GAT network can assign different learning weights to different neighbor nodes. The importance of the adjacent targets that have a greater influence on the motion target is greater, so that the extracted spatial interaction information is more perfect.

[0011] Specifically, the attention coefficient of the node pair (i, j) in GAT is calculated as follows:

[0012] c(i,j)=a(W*v t (i),W*v t (j))

[0013] Where W is a weight matrix that applies a linear transformation at each node, and a(·) is a function for calculating the correlation between the features of two nodes.

[0014] To maintain the integrity of the structural information, GAT incorporates attention mechanism into the graph structure by masked attention using adjacency matrix. The attention coefficients between the interconnected nodes are calculated by masked attention, which are then normalized between different nodes by softmax function for comparison between selected different nodes. In addition, LeakyReLU function is also applied in the feedforward layer, and the coefficients calculated by attention mechanism can be expressed as follows:

[0015]

[0016] Where || is the connection operation, · T denotes the transpose, is the attention coefficient of node j to i at time step T, denotes the neighbors of node i on the graph. W is the weight matrix of the shared linear transformation applied to each node, and a is the weight vector.

[0017] In order to output the final features of each node at time t, the normalized attention coefficients are used to obtain the linear combination of the corresponding features, which can be expressed as follows:

[0018]

[0019] Where σ is the nonlinear function ELU.

[0020] In order to improve the stability of the learning process, the network combines multi-head attention, uses K attention heads and takes the average of the K outputs to obtain the output features, and the calculation process is as follows:

[0021]

[0022] The application adopts two graph attention networks (GATs) to capture high-level interactions between targets, and applies residual connection to improve the performance of the model.

[0023] Step 4: In order to more fully extract the spatio-temporal graph features, the application additionally constructs a single-step spatio-temporal relationship graph, constructs a spatial interaction graph between targets in each frame scene, introduces a graph convolution network GCN to extract spatial interaction information between targets, and introduces a time convolution network TCN to process time dimension information.

[0024] Specifically, the graph convolutional network first passes through a (1x1) convolutional layer to increase the number of channels. This maps the two-dimensional input data to a higher-dimensional space, which helps the model learn a good representation for the trajectory prediction task. After that, 3 graph convolutional networks (GCNs) and 3 temporal convolutional networks (TCNs) are used to process the interactions between targets in the spatial and temporal dimensions, respectively. To improve the stability of model training, batch normalization layers are also adopted. In addition, skip connections are used to ensure that the model can propagate large gradients to the initial layers and make the learning speed of these layers as fast as the final layers.

[0025] Step 5: The spatio-temporal feature information extracted by the two models is fused through a multi-layer interactive feature fusion network. This fusion module removes the mask self-attention module in the transformer decoder and only retains the multi-head cross-attention and FFN modules. Through cross-attention, the network can learn the global connection, difference, and importance level between the two features; through multi-layer connection and the storage function of FFN, a fusion graph feature compatible with the two inputs can be obtained.

[0026] Step 6: The fused spatio-temporal graph features are used as the input of the ConvGRU-based encoder-decoder, and the motion trajectories of each target at the future time steps are predicted.

[0027] Specifically, the spatial dimension of the spatio-temporal fusion block can be used by the ConvGRU encoder to further capture the spatial information of the target; on the other hand, the temporal dimension of the spatio-temporal fusion block can be used by the ConvGRU encoder to identify the relevant temporal features of each target. The encoder compresses the entire input sequence into a hidden state tensor, which is then used as the initial hidden state of the ConvGRU-based decoder.

[0028] In the ConvGRU-based decoder, the last observation is used as the initial input to predict the velocity of each target at the next time stamp. To ensure consistency in input dimensions, the last observation is first fed into the linear layer of the prediction head for linear transformation. Then the hidden state obtained from the ConvGRU encoder is unfolded for prediction. The output and hidden state are used as the initial input and initial hidden state of the next unit of the ConvGRU decoder. The decoding process is iterated multiple times until the model generates predictions for all expected future time steps.

[0029] Step 7: Model training. The model training loss calculation formula is as follows, where t pred is the prediction time. Y pred and Y GT are the predicted positions and true values, respectively.

[0030]

[0031] The present application adopts average displacement error (ADE) and final displacement error (FDE) to measure the performance of the algorithm.

[0032] ADE is the average Euclidean distance between all predicted positions and ground truth positions within the prediction time, and the specific formula is as follows:

[0033]

[0034] FDE is the displacement error between the final time step true trajectory and the predicted trajectory, and the specific formula is as follows:

[0035]

[0036] Step 8: Complete the model test. Input the test set into the spatio-temporal graph feature fusion model to obtain the result of trajectory prediction.

[0037] The beneficial effects produced by the above technical scheme are that the spatio-temporal graph fusion trajectory prediction method of cross-time step interaction provided by the present application establishes the information interaction between each target in the scene by constructing a cross-time step 3D spatio-temporal relationship graph, and completes spatio-temporal feature extraction by using an improved graph network, greatly improving the prediction accuracy. On the ApolloScape data set, model training and testing are performed, and compared with the minimum error obtained by the S-LSTM, S-GAN trajectory prediction method published in the recent IEEE Conference on Computer Vision and Pattern Recognition (CVPR) and the STAR trajectory prediction method published in the European Conference on Computer Vision (ECCV), the weighted average ADE and FDE are reduced by 28.2% and 42.9%, respectively. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The overall structure diagram provided for the embodiment of the present application;

[0039] Figure 2 The cross-time step 3D spatio-temporal relationship graph provided for the embodiment of the present application;

[0040] Figure 3 The single attention layer schematic diagram provided for the embodiment of the present application;

[0041] Figure 4 The model experiment result comparison diagram provided for the embodiment of the present application; DETAILED DESCRIPTION

[0042] The specific embodiments of the present application are described in further detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0043] As shown in the embodiment, the technical scheme provided by the embodiment is as follows: a space-time graph fusion trajectory prediction method for cross-time step interaction, specifically comprising the following steps: Figure 1

[0044] First, the coordinates of each observed target in the past T h time steps in a certain scene are given, and the future trajectories of each target in the future T p time steps are predicted, wherein X and Y respectively represent the historical trajectory and the future trajectory of each observed target, p (t) indicates the trajectory of each target at time t, and the specific formula is as follows:

[0045]

[0046] Considering that it is easier to predict the speed of an object than to predict its position, the speed (p t+1 -p t ) of each moving object is calculated and input into the subsequent network.

[0047] In order to represent the space-time interaction of different moving objects, the present application constructs a cross-time step 3D space-time relation graph G={V, E}.

[0048] The node set V={v it | i=1, …, n, t=1, …, t h} is constructed, wherein each node represents a target.

[0049] The edge set E can be divided into two parts: time edges E F and space edges E S .

[0050] The time edges are represented by frame-by-frame historical information. When constructing the time edges, not only the connection between each target at the previous time and itself at the next time is considered, but also the interaction relationship between different targets across time steps is additionally introduced, which can be represented as E F ={v it v j(t+1)}.

[0051] The space edges are used to represent the interaction relationship between two targets, which can be represented as E S ={v it v jt ​| (i, j∈D)}, specifically by setting a neighborhood distance threshold D to distinguish the relationship between each target and the surrounding targets, if the distance between two nodes is less than the set neighborhood distance, a spatial edge is created between the two nodes at each timestamp. In the experiment, the graph structure is constructed by setting different thresholds for multiple times, and the trajectory prediction effect is verified by calculating the root mean square error RMSE of 6 future frames, and the neighborhood distance threshold is set to 7.62 meters for the data set used in the application.

[0052] The schematic diagram is as shown in Figure 2 Wherein the solid line represents the spatial edge, and the dashed line represents the time edge.

[0053] In order to improve the calculation efficiency, the application uses a neighborhood matrix to store such space-time connection relationship, wherein A0 is a unit matrix representing self-connection in time, and A1 is a spatial connection neighborhood matrix, as follows:

[0054]

[0055] After processing the original trajectory data, the node features and edge features are obtained as the input of the space-time graph feature fusion network.

[0056] In order to better learn the complex data distribution in the real scene, the application adds a position feedforward network FFN, which mainly adopts multiple linear and nonlinear transformations.

[0057] Specifically, it takes the node features and edge features, and performs forward propagation through two linear layers and an activation function. The first linear layer expands the representation of each position, providing the possibility of learning more complex features. The activation function helps the model learn more complex nonlinear features, as shown in the following formula:

[0058] FFN(v,W1,W2,b1,b2)=max(0,vW1+b1)W2+b2

[0059] Wherein v refers to the input vector, and W1, W2, b1, and b2 refer to learnable parameters.

[0060] After that, the output of the position feedforward network module is taken as the input of the BatchNorm layer to speed up the training process of the model and ensure the stability of the training.

[0061] Then, a graph attention network GAT with attention mechanism is used to realize space-time feature extraction. Through self-attention mechanism, GAT extracts the hidden representation of each node by focusing on the adjacent nodes in the graph.

[0062] The following Figure 3 A single graph attention layer is shown.

[0063] The input of the graph attention layer can be represented as wherein N is the number of nodes and F is the dimension of each node feature.

[0064] GAT network is implemented by stacking the above graph attention layers, wherein each attention layer calculates the attention coefficient of node pair (i, j) as follows:

[0065] c(i, j) = a(W * v t (i), W * v t (j))

[0066] where W is the weight matrix applying linear transformation on each node, and a(·) is a function for calculating the correlation between two node features.

[0067] In order to maintain the integrity of the structure information, GAT incorporates attention mechanism into the graph structure through masked attention by using adjacency matrix.

[0068] Through masked attention, the attention coefficients of the interconnected nodes are calculated, which are then normalized between different nodes by the softmax function so as to compare between different nodes selected

[0069] In addition, LeakyReLU function is also applied in the feedforward layer, and the coefficient calculated by the attention mechanism can be expressed as follows:

[0070]

[0071] where || is the concatenation operation, · T denotes the transpose, is the attention coefficient of node j to i at time step T, denotes the neighbors of node i on the graph. W is the weight matrix of the shared linear transformation applied to each node, and a is the weight vector.

[0072] In order to output the final feature of each node at time t, the normalized attention coefficient is used to obtain the linear combination of the corresponding feature, which can be expressed as follows:

[0073]

[0074] where σ is the nonlinear function ELU.

[0075] In order to improve the stability of the learning process, the network combines multi-head attention, uses K attention heads and takes the average of the K outputs to obtain the output feature, and the calculation process is as follows:

[0076]

[0077] To avoid overfitting, the output features from the GAT network are input into a Dropout layer.

[0078] To keep the input and output feature dimensions unchanged, the dimensions of the graph features are restored to their initial dimensions again using the FFN module at the end.

[0079] The application adopts two graph attention layers to capture high-level interactions between traffic agents, and applies residual connections to improve model performance.

[0080] To more fully extract the spatio-temporal information in the scene, the application additionally constructs a single-step spatio-temporal relationship graph, constructs a spatial interaction graph between targets in each frame of the scene, and introduces three graph convolution models GCN and three time convolution models TCN to process the interactions between targets in the spatial and temporal dimensions.

[0081] The spatio-temporal feature information extracted by the two models is fused through a multi-layer interactive feature fusion network.

[0082] The fusion module mainly consists of a multi-head cross-attention and an FFN.

[0083] For multi-head cross-attention, in addition to internal linear transformation, the calculation of the attention mechanism is as follows, where

[0084]

[0085] The mechanism of the fusion module can be represented as follows:

[0086] IFF(Q,K,V)=X F +FFN(X F )

[0087] X F =Q+MHCA(Q+P q ,K+P kv ,V)

[0088] is the feature output of the graph attention model, K and is the feature output of the graph convolution model. and are spatial position encodings. MHCA(.,.,.) is a multi-head cross-attention mechanism.

[0089] Through cross-attention, the network can learn the global connection, difference and importance level between the two features.

[0090] Through multi-layer connection and the storage function of FFN, a fused graph feature compatible with the two inputs can be obtained.

[0091] The fused graph features are taken as inputs of the ConvGRU-based encoder.

[0092] On the one hand, the spatial dimension of the spatio-temporal graph features is utilized to further capture the spatial information of multiple targets by the encoder; on the other hand, the temporal dimension of the spatio-temporal graph features is utilized to identify the relevant temporal features of each target by the encoder.

[0093] The key equations of the ConvGRU are shown as follows:

[0094]

[0095] where r t , z t , and h are the reset gate, update gate, candidate hidden state and final hidden state of the encoder at time t, respectively. * U is the weight matrix of the input, and U * is the weight matrix of the previous hidden state h t-1 . b * is the bias learned by the model. The operator is the element-wise vector product. and denote the sigmoid activation operation and tanh activation operation, respectively. sigmoid tanh

[0096] Finally, the encoder compresses the entire input sequence into a hidden state tensor, which is then used as the initial hidden state of the ConvGRU-based decoder.

[0097] In the ConvGRU-based decoder, the last observation is used as the initial input to predict the velocity of each moving target at the next time stamp.

[0098] To ensure the consistency of the input dimension of the ConvGRU, the last observation is first fed into the linear layer of the prediction head for linear transformation. Then the hidden state obtained from the ConvGRU encoder is unfolded for prediction, and its output and hidden state are used as the initial input and initial hidden state of the next unit of the ConvGRU decoder. The decoding process is iterated multiple times until the model generates predictions for all expected future time steps.

[0099] Given that only a few traffic objects maintain constant velocities, we impose the requirement of predicting velocity changes on the model. This is achieved by adding a residual connection between the input and output of each ConvGRU decoder unit. Therefore, the model can accurately capture and predict changes in velocity.

[0100] ​​Once the prediction results of the trajectory prediction module are obtained, the average value of the predicted speed of each time step can be calculated. Then it can be merged into the previous historical position to convert the prediction results into coordinates.

[0101] The ApolloScape trajectory prediction dataset is used in the application, which mixes vehicles, cyclists and pedestrians, contains various types of traffic participants, and the interaction process is complex. The dataset includes 53 minutes of training sequences and 50 minutes of test sequences, each sequence contains frame number, target type, number, position and angle, 2 frames are captured per second, and 6 future frames need to be predicted according to 6 historical frames. The proposed model is trained and tested on the dataset, the epoch is set to 200 during training, the batch size is 32, the model is optimized using the optimizer Adam, the initial learning rate is 0.001, and the model training loss calculation formula is as follows:

[0102]

[0103] Where t pred is the prediction time. Y pred and Y GT are the predicted position and the true value respectively.

[0104] The application adopts average displacement error (ADE) and final displacement error (FDE) to measure the performance of the algorithm. ADE is the average Euclidean distance between all predicted positions and ground truth positions within the prediction time, and FDE is the displacement error between the final time step true trajectory and the predicted trajectory.

[0105] Since the trajectories of different types of targets are different in scale, weighted ADE and weighted FDE are also used as metrics:

[0106] WSADE=D v ·ADE v +D p ·ADE p +D b ·ADE b

[0107] WSFDE=D v ·FDE v +D p ·FDE p +D b ·FDE b

[0108] Here D v =0.20, D p =0.58, D b =0.22 are the inverses of the average speeds of vehicles, pedestrians and cyclists in the dataset respectively.

[0109] To evaluate the performance of the model, it is compared with many baseline networks, and the experimental results are as shown in the following table: Figure 4 From the table, it can be seen that the model of the present application shows better performance than other models in all indicators in all aspects.

[0110] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A spatiotemporal graph fusion trajectory prediction method with cross-time step interaction, characterized in that: By constructing a 3D spatiotemporal relationship graph across time steps, information interaction between various targets in the scene across time steps is established, and an improved graph network is used to complete spatiotemporal feature extraction, which greatly improves the accuracy of prediction. The specific steps are as follows: Step 1: Construct a 3D spatiotemporal relationship graph spanning time steps; Step 2: Construct an adjacency matrix based on the spatiotemporal relationship graph as input to the neural network; Step 3: Simultaneously extract spatiotemporal interaction features using a location-feedforward network and a graph attention network (GAT) based on a multi-head attention mechanism; Step 4: In order to extract spatiotemporal graph features more fully, a graph convolutional network (GCN) and a temporal convolutional network (TCN) are introduced to handle the interactions between targets in the spatial and temporal dimensions, respectively. Step 5: Fuse the spatiotemporal feature information extracted by the two models. The fusion module removes the mask self-attention module in the Transformer decoder and retains only the multi-head cross attention and FFN module. Through cross-attention, the network can learn the global connectivity, distinction, and importance level between two features; through multi-layer connections and the storage function of FFN, fused graph features compatible with both inputs can be obtained. Step 6: Use the fused spatiotemporal graph features as input to the ConvGRU-based encoder and decoder, and predict the motion trajectories of each target in future time steps. Step 7: Train the model using the training and validation datasets to determine the parameters of the network model; Step 8: Use the trained network model to predict trajectories on the test dataset; Step 1 includes the following: A 3D spatiotemporal graph G = {V, E} spanning time steps was constructed; the node set V = {v it |i=1,…,n,t=1,…,t h }, where each node represents a goal; the edge set E can be divided into time edges E F and space edge E S Two parts; the time edge is represented by frame-by-frame historical information, which not only considers connecting each target in the previous time step with itself in the next time step, but also introduces the interaction relationship between different targets across time steps, which can be represented as E. F ={v it v j(t+1) Spatial edges are used to represent the interaction relationship between two targets, and can be represented as E. S ={v it v jt |(i,j∈D)}, by setting an adjacency distance threshold D, the relationship between each target and its surrounding targets is distinguished. If the distance between two nodes is less than the set adjacency distance, a spatial edge is created between the two nodes at each timestamp. The graph structure is constructed by setting different thresholds multiple times, and the trajectory prediction effect is verified by calculating the mean square error (RMSE) of k future frames. To improve computational efficiency, an adjacency matrix is ​​used to store this spatiotemporal connection relationship. A0 is the identity matrix representing temporal self-connection, A1 is the spatial connection adjacency matrix, and E is the edge set. If v it ,v jt If an edge relationship is established between two points, the element at that position in the adjacency matrix is ​​1; otherwise, it is 0, as shown in the following formula: Step 3 includes the following: A location-based feedforward network (FFN) was added, primarily employing multiple linear and nonlinear transformations. Node and edge features were extracted and forward propagated through two linear layers and an activation function. The linear layers expanded the representation at each location, and the activation function learned nonlinear features, as shown in the following equation: FFN(v,W1,W2,b1,b2)=max(0,vW1+b1)W2+b2 Here, v refers to the input vector, and W1, W2, b1, b2 refer to the learnable parameters. The output of the position feedforward network module is used as the input to the BatchNorm layer to accelerate model training. A graph attention network (GAT) with an attention mechanism is used to extract spatiotemporal features. Through the attention mechanism, the GAT network can assign different learning weights to different neighbor nodes, improving the network's ability to extract spatial interaction information. Two graph attention networks (GATs) are used to capture high-level interactions between targets, and residual connections are applied to improve model performance. Step 5 includes the following: The feature fusion module mainly consists of two parts: multi-head cross-attention and FFN. For multi-head cross-attention, in addition to the internal linear transformation, the calculation of the attention mechanism is as follows: The mechanism of the fusion module is expressed as follows: IFF(Q,K,V)=X F +FFN(X F ) X F =Q+MHCA(Q+P q ,K+P kv ,V) The feature outputs of the graph attention model, K and It is the feature output of the graph convolution model. and It is a spatial location encoding, and MHCA(.,.,.) is a multi-head cross attention mechanism; the fused graph features are used as the input to the subsequent ConvGRU-based trajectory prediction module.

2. The spatiotemporal graph fusion trajectory prediction method with cross-time step interaction according to claim 1, characterized in that: Step 7 includes the following: The loss calculation formula is as follows, where t pred To predict time, Y pred and Y GT These are the predicted location and the true value, respectively. The performance of the algorithm is measured using the average displacement error (ADE) and the final displacement error (FDE). ADE is the average Euclidean distance between all predicted locations and the ground truth locations within the prediction time, as detailed below: FDE represents the displacement error between the actual trajectory and the predicted trajectory at the final time step, as detailed below:

Images