Cross-time-step interaction space-time diagram fusion trajectory prediction method

Abstract

The invention relates to a cross-time-step interaction space-time diagram fusion trajectory prediction method, belongs to the technical field of trajectory prediction, and solves the problem that a conventional model is insufficient in cross-time-step interaction feature extraction. According to the technical scheme, the method comprises the following steps: firstly, constructing a cross-time-step 3D space-time relation graph to represent information interaction of each target in a scene on a space domain and a time domain; then, cross-time-step interaction information is extracted by adopting a position feedforward network and a graph attention network; meanwhile, constructing a single-step space-time relation graph, introducing a graph convolutional network and a time convolutional network, respectively extracting spatial features at the current moment and a time dependency relationship of each target, and fusing the graph features of the two models by adopting a feature fusion network; and finally, outputting a prediction track of each target through a ConvGRU codec. Compared with other recent methods, the method has the advantages that the average ADE and FDE indexes are obviously reduced, and the trajectory prediction accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the field of trajectory prediction of moving objects, and in particular relates to a space-time graph fusion trajectory prediction method with cross-time step interaction. Background Art

[0002] Trajectory prediction technology mainly refers to predicting the future movement trends of traffic participants based on their historical trajectories and high-precision maps. Accurate and effective trajectory prediction can guide subsequent decision-making and planning, and improve the safety and reliability of the vehicle during driving.

[0003] At present, there are two main types of trajectory prediction research: 1) Modeling based on the dynamics or kinematic constraints of moving objects to predict their future motion state, but the premise of using this type of method is to assume that the vehicle state and trajectory will not change dramatically in a short period of time, so it has certain limitations when facing complex traffic scenes. 2) Trajectory prediction methods based on deep learning. With the continuous increase in data volume and computer computing power, trajectory prediction methods based on deep learning are data-driven, and at the same time combine the vehicle's surrounding map perception information and historical trajectory characteristics, so that the model can better adapt to complex road conditions, and gradually become the mainstream of the industry.

[0004] Among many deep learning trajectory prediction methods, prediction algorithms with LSTM and GRU networks in recurrent neural networks as the core architecture are widely used. Patent CN116872963A provides a method for vehicle trajectory prediction model based on LSTM, including an encoder and decoder built based on LSTM; Patent CN113408588A proposes a bidirectional GRU trajectory prediction based on attention mechanism, which uses the attention mechanism to mine important features and reduce the loss of key information, thereby effectively improving the accuracy of the prediction model, and combines the WOA algorithm to optimize the weights in the forward and backward propagation process of the bidirectional GRU neural network, but they only focus on the motion information of the target itself. When facing complex traffic scenes such as multiple targets coexisting, more accurate results cannot be obtained. Therefore, it is necessary to extract the interactive feature information in the scene before trajectory prediction. Among them, the graph neural network (GNN) represents the interaction between targets in the scene by constructing a graph structure, providing richer spatiotemporal features for subsequent trajectory prediction. Patent CN115147790A provides a multimodal trajectory prediction algorithm based on graph neural network (GNN). The road map and the time series of the vehicle to be predicted will be input into the deep learning model to output the predicted trajectory; Patent CN113505878A inputs the track sequence data set to be detected into the graph neural network model to obtain the track prediction trajectory; Patent CN113869170A designs a set of pedestrian trajectory prediction models for complex scenes, based on the graph partitioning convolutional neural network GP-CNN, which uses a combination of two channels to extract the interactive features of the scene, and then obtains the predicted trajectory through multiple CNN layers.

[0005] Most of the above graph-based studies consider surrounding traffic participants equally, which cannot effectively capture the information interaction with the greatest relationship between targets. Therefore, many scholars at home and abroad use attention-based methods to capture important interactions between targets based on graph networks. For example, patent CN113888638A uses a spatiotemporal graph convolutional neural network to extract the spatiotemporal features of pedestrian trajectory node graphs, and uses the graph channel attention mechanism to screen out important spatiotemporal features and input them into the predictor to obtain future trajectories. However, most existing methods still extract temporal and spatial information in the scene separately, ignoring the spatial interactions between moving objects across time steps. Summary of the invention

[0006] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a cross-time-step interactive spatiotemporal graph fusion trajectory prediction method, by constructing a cross-time-step 3D spatiotemporal relationship graph, focusing on both spatial correlation and temporal correlation in the feature extraction module, and focusing on adjacent targets that have a greater impact on the moving target through an attention mechanism, specifically using the following steps:

[0007] Step 1: Construct a 3D spatiotemporal relationship graph across time steps. The graph consists of two parts: nodes and edges. One target represents one node, and the edges are divided into two parts: spatial edges and temporal edges. The spatial edges are used to represent the interactive relationship between two targets. The present invention distinguishes the relationship between each target and the surrounding targets by setting an adjacent distance threshold. If the distance between two nodes is less than the set adjacent distance, a spatial edge is created between the two nodes at each timestamp; and the temporal edge is represented by frame-by-frame historical information. Here, when constructing the temporal edge, not only is it considered to connect each target at the previous moment with itself at the next moment, but also the interactive relationship between different targets across time steps is additionally introduced. The known target information is constructed into a 3D spatiotemporal relationship graph across time steps through the above method. In the experiment, the graph structure is constructed by setting different thresholds multiple times and the trajectory prediction effect is verified by calculating the mean square error RMSE of 6 future frames.

[0008] Step 2: Construct an adjacency matrix based on the spatiotemporal relationship graph in step 1 as the neural network input, where A0 is the identity matrix representing the self-connection in time, A1 is the spatial connection adjacency matrix, and E is the edge set. it ,v jt If an edge relationship is established between them, the element at that position in the adjacency matrix is ​​1, otherwise it is 0, as shown in the following formula:

[0009] Step 3: Use the position feedforward network and the graph attention network GAT based on the multi-head attention mechanism to model the spatial interaction characteristics between moving targets across time steps. Through the attention mechanism, the GAT network can assign different learning weights to different neighbor nodes. The neighboring targets that have a greater impact on the moving target itself are more important, thereby making the extracted spatial interaction information more complete.

[0010] Specifically, each attention layer in GAT calculates the attention coefficient of node pair (i, j) as follows: c(i,j)=a(W*v t (i),W*v t (j)) where W is the weight matrix for applying the linear transformation on each node and a(·) is the function used to calculate the correlation between two node features.

[0011] In order to maintain the integrity of the structural information, GAT incorporates the attention mechanism into the graph structure through masked attention using the adjacency matrix. The attention coefficients between interconnected nodes are calculated through masked attention, and these coefficients are then normalized between different nodes through the softmax function so that they can be compared between different selected nodes. In addition, the LeakyReLU function is applied in the feedforward layer, and the coefficients calculated by the attention mechanism can be expressed as follows: where || is a join operation, T represents transpose, is the attention coefficient from node j to i at time step T, represents the neighbors of node i on the graph. W is the weight matrix of the shared linear transformation applied to each node, and a is the weight vector. In order to output the final features of each node at time t, the normalized attention coefficient is used to obtain the linear combination of the corresponding features, which can be expressed as follows: Where σ is the nonlinear function ELU.

[0012] In order to improve the stability of the learning process, the network integrates multi-head attention, uses K attention heads and takes the average of its K outputs to obtain the output features. The calculation process is shown in the following formula:

[0013] The present invention adopts two graph attention networks (GATs) to capture high-level interactions between objects and applies residual connections to improve model performance.

[0014] Step 4: In order to more fully extract the spatiotemporal graph features, the present invention further constructs a single-step spatiotemporal relationship graph, constructs a spatial interaction graph between targets in each frame scene, introduces a graph convolutional network (GCN) to extract spatial interaction information between targets, and introduces a temporal convolutional network (TCN) to process temporal dimension information.

[0015] Specifically, the graph convolutional network first passes it through a (1×1) convolutional layer to increase the number of channels. This maps the two-dimensional input data to a higher-dimensional space, which helps the model learn good representations for the trajectory prediction task. Afterwards, three graph convolutional networks (GCNs) and three temporal convolutional networks (TCNs) are used to handle the interactions between objects in the spatial and temporal dimensions, respectively. In order to improve the stability of model training, a batch normalization layer is also used. In addition, skip connections are used to ensure that the model can propagate large gradients to the initial layers and make these layers learn as fast as the final layers.

[0016] Step 5: The spatiotemporal feature information extracted by the two models is fused through a multi-layer interactive feature fusion network. This fusion module removes the mask self-attention module in the transformer decoder and only retains the multi-head cross attention and FFN modules. Through cross attention, the network can learn the global connection, distinction, and importance level between the two features; through multi-layer connections and the storage function of FFN, a fusion graph feature compatible with the two inputs can be obtained.

[0017] Step 6: Use the fused spatiotemporal graph features as the input of the ConvGRU-based encoder-decoder and predict the motion trajectory of each target in the future time step.

[0018] Specifically, using the spatial dimension of the spatiotemporal fusion block, the ConvGRU encoder can further capture the spatial information of the target; on the other hand, using the temporal dimension of the spatiotemporal fusion block, the ConvGRU encoder can identify the relevant temporal features of each target. The encoder compresses the entire input sequence into a hidden state tensor, which is then used as the initial hidden state of the ConvGRU-based decoder.

[0019] In the ConvGRU based decoder, the last observation is used as the initial input to predict the velocity of each object at the next time stamp. To ensure the consistency of the input dimension, the last observation is first fed into the linear layer of the prediction head for linear transformation. The prediction is then made by unfolding the hidden state obtained from the ConvGRU encoder. The output and hidden state are used as the initial input and initial hidden state of the next unit of the ConvGRU decoder. The decoding process is iterated multiple times until the model generates predictions for all expected future time steps.

[0020] Step 7: Perform model training. The model training loss calculation formula is as follows, where t pred Y is the prediction time. pred and Y GT are the predicted position and the true value respectively.

[0021] The present invention uses average displacement error (ADE) and final displacement error (FDE) to measure the performance of the algorithm.

[0022] ADE is the average Euclidean distance between all predicted positions and ground truth positions within the prediction time. The specific formula is as follows:

[0023] FDE is the displacement error between the actual trajectory and the predicted trajectory at the final time step. The specific formula is as follows:

[0024] Step 8: Complete model testing. Input the test set into the spatiotemporal graph feature fusion model to obtain the trajectory prediction result.

[0025] The beneficial effect of adopting the above technical solution is that a cross-time-step interactive spatiotemporal graph fusion trajectory prediction method provided by the present invention establishes information interaction between targets in the scene by constructing a cross-time-step 3D spatiotemporal relationship graph, and uses an improved graph network to complete spatiotemporal feature extraction, thereby greatly improving the prediction accuracy. The model is trained and tested on the ApolloScape dataset. Compared with the lowest errors achieved by trajectory prediction methods such as S-LSTM, S-GAN published at the recent IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) and STAR published at the European Conference on Computer Vision (ECCV), the weighted average ADE and FDE are reduced by 28.2% and 42.9% respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 An overall structural diagram provided for an embodiment of the present invention;

[0027] Figure 2 A 3D spatiotemporal relationship diagram across time steps provided by an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of a single attention layer provided for an embodiment of the present invention;

[0029] Figure 4 A comparison chart of model experiment results provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0030] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0031] like Figure 1 As shown, the technical solution provided by this embodiment is: a cross-time step interactive spatiotemporal graph fusion trajectory prediction method, which specifically includes the following steps:

[0032] First, given a certain scene, the past T h The coordinates of each observed target in time steps are used to predict the future T p The future trajectory of each target at time steps, where X and Y refer to the historical trajectory and future trajectory of each observed target, respectively. (t) The trajectory composition of each target at time t is explained as follows:

[0033] Considering that it is easier to predict the speed of an object than its position, the speed (p t+1 -p t ) and enter the subsequent network.

[0034] In order to represent the spatiotemporal interaction of different moving objects, the present invention constructs a 3D spatiotemporal relationship graph G={V, E} across time steps.

[0035] Node set V = {v it |i=1,…,n,t=1,…,t h}, where each node represents a target.

[0036] The edge set E can be divided into time edges E F and the spatial edge E S Two parts.

[0037] The temporal edge is represented by frame-by-frame historical information. Here, when constructing the temporal edge, we not only consider connecting each target at the previous moment with itself at the next moment, but also introduce the interaction relationship between different targets across time steps, which can be expressed as E F = {v it v j(t+1)}.

[0038] The spatial edge is used to represent the interaction relationship between two targets and can be expressed as E S = {v it v jt |(i,j∈D)}, specifically, by setting an adjacent distance threshold D to distinguish the relationship between each target and the surrounding targets, if the distance between two nodes is less than the set adjacent distance, a spatial edge is created between the two nodes at each timestamp. In the experiment, the graph structure is constructed by setting different thresholds multiple times and the trajectory prediction effect is verified by calculating the mean square error RMSE of 6 future frames. For the dataset used in the present invention, the adjacent distance threshold is set to 7.62 meters.

[0039] Its schematic diagram is as follows Figure 2 , where the solid lines represent spatial edges and the dashed lines represent temporal edges.

[0040] In order to improve the computational efficiency, the present invention uses an adjacency matrix to store this spatiotemporal connection relationship, where A0 is the identity matrix representing the temporal self-connection, and A1 is the spatial connection adjacency matrix, as shown in the following formula:

[0041] After processing the original trajectory data, the node features and edge features are obtained as the input of the spatiotemporal graph feature fusion network.

[0042] In order to better learn the complex data distribution in real scenes, the present invention adds a position feed-forward network FFN, which mainly adopts the form of multiple linear and nonlinear transformations.

[0043] Specifically, it takes node features and edge features and forward propagates through two linear layers and an activation function. The first linear layer expands the representation of each position, making it possible to learn more complex features. The activation function helps the model learn more complex nonlinear features, as shown in the following formula: FFN(v,W1,W2,b1,b2)=max(0,vW1+b1)W2+b2

[0044] Where v refers to the input vector, and W1, W2, b1, b2 refer to learnable parameters.

[0045] Afterwards, the output of the position feedforward network module is used as the input of the BatchNorm layer to accelerate the training process of the model and ensure the stability of the training.

[0046] Then, the graph attention network GAT with attention mechanism is used to realize spatiotemporal feature extraction. Through the self-attention mechanism, GAT extracts the hidden representation of each node by paying attention to the adjacent nodes in the graph.

[0047] Down Figure 3 Shown is a single graph attention layer.

[0048] The input of the graph attention layer can be expressed as in N is the number of nodes, and F is the dimension of each node feature.

[0049] The GAT network is implemented by stacking the above graph attention layers, where each attention layer calculates the attention coefficient of the node pair (i, j) as follows: c(i,j)=a(W*v t (i),W*v t (j))

[0050] where W is the weight matrix for applying the linear transformation on each node and a(·) is the function used to calculate the correlation between two node features.

[0051] In order to keep the integrity of the structural information, GAT utilizes the adjacency matrix to incorporate the attention mechanism into the graph structure through masked attention.

[0052] Through masked attention, attention coefficients of interconnected nodes are calculated, which are then normalized between different nodes through a softmax function so that they can be compared between different selected nodes.

[0053] In addition, the LeakyReLU function is applied in the feed-forward layer, and the coefficients calculated by the attention mechanism can be expressed as follows:

[0054] where || is a join operation, T represents transpose, is the attention coefficient from node j to i at time step T, represents the neighbors of node i on the graph. W is the weight matrix of the shared linear transformation applied to each node, and a is the weight vector.

[0055] In order to output the final features of each node at time t, the normalized attention coefficient is used to obtain the linear combination of the corresponding features, which can be expressed as follows: Where σ is the nonlinear function ELU.

[0056] In order to improve the stability of the learning process, the network integrates multi-head attention, uses K attention heads and takes the average of its K outputs to obtain the output features. The calculation process is shown in the following formula:

[0057] To avoid overfitting, the output features from the GAT network are input into the Dropout layer.

[0058] In order to keep the input and output feature dimensions unchanged, the FFN module is finally used again to restore the dimensions of the graph features to their initial dimensions.

[0059] The present invention adopts two graph attention layers to capture the high-level interactions between traffic agents and applies residual connections to improve model performance.

[0060] In order to more fully extract the spatiotemporal information in the scene, the present invention further constructs a single-step spatiotemporal relationship graph, constructs a spatial interaction graph between targets in each frame of the scene, and introduces three graph convolution models GCN and three temporal convolution models TCN to process the interactions between targets in the spatial and temporal dimensions respectively.

[0061] The spatiotemporal feature information extracted by the two models is fused through a multi-layer interactive feature fusion network.

[0062] The fusion module mainly consists of two parts: multi-head cross attention and FFN.

[0063] For multi-head cross attention, in addition to the internal linear transformation, the calculation of the attention mechanism is as follows, where

[0064] The mechanism of action of this fusion module can be expressed as follows: IFF(Q,K,V)=X F +FFN(X F ) X F =Q+MHCA(Q+P q ,K+P kv ,V)

[0065] is the feature output of the graph attention model, K and It is the feature output of the graph convolutional model. and is the spatial position encoding. MHCA(.,.,.) is the multi-head cross attention mechanism.

[0066] Through cross-attention, the network can learn the global connection, distinction, and importance level between two features.

[0067] Through multi-layer connections and the storage function of FFN, fused graph features compatible with both inputs can be obtained.

[0068] The present invention uses the fused graph features as the input of the ConvGRU-based encoder.

[0069] On the one hand, by utilizing the spatial dimension of the spatiotemporal graph features, the encoder can further capture the spatial information of multiple targets; on the other hand, by utilizing the temporal dimension of the spatiotemporal graph features, the encoder can identify the relevant temporal features of each target.

[0070] The key equation of ConvGRU is shown below:

[0071] where r t 、z t , and are the reset gate, update gate, candidate hidden state and final hidden state of the encoder at time t. * Yes Input The weight matrix, U * is the previous hidden state h t-1 The weight matrix of b. * is the bias learned by the model. The operator ⊙ is the element-wise vector product. σ sigmoid and σ tanh Represent the sigmoid activation operation and tanh activation operation respectively.

[0072] Finally, the encoder compresses the entire input sequence into a hidden state tensor, which is then used as the initial hidden state of the ConvGRU-based decoder.

[0073] In the ConvGRU-based decoder, the last observation is used as the initial input to predict the velocity of each moving object at the next timestamp.

[0074] To ensure the consistency of the input dimensions of the ConvGRU, the last observation is first fed into the linear layer of the prediction head for linear transformation. Prediction is then made by unfolding the hidden state obtained from the ConvGRU encoder, whose output and hidden state are used as the initial input and initial hidden state of the next unit of the ConvGRU decoder. The decoding process is iterated multiple times until the model generates predictions for all expected future time steps.

[0075] Given that only a few traffic objects maintain a constant speed, we impose the requirement on the model to predict speed changes. This is achieved by adding a residual connection between the input and output of each ConvGRU decoder unit. As a result, the model can accurately capture and predict changes in speed.

[0076] Once the predictions from the trajectory prediction module are obtained, the average of the predicted speed at each time step can be calculated. This can then be merged into the previous historical positions to convert the predictions into coordinates.

[0077] The present invention uses the ApolloScape trajectory prediction dataset, which is a mixture of vehicles, cyclists and pedestrians, with various types of traffic participants and complex interaction processes. This dataset includes 53 minutes of training sequences and 50 minutes of test sequences. Each sequence contains the number of frames, target type, number, position and angle. Two frames are captured per second, and six future frames need to be predicted based on six historical frames. The proposed model is trained and tested on this dataset. The epoch is set to 200 and the batch size is 32 during training. The optimizer Adam is used to optimize the model. The initial learning rate is 0.001. The model training loss calculation formula is as follows:

[0078] where t pred Y is the prediction time. pred and Y GT are the predicted position and the true value respectively.

[0079] The present invention uses the average displacement error (ADE) and the final displacement error (FDE) to measure the performance of the algorithm. ADE is the average Euclidean distance between all predicted positions and the ground truth position within the prediction time, and FDE is the displacement error between the real trajectory and the predicted trajectory at the final time step.

[0080] Since the trajectories of different types of targets are different in scale, weighted ADE and weighted FDE are also used as metrics: WSADE=D v ·ADE v +D p ·ADE p +D b ·ADE b WSFDE=D v ·FDE v +D p ·FDE p +D b ·FDE b

[0081] Here D v =0.20,D p =0.58,D b = 0.22 are the reciprocals of the average speeds of vehicles, pedestrians, and cyclists in the dataset, respectively.

[0082] In order to evaluate the performance of this model, we compare it with many baseline networks. The experimental results are as follows: Figure 4 As shown, it can be seen that the model of the present invention shows better performance than other models in all aspects in all indicators.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A cross-time-step interactive spatiotemporal graph fusion trajectory prediction method, characterized by: By constructing a 3D spatiotemporal relationship graph across time steps, the information interaction between the targets in the scene across time steps is established, and the spatiotemporal feature extraction is completed using the improved graph network, which greatly improves the accuracy of the prediction. The specific steps are as follows: Step 1: Construct a 3D spatiotemporal relationship graph across time steps. Step 2: Construct an adjacency matrix based on the spatiotemporal relationship graph as the neural network input. Step 3: Use the position feedforward network and the graph attention network GAT based on the multi-head attention mechanism to simultaneously extract spatiotemporal interaction features. Step 4: In order to more fully extract the spatiotemporal graph features, the graph convolutional network GCN and the temporal convolutional network TCN are introduced to handle the interactions between targets in the spatial and temporal dimensions respectively. Step 5: Fuse the spatiotemporal feature information extracted by the two models. This fusion module removes the masked self-attention module in the Transformer decoder and only retains the multi-head cross attention and FFN modules. Through cross attention, the network can learn the global connection, distinction, and importance level between the two features; through multi-layer connections and the storage function of FFN, a fusion graph feature compatible with the two inputs can be obtained. Step 6: Use the fused spatiotemporal graph features as the input of the ConvGRU-based encoder-decoder and predict the motion trajectory of each target in the future time step. Step 7: Use the training data set and the validation data set to train the model and determine the parameters of the network model. Step 8: Use the trained network model to perform trajectory prediction on the test dataset.

2. The cross-time-step interactive spatiotemporal graph fusion trajectory prediction method according to claim 1 is characterized by: Step 1 specifically includes the following: Considering the cross-time-step interaction relationship between various targets in the time domain, the present invention constructs a cross-time-step 3D spatiotemporal relationship graph G = {V, E}. The node set V = {v it |i=1,…,n,t=1,…,t h }, where each node represents a target. The edge set E can be divided into time edges E F and the spatial edge E S The time edge is represented by frame-by-frame historical information. When constructing the time edge, we not only consider connecting each target at the previous moment with itself at the next moment, but also introduce the interaction relationship between different targets across time steps, which can be expressed as E F = {v it v j(t+1) }. The spatial edge is used to represent the interaction between two targets and can be expressed as E S = {v it v jt |(i,j∈D)}, specifically, by setting an adjacency distance threshold D to distinguish the relationship between each target and the surrounding targets, if the distance between two nodes is less than the set adjacency distance, a spatial edge is created between the two nodes at each timestamp. The present invention constructs a graph structure by setting different thresholds multiple times and verifies the trajectory prediction effect by calculating the mean square error RMSE of k (k=6 in the present invention) future frames. In order to improve the computational efficiency, the present invention uses an adjacency matrix to store this spatiotemporal connection relationship, where A0 is the identity matrix representing the temporal self-connection, A1 is the spatial connection adjacency matrix, and E is the edge set. it ,v jt If an edge relationship is established between them, the element at that position in the adjacency matrix is ​​1, otherwise it is 0, as shown in the following formula:

3. The cross-time-step interactive spatiotemporal graph fusion trajectory prediction method according to claim 1 is characterized by: Step 3 specifically includes the following: In order to better learn the complex data distribution in real-world scenarios, the present invention adds a position feed-forward network FFN, which mainly adopts the form of multiple linear and nonlinear transformations. Specifically, it takes node features and edge features and forward propagates through two linear layers and an activation function. The first linear layer expands the representation of each position, providing the possibility of learning more complex features. The activation function helps the model learn more complex nonlinear features, as shown in the following formula: FFN(v,W1,W2,b1,b2)=max(0,vW1+b1)W2+b2 Where v refers to the input vector, and W1, W2, b1, b2 refer to learnable parameters. Afterwards, the output of the position feedforward network module is used as the input of the BatchNorm layer to accelerate the training process of the model and ensure the stability of the training. Then, the graph attention network (GAT) with an attention mechanism is used to realize spatiotemporal feature extraction. Through the attention mechanism, the GAT network can assign different learning weights to different neighbor nodes, and the neighboring targets that have a greater impact on the moving target itself are more important, so that the extracted spatial interaction information is more complete. In order to improve the stability of the learning process, the GAT network integrates multi-head attention, uses K attention heads and takes the average of their K outputs to obtain the output features. In order to avoid overfitting, the output features obtained from the GAT network are input into the Dropout layer. In order to keep the input and output feature dimensions unchanged, the FFN module is finally used again to restore the dimensions of the graph features to their initial dimensions. The present invention adopts two graph attention networks (GATs) to capture high-level interactions between objects and applies residual connections to improve model performance.

4. The cross-time-step interactive spatiotemporal graph fusion trajectory prediction method according to claim 1 is characterized by: Step 5 specifically includes the following: The feature fusion module is mainly composed of two parts: multi-head cross attention and FFN. For multi-head cross attention, in addition to the internal linear transformation, the calculation of the attention mechanism is as follows, where The mechanism of action of this fusion module can be expressed as follows: IFF(Q,K,V)=X F +FFN(X F ) X F =Q+MHCA(Q+P q ,K+P kv ,V) is the feature output of the graph attention model, K and It is the feature output of the graph convolutional model. and is the spatial position encoding. MHCA(.,.,.) is the multi-head cross attention mechanism. The fused graph features are used as the input of the subsequent ConvGRU-based trajectory prediction module.

5. The cross-time-step interactive spatiotemporal graph fusion trajectory prediction method according to claim 1 is characterized by: Step 7 specifically includes the following: The proposed model is trained on the Apolloscape trajectory dataset, and the loss calculation formula is as follows, where t pred Y is the prediction time. pred and Y GT are the predicted position and the true value respectively. The present invention uses average displacement error (ADE) and final displacement error (FDE) to measure the performance of the algorithm. ADE is the average Euclidean distance between all predicted positions and the ground truth positions within the prediction time. The specific formula is as follows: FDE is the displacement error between the actual trajectory and the predicted trajectory at the final time step. The specific formula is as follows: The Adam optimization method is used to update the parameters of each module of the network model.

Images