A trajectory prediction method, device, equipment and medium based on spatiotemporal attention

Abstract

The present application discloses a trajectory prediction method, apparatus, device and medium based on spatiotemporal attention, relating to the field of trajectory prediction. The method comprises constructing a spatiotemporal graph at the current moment according to a target in a target scene; inputting the spatiotemporal graph at the current moment into a trajectory prediction model to obtain the position of the target in the target scene at the next moment; wherein the trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network comprises: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on a recurrent neural network; the state node transition layer is constructed based on a multi-head attention mechanism and a gated recurrent unit. The present application can improve the trajectory prediction accuracy of the target.

Description

Technical Field

[0001] The present application relates to the field of trajectory prediction, and in particular to a trajectory prediction method, apparatus, device, and medium based on spatiotemporal attention. Background Art

[0002] In recent years, spatiotemporal prediction systems have garnered increasing attention, particularly in areas such as autonomous driving, intelligent surveillance, and pedestrian trajectory prediction. As various application scenarios become increasingly complex and intelligent, accurately predicting a target's future trajectory has become a crucial research topic. In these scenarios, a target's trajectory is influenced by a variety of factors, including its historical trajectory, dynamic changes in its spatial position, and interference from the surrounding environment.

[0003] Conventional trajectory prediction methods still suffer from low accuracy and computational efficiency when dealing with high-density scenarios and dynamic interactions among multiple targets. For example, in densely populated environments, the interactions between different targets are complex and dynamic, making it difficult for conventional trajectory prediction methods to capture these complex spatial dependencies, resulting in large prediction errors. Furthermore, as the prediction time step increases, prediction accuracy gradually decreases, making the model unable to effectively handle long-term trajectory prediction tasks. Summary of the Invention

[0004] The purpose of this application is to provide a trajectory prediction method, device, equipment and medium based on spatiotemporal attention, which can improve the trajectory prediction accuracy of the target.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a trajectory prediction method based on spatiotemporal attention, comprising:

[0007] A spatiotemporal graph is constructed at the current moment based on the targets in the target scene; the spatiotemporal graph includes a node information set and an edge information set; the nodes in the node information set represent the positions of the targets at a certain moment; the edge information set includes spatial edges, temporal edges, and implicit edges; the spatial edges represent the relative positional relationships between different targets at the same moment; the temporal edges represent the motion change relationship of the same target between adjacent time steps; and the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship.

[0008] Input the current time-space graph into the trajectory prediction model to obtain the position of the target in the target scene at the next moment;

[0009] The trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on a recurrent neural network; the state node transition layer is constructed based on a multi-head attention mechanism and a gated recurrent unit;

[0010] The edge coding layer is used to generate the hidden state at the current moment based on the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment based on the position of the target at the current moment, the edge information set at the current moment, and the hidden state at the current moment.

[0011] In a second aspect, the present application provides a trajectory prediction device based on spatiotemporal attention, comprising:

[0012] A spatiotemporal graph construction module is used to construct a spatiotemporal graph at the current moment based on the targets in the target scene; the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the positions of the targets at a certain moment; the edge information set includes: spatial edges, temporal edges, and implicit edges; the spatial edges represent the relative positional relationships between different targets at the same moment; the temporal edges represent the motion change relationship of the same target between adjacent time steps; and the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship;

[0013] The trajectory prediction module is used to input the space-time graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment;

[0014] The trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on a recurrent neural network; the state node transition layer is constructed based on a multi-head attention mechanism and a gated recurrent unit;

[0015] The edge coding layer is used to generate the hidden state at the current moment based on the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment based on the position of the target at the current moment, the edge information set at the current moment, and the hidden state at the current moment.

[0016] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any one of the above-described trajectory prediction methods based on spatiotemporal attention.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned trajectory prediction methods based on spatiotemporal attention.

[0018] According to the specific embodiments provided in this application, this application has the following technical effects:

[0019] The present application provides a trajectory prediction method, apparatus, device and medium based on spatiotemporal attention. According to the spatiotemporal graph (ST-Graph) at the current moment and a trajectory prediction model constructed based on a spatiotemporal attention recurrent neural network (STA-RNN), the position of a target in a target scene at the next moment is predicted. The present application adopts STA-RNN and combines spatiotemporal graph modeling, a multi-edge encoding scheme and a multi-head attention mechanism to enable the trajectory prediction model to capture the complex spatiotemporal dependencies between targets and achieve high-precision trajectory prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a diagram of an application environment of a trajectory prediction method based on spatiotemporal attention in one embodiment of the present application;

[0022] Figure 2 A flowchart of a trajectory prediction method based on spatiotemporal attention provided in one embodiment of the present application;

[0023] Figure 3 A schematic diagram of the structure of a time-space diagram provided in one embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of an edge coding layer provided in one embodiment of the present application;

[0025] Figure 5 This is a flowchart of the internal calculation of the multi-head attention mechanism provided in one embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of a node state conversion layer provided in one embodiment of the present application;

[0027] Figure 7 A schematic diagram of the entire prediction process of trajectory prediction based on spatiotemporal attention provided in one embodiment of the present application;

[0028] Figure 8 A schematic diagram of the functional modules of a trajectory prediction method and apparatus based on spatiotemporal attention provided in another embodiment of the present application;

[0029] Figure 9 A schematic diagram of a flow chart of a trajectory prediction device provided in one embodiment of the present application;

[0030] Figure 10 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0032] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0033] The trajectory prediction method based on spatiotemporal attention provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the spatiotemporal graph to the server 104. After the server 104 receives the spatiotemporal graph, for the spatiotemporal graph, the server 104 inputs the spatiotemporal graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment; wherein, the trajectory prediction model is constructed based on the spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on the recurrent neural network; the state node transition layer is constructed based on the multi-head attention mechanism and the gated recurrent unit.

[0034] The server 104 can feed back the obtained position of the target at the next moment to the terminal 102. In addition, in some embodiments, the trajectory prediction method based on spatiotemporal attention can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly process the spatiotemporal graph, or the server 104 can obtain the spatiotemporal graph from a data storage system and process the spatiotemporal graph.

[0035] Terminal 102 may include, but is not limited to, various desktop computers, laptops, smartphones, tablet computers, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers, or may be a cloud server.

[0036] In an exemplary embodiment, Figure 2 As shown, a trajectory prediction method based on spatiotemporal attention is provided. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the process, including the following steps 201 to 208.

[0037] Step 201, constructing a spatiotemporal graph at the current moment based on the target in the target scene; wherein the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the position of the target at a certain moment; the edge information set includes: spatial edges, temporal edges and implicit edges.

[0038] The spatial edge represents the relative position relationship between different targets at the same moment; the temporal edge represents the motion change relationship of the same target between adjacent time steps; and the implicit edge represents the potential interaction relationship between targets that do not have a connection relationship.

[0039] Step 202: Input the spatiotemporal graph at the current moment into a trajectory prediction model to obtain the position of the target in the target scene at the next moment; wherein the trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on a recurrent neural network; the state node transition layer is constructed based on a multi-head attention mechanism and a gated recurrent unit.

[0040] The edge coding layer is used to generate the hidden state at the current moment based on the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment based on the position of the target at the current moment, the edge information set at the current moment, and the hidden state at the current moment.

[0041] The target scenario in the above embodiment can be application scenarios such as pedestrian trajectory prediction in crowded scenes, traffic flow prediction, and robot navigation. The target can be a pedestrian or an intelligent agent. Implementing steps 201 to 202 above can achieve high-precision trajectory prediction.

[0042] In another exemplary embodiment of the present application, the space-time diagram is introduced with emphasis.

[0043] The multiple moving targets in the target scene are modeled as a spatiotemporal graph consisting of nodes and edges. The structure of the spatiotemporal graph is as follows: Figure 3 As shown in , each node represents the position and attributes of a target at a specific time step, while the edge is used to describe the relationship between different time steps and spatial positions. Each node represents the position and attributes of a target at a specific time step, while the edge is used to describe the relationship between different time steps and spatial positions. Based on the position of the target, the constructed spatiotemporal graph is represented as

[0044] The node information set is represented as:

[0045]

[0046] Among them, G V Represents a node information set; represents the position of target i at the current time t; The horizontal coordinate of the position of target i at the current time t; Indicates the vertical coordinate of the position of target i at the current time t; where is used to indicate the value range or condition of a specific variable or parameter, here it is used to indicate The value of .

[0047] The edge information set is used to connect the spatiotemporal relationships between different time steps or different targets. The edges are divided into two categories, and it is assumed that all edges are undirected.

[0048] The spatial edge connects the positions of two different targets at the same time step, indicating their proximity in space. The set of spatial edges can represent the spatial interaction of all targets at a specific time step. The spatial edge is represented as:

[0049]

[0050] in, Represents a spatial edge; represents the relative position relationship between target i and target j at the current time t; p represents the distance function.

[0051] The temporal edge connects different positions of the same object in adjacent time steps, describing the time series changes of the object. The temporal edge is used to capture the motion changes of the object in adjacent time steps. Assuming that all edges are undirected, the temporal edge is represented as:

[0052]

[0053] in, represents the time edge; Represents the motion change relationship between target i at the current time t and the previous time t-1; Represents the position of target i at the previous time t-1.

[0054] The implicit edges include: Spatial-Spatial Edge Factor, Spatial-Temporal Edge Factor, Temporal-SpatialEdge Factor and Temporal-Temporal Edge Factor. Implicit edge factor is an analysis method for extracting indirect and complex relationships from graph structures. Traditional spatiotemporal graph methods usually only rely on spatial information or temporal information at adjacent moments, while ignoring more complex indirect relationships. For example, although two targets (such as pedestrians) are not close to each other, they may be affected by each other's movement during prediction. The embodiment is based on a feature extraction method of implicit edge factors, which can generate rich spatiotemporal dependency features by crossing different edge types in spatial and temporal dimensions. This embodiment introduces four implicit edge factors.

[0055] The space-space edge factor is expressed as:

[0056]

[0057] in, represents the space-space edge factor, which characterizes the position of target i relative to target j; represents the relative position relationship between target i and target k at the current time t; represents the relative position relationship between target k and target j at the current time t; ⊙ represents the element-wise product, also known as the Hadamard product, which is an operator commonly used in graph computing and deep learning to multiply corresponding elements of two matrices; there is no connection between target i and target j.

[0058] The space-time edge factor is expressed as:

[0059]

[0060] in, represents the space-time edge factor, characterizing the position of target i relative to the past position of target j; It represents the motion change relationship between the target j at the current time t and the previous time t-1, and represents the movement of the target j from the previous time t-1 to the current time t.

[0061] The time-space edge factor is expressed as:

[0062]

[0063] in, represents the time-space edge factor, which represents the position of target i relative to target j at the previous moment; Represents the relative position relationship between target i and target j at the previous moment t-1.

[0064] The time-time edge factor is expressed as:

[0065]

[0066] Among them, F i TT (t) represents the time-time edge factor, which represents the position change of target i between the current time t and the previous time t-1; It represents the motion change relationship between target i at the previous moment t-1 and the previous moment t-2, and represents the movement of target j from the previous moment t-2 to the previous moment t-1.

[0067] By constructing a spatiotemporal graph, subsequent models can better capture dynamic changes in the scene, especially the interactions between multiple targets. These relationships are not limited to spatial interactions at the same time step, but also include temporal associations across time steps, providing important input information for subsequent trajectory prediction.

[0068] In another exemplary embodiment of the present application, the method for determining the trajectory prediction model includes:

[0069] (1) Constructing an edge coding layer; the edge coding layer includes multiple embedding layers and multiple recurrent neural network (RNN) layers; the embedding layers and the recurrent neural network layers are connected in a one-to-one correspondence; the number of the embedding layers and the recurrent neural network layers is determined according to the number of edge types in the edge information set; the embedding layer is used to embed a certain type of edge in the input edge information set to generate a feature vector of uniform dimension; the recurrent neural network layer is used to generate the hidden state of the edge type at the current moment based on the feature vector output by the corresponding embedding layer and the hidden state of the edge type at the previous moment. For example, the number of the embedding layers and the recurrent neural network layers is equal to the number of edge types in the edge information set, and the edge types in the edge information set include 1 spatial edge, 1 temporal edge, and 4 implicit edges, that is, the number of the embedding layers and the recurrent neural network layers can be 6.

[0070] Specifically, the edge coding layer models and encodes the relationships between objects. Since the interactions between objects in a scene can be represented by different types of edges (e.g., spatial edges, temporal edges, and implicit edges), the edge coding layer is specifically designed to capture information about these edges. The primary task of this layer is to process different types of edge information through multiple independent RNNs. For example, spatial edges represent the relative positional relationships between different objects within the same timeframe, temporal edges represent the motion changes of objects between adjacent time steps, and implicit edges are used to capture potential interactions that are not explicitly connected. Through these encodings, the model can extract more refined and diverse spatiotemporal relationships from the spatiotemporal graph. By comprehensively analyzing different types of edge information, the edge coding layer can accurately reflect the interactions and spatiotemporal dependencies between objects.

[0071] The structure of the edge coding layer is as follows Figure 4 shown. Figure 4 In the example, we first select the same type of edge information connected to the same node and add them together to form a vector. This will generate six different types of vectors, corresponding to six different outputs. Then they are sent to the embedding layer for embedding. The role of the embedding layer is to convert edge information of different dimensions into a unified low-dimensional vector, which can be represented as So that the subsequent RNN layers can process this information uniformly.

[0072] Through the embedding operation, the original high-dimensional input features are mapped to the feature space that the model can process, and the spatiotemporal dependency information of the original edges is retained. It is input to the RNN layer for time series processing. The RNN layer is responsible for combining the edge information of each time step with the hidden state of the previous time step to generate a new hidden state. This hidden state is used to represent the spatiotemporal dependencies of the target at the current time step.

[0073] Through the iterative processing of the RNN layer, the model is able to capture the dynamic characteristics of edge information over time, especially the interaction between objects in different time and space. The RNN layer also effectively alleviates the vanishing gradient problem in time series modeling, enabling the model to handle longer time dependencies.

[0074] The processed hidden state serves as input to the subsequent node state transition layer to dynamically adjust the state of the target node. The hidden state of each node is determined not only by its own historical information but also by its associated edge relationships. This enables the model to flexibly simulate spatiotemporal interactions between targets and predict future trajectories based on these interactions.

[0075] The edge encoding layer, through multi-type edge information input, embedding, and RNN processing, enables the model to capture the complex spatiotemporal dependencies between objects in the spatiotemporal graph. It encodes and processes the interactions between objects, providing key input information for updating node states, thereby enhancing the model's ability to predict trajectories in dynamic scenarios.

[0076] (2) Constructing a state node conversion layer; the state node conversion layer includes: an embedder, a multi-head attention mechanism (MHA) layer, a gated recurrent unit (GRU) and a decoder. The embedder is used to perform embedding processing based on the position of the target at the current moment to generate the embedded position at the current moment; the multi-head attention mechanism layer is used to process the hidden state of all types of edges output by all recurrent neural network layers and the embedded position at the current moment to obtain the output result of the multi-head attention mechanism; the gated recurrent unit is used to obtain the hidden state at the next moment based on the output result of the multi-head attention mechanism and the hidden state at the current moment; the decoder is used to determine the offset at the current moment based on the hidden state at the next moment; the embedded position at the current moment and the offset at the current moment are added to obtain the position of the target at the next moment.

[0077] (3) Connecting the edge coding layer and the state node conversion layer to obtain a spatiotemporal attention mechanism recurrent neural network (STA-RNN).

[0078] (4) Using training data to train the spatiotemporal attention mechanism recurrent neural network, and determining the trained spatiotemporal attention mechanism recurrent neural network as the trajectory prediction model.

[0079] This step specifically includes:

[0080] Acquire training data; the training data includes: a spatiotemporal graph of a target in a target scene at a historical moment and its actual position at the next moment after the historical moment; input the training data into the spatiotemporal attention mechanism recurrent neural network, use an Adam optimizer to train with the goal of minimizing the loss function, obtain a trained spatiotemporal attention mechanism recurrent neural network, and determine the trained spatiotemporal attention mechanism recurrent neural network as the trajectory prediction model; the loss function represents the average deviation of the target trajectory.

[0081] The loss function adopts the mean square error function, which is expressed as:

[0082]

[0083] Among them, MSE represents the loss function; represents the predicted position at the historical moment t1; p t1 represents the real position at the historical moment t1; T represents all historical moments.

[0084] In another exemplary embodiment of the present application, the state node conversion layer is further introduced.

[0085] First, the multi-head attention mechanism layer includes: a first linear layer, a scaled dot product attention layer, a concatenation layer, and a second linear layer; there are multiple first linear layers and multiple scaled dot product attention layers; the first linear layer is connected to the scaled dot product attention layer in a one-to-one correspondence; each scaled dot product attention layer is connected to the concatenation layer; the concatenation layer is connected to the second linear layer; the number of first linear layers and scaled dot product attention layers is greater than the number of recurrent neural network layers, the input of one of the first linear layers is the position after embedding at the current moment, the output of the recurrent neural network layer is connected to the input of the first linear layer in a one-to-one correspondence, and the input of the remaining first linear layers is a zero vector. For example, the number of first linear layers and scaled dot product attention layers can be 8, the input of one first linear layer is the position after embedding at the current moment, the outputs of 6 recurrent neural network layers are connected to the inputs of 6 first linear layers in a one-to-one correspondence, and the input of the remaining first linear layer is a zero vector. The zero vector here is only used for padding and does not contain any meaningful feature information. Therefore, it does not interfere with model learning and does not affect the attention calculation of other input features.

[0086] The first linear layer is used to perform a linear transformation on the input data to generate a query vector, a key vector and a value vector; the scaled dot product attention layer is used to perform attention calculation on the output of the corresponding first linear layer; the splicing layer is used to splice the outputs of all scaled dot product attention layers to obtain a splicing result; the second linear layer is used to perform a linear transformation on the splicing result to obtain the output result of the multi-head attention mechanism.

[0087] Specifically, when updating a node state, the node state transition layer uses a multi-head attention mechanism (MHA) to process hidden states encoded by different edges. MHA processes information from different edges in parallel at each time step, focusing on the most relevant spatiotemporal features and strengthening attention to important information through a weight distribution mechanism. MHA enables the model to distinguish information from different time steps and spatial locations in the time series, avoiding information redundancy caused by large data volumes.

[0088] In STA-RNN, MHA processes different spatiotemporal information from spatial edges, temporal edges, and implicit edges in parallel to better capture complex spatiotemporal dependencies. Through this multi-head mechanism, the model can simultaneously focus on the interactions between multiple targets, enhancing its ability to handle complex scenes. The internal calculation process of the multi-head attention mechanism is as follows Figure 5 As shown, combined Figure 5 Explain the computational logic of the multi-head attention mechanism.

[0089] (1) Input and mapping of Q, K, and V.

[0090] In the multi-head attention mechanism network, the input feature vector (The hidden state from the RNN layer) generates Q, K, and V through different linear transformations. Each attention head has its own independent parameter matrix to generate Q, K, and V. The calculation formula is as follows:

[0091]

[0092] Among them, W Q 、W K 、W V It is the weight matrix that needs to be learned, which is used to calculate Query, Key, and Value respectively.

[0093] (2) Calculation of attention weight.

[0094] Next, the attention weight is obtained by calculating the similarity between the query Q and the key K. The commonly used similarity metric is the dot product, and the result is scaled and then normalized by the softmax function. The specific formula is:

[0095]

[0096] Where Q is the query vector. K T is the transpose of the key vector. k is the dimension of the query and key, used to scale the dot product to prevent it from becoming too large as the dimension increases. V is the value vector. Softmax is used for normalization so that the sum of the attention weights is 1.

[0097] (3) Calculation of multi-head attention.

[0098] In the multi-head attention mechanism, there will be multiple independent attention heads, calculating multiple different Q, K, V in parallel. Figure 5 The number of attention heads n can be set to 8. Each attention head will produce different weights and weighted values. For any attention head, the calculation formula is as follows:

[0099]

[0100] Finally, the outputs of all attention heads are concatenated and passed through a linear transformation W L The final multi-head attention output is calculated as follows:

[0101] MultiHead(Q,K,V)=Concat(head1,head2,...,head n )W L

[0102] The multi-head attention mechanism effectively improves the STA-RNN model's ability to capture long-term and multi-dimensional spatial dependencies. By processing multiple spatiotemporal features in parallel, the model avoids the limitations of relying solely on a single spatiotemporal relationship, especially when dealing with scenes with complex dynamic interactions (such as pedestrians in a crowd). MHA can flexibly focus on different spatiotemporal layers, resulting in more accurate results in trajectory prediction.

[0103] The node state transition layer is a key component in the STA-RNN model for updating and adjusting node states. Its primary function is to update the node state at the current time step based on edge information at each time step in the spatiotemporal graph, the output of the multi-head attention mechanism, and the hidden state from the previous time step. This layer's design, by continuously iteratively updating node states, enables the model to adapt to changes in the target's trajectory in complex dynamic scenarios. Figure 6 The network structure of the node state transition layer is shown.

[0104] Figure 6At each time step, the node state transition layer first updates the hidden state based on the output from the multi-head attention mechanism. The six hidden states in the figure, from top to bottom, represent the different hidden states obtained by the edge encoding layer when connecting to the same node via spatial edges, temporal edges, space-space edge factors, space-time edge factors, time-space edge factors, and time-time edge factors.

[0105] This process combines the edge-level hidden state of the current time step with the node hidden state of the previous time step, and specifically uses the gated recurrent unit (GRU) to process this spatiotemporal information.

[0106] Multi-head attention output: The multi-head attention mechanism processes inputs from different edge types to generate a set of weighted edge-level hidden states. These states contain the spatiotemporal relationships between targets and are used to guide the update of node states.

[0107] GRU: An improved recurrent neural network that effectively controls information transfer through a gating mechanism, avoiding the vanishing gradient problem common in traditional RNNs. The specific functions of GRU in the node state transition layer are as follows:

[0108] Update gate: The update gate in GRU controls how much of the current hidden state comes from the information of the previous time step and how much comes from the multi-head attention result of the current time step. This ensures that the model captures short-term dependencies while not ignoring long-term historical information.

[0109] Reset gate: By controlling the reset gate, the GRU can decide whether to rely entirely on the current input information or retain part of the previous state. This can dynamically adjust the model's reliance on historical information, making the model adaptive at different time steps.

[0110] Updated hidden state It is the result of GRU processing, combining the node's historical trajectory information and the current spatiotemporal edge relationship. This hidden state not only represents the state of the current time step, but also provides the necessary input for the prediction of the next time step. The current hidden state is passed through the decoder composed of the fully connected layers to obtain the offset of the node position ξ i (t), used to calculate the landing point at the next moment.

[0111] Because trajectory prediction is a dynamic time series task, the node state conversion layer updates the node state at each time step. Through the combined action of the GRU and MHA, the node state at each time step is gradually iteratively updated, providing a more accurate foundation for trajectory prediction in future time steps. This iterative update not only captures the target's short-term motion trends but also captures long-term dependencies by accumulating historical information.

[0112] Overall, the node state transition layer in this application achieves dynamic updates of node states by combining a multi-head attention mechanism with GRU. By weighting edge-level information at each time step, node states can adapt to complex spatiotemporal changes, providing accurate input for predictions at subsequent time steps. The introduction of GRU further enhances the model's ability to handle long-term dependencies, ensuring the continuity and accuracy of trajectory predictions.

[0113] The entire prediction process of trajectory prediction based on spatiotemporal attention is as follows Figure 7 shown.

[0114] Based on the STA-RNN, a trajectory prediction model was constructed that combines spatiotemporal graph modeling with an attention mechanism. By jointly processing spatial and temporal information, the STA-RNN accurately captures the complex spatiotemporal dependencies between objects. The key to this model lies in the design of the edge encoding layer and the node state transition layer. These two layers are crucial for model performance. Their synergy effectively handles the interactions and dynamic changes of objects in the spatiotemporal dimension, improving trajectory prediction accuracy.

[0115] In another exemplary embodiment of this application, pedestrian prediction is used as an example to introduce the optimization training of the trajectory prediction model. This embodiment uses the ETH and UCY datasets, observing for 3.2 seconds and predicting the target trajectory for 4.8 seconds. These datasets are widely used for pedestrian trajectory prediction. The number of pedestrians in the scene is large and the interactions are complex. The movement patterns between pedestrians are affected not only by historical trajectories but also by the dynamic behavior of surrounding pedestrians.

[0116] Trajectory prediction optimization is a key component of the STA-RNN model, ensuring it can accurately predict future target trajectories. To improve the accuracy and robustness of trajectory prediction, this application's prediction algorithm incorporates multiple optimization strategies, including loss function design, parameter optimization algorithms, and model training methods. These optimization mechanisms enable the model to excel in complex trajectory prediction tasks and quickly converge to ideal prediction results.

[0117] (1) Loss function design.

[0118] The mean squared error (MSE) is the primary loss function used to evaluate the difference between predicted and true trajectories. At each time step, the STA-RNN calculates the Euclidean distance between the predicted and true positions to measure the model's prediction accuracy. Specifically, the MSE measures the average deviation of the target trajectory; the calculation formula is not detailed here. By minimizing the MSE, the model can continuously adjust its parameters during training to improve trajectory prediction accuracy.

[0119] (2) Parameter optimization algorithm.

[0120] Adam Optimizer: To accelerate model convergence and ensure the stability of parameter updates, the prediction network uses the Adam optimizer. Adam combines the advantages of momentum and adaptive learning rate adjustment, allowing the model to find the global optimal solution more quickly during training. The learning rate during prediction network training in this application was set to 0.001.

[0121] (3) Model training method.

[0122] Dataset: The ETH and UCY datasets are used, which contain pedestrian motion trajectories in five scenarios.

[0123] Input data preprocessing: The trajectory data is first normalized and the coordinate range is scaled to [-1, 1] to ensure that the eigenvalues ​​of each dimension are at the same level during model training.

[0124] Observation period and prediction period: During training, the observed trajectories of the first 3.2 seconds are used as input, and the model predicts the trajectory of the next 4.8 seconds based on these trajectories.

[0125] The trajectory prediction method based on spatiotemporal attention proposed in this application embodiment uses a spatiotemporal graph to represent the interrelationships of multiple targets (pedestrians or intelligent agents) in space and time. On this basis, the information of nodes and edges in the graph is encoded, updated, and decoded. A recurrent neural network is used to process time series data, and the prediction accuracy of the model is improved through multiple edge encoding mechanisms. This method is particularly suitable for trajectory prediction tasks in complex dynamic environments, such as pedestrian trajectory prediction in crowded scenes, traffic flow prediction, and robot navigation.

[0126] The embodiments of the present application focus on the following two aspects: First, the spatiotemporal graph modeling and multi-edge encoding process: by modeling the navigation scene as a spatiotemporal graph, the dynamic changes of pedestrian positions and time are represented. The pedestrian positions in each time step are connected through spatial edges and temporal edges to establish complex spatiotemporal relationships. Six different types of edge encoding schemes are introduced through the edge encoder layer, and multiple independent RNNs are used to process edge information, thereby learning the various connections between pedestrians and other nodes and capturing implicit spatiotemporal dependencies. This solution can effectively cope with trajectory prediction in dynamic scenarios and provide accurate path calculation. Second, the trajectory prediction process based on the spatiotemporal attention mechanism: the multi-head attention mechanism (MHA) is combined with the gated recurrent unit (GRU) to realize spatiotemporal trajectory prediction. In the trajectory prediction process, the spatiotemporal edge information is first processed in parallel by MHA to capture the long-term spatiotemporal dependencies in the pedestrian trajectory, so that the model can more accurately distinguish the impact of different moments and spatial positions on the future trajectory. Subsequently, the GRU is used to dynamically update the node state. The node state is gradually iteratively updated by jointly calculating the attention output at the current moment and the node hidden state at the previous moment. Finally, the GRU output is decoded through a fully connected layer to generate a prediction of the pedestrian's future position. This method fully utilizes spatiotemporal features to enhance the accuracy and robustness of trajectory prediction, making it particularly suitable for multi-step trajectory prediction tasks in complex and dynamic scenarios.

[0127] The effectiveness of the trajectory prediction method based on spatiotemporal attention in this embodiment is verified through experiments below.

[0128] The trajectory prediction method based on spatiotemporal attention proposed in this application has shown superior performance in multiple practical application scenarios, especially in dealing with complex dynamic scenes. The comparison algorithms are Social LSTM (SLSTM), which uses a social pooling layer combined with a long short-term memory network to receive the current position and predict the output position; Structural-RNN (SRNN), which uses structured modeling to represent the scene as a bitmap, explicitly modeling the interaction between individuals and modular design for flexible deployment, and modeling different types of interactions separately; Trajectron++, which is a graph-based model that uses spatiotemporal graphs and variational autoencoder architectures to simulate multimodal trajectory distributions and predict future positions in dynamic environments; the model STA-RNN in this invention combines spatiotemporal graph modeling with a multi-edge encoding scheme and a multi-head attention mechanism, enabling the model to capture the complex spatiotemporal dependencies between targets and achieve high-precision trajectory prediction.

[0129] The simulation parameters are set as follows: This experiment uses the ETH / UCY dataset, which contains trajectory data of multiple pedestrians in different scenarios. The data is sampled at a frequency of 2.5 frames per second, for a total of 3837 scenarios. The observation time is 3.2 seconds, the prediction time is 4.8 seconds, and all trajectory data is normalized to the range [-1, 1]. The model uses the Adam optimizer with a learning rate of 0.001 and is trained for 10 epochs. At each time step, the model predicts the trajectory through the edge encoding layer and the node state transition layer, with the dimensions of the edge embedding and node embedding being 64 and 128, respectively. The mean squared error (MSE) is used as the loss function during training to ensure stable convergence.

[0130] ADE (Average Displacement Error) and FDE (Final Displacement Error) are common metrics for measuring the performance of trajectory prediction models. ADE is the average Euclidean distance between the predicted coordinates and the true historical coordinates over all time steps. FDE is the Euclidean distance between the final position of the predicted trajectory and the final position in the true trajectory. The calculation formula is as follows:

[0131]

[0132] in are the predicted coordinates and true coordinates at time t, are the predicted coordinates and actual coordinates of the endpoint time T respectively.

[0133] Table 1 summarizes the performance of these four trajectory prediction models on five datasets, showing the ADE and FDE comparisons of different models.

[0134] Table 1 Performance comparison of different models on the ETH / UCY dataset

[0135]

[0136] S-LSTM performs the worst on the five datasets, especially on the densely populated UCY dataset, where it lags behind the ETH dataset. This poor performance is due to the fact that as pedestrian density increases, the complexity of capturing social patterns between pedestrians also increases, and S-LSTM struggles to fully capture social dynamics. SRNN improves upon S-LSTM, but its lack of an attention mechanism means it cannot effectively integrate spatial relationships and time series information, nor can it focus on important states. Trajectron++ further improves trajectory prediction by using a spatiotemporal graph-based approach that better captures spatiotemporal dependencies. However, without an explicit attention mechanism, it may still face the challenge of effectively prioritizing relevant interactions in complex environments.

[0137] In terms of ADE / FDE metrics, our model (STA-RNN) achieved improvements of 24.7% and 45.9% compared to S-LSTM, 12.7% and 25.6% compared to SRNN, and 2.11% and 4.92% compared to Trajectron++, respectively, demonstrating superior performance in dense environments. Through its attention mechanism and more efficient spatial and temporal reasoning, STA-RNN effectively leverages edge encoding information derived from spatiotemporal graph reasoning, significantly outperforming previous models and achieving better prediction performance.

[0138] The embodiment of the present application models multiple mobile targets in the target scene as a spatiotemporal graph structure composed of nodes and edges, where the nodes represent the state of the target at a specific time step, and the edges describe the relationship between different time steps and spatial positions. Through this modeling method, a complex spatiotemporal relationship graph is constructed to capture the spatiotemporal dependencies between targets. After the modeling is completed, a multi-type edge coding scheme of spatial edges, temporal edges and implicit edges is adopted, and an independent recurrent neural network is used to process these edge information to capture the complex interactions between targets and improve the prediction accuracy. Finally, a multi-head attention mechanism is introduced in the node state conversion layer to process spatiotemporal edge information and capture the long spatiotemporal dependencies in pedestrian trajectories. The multi-head attention mechanism can process information from different edges in parallel, enabling the model to predict future trajectories more accurately. At the same time, a gated recurrent unit is introduced to dynamically update the node state, combine the attention output at the current moment with the node hidden state at the previous moment, and gradually iterate to update the trajectory prediction.

[0139] Based on the same inventive concept, the embodiments of the present application also provide a spatiotemporal attention-based trajectory prediction device for implementing the spatiotemporal attention-based trajectory prediction method involved above. The implementation solution provided by the device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more spatiotemporal attention-based trajectory prediction device embodiments provided below can be referred to the limitations of the spatiotemporal attention-based trajectory prediction method above, and will not be repeated here.

[0140] In an exemplary embodiment, Figure 8 As shown, a trajectory prediction device based on spatiotemporal attention is provided, comprising:

[0141] The spatiotemporal graph construction module 301 is used to construct a spatiotemporal graph at the current moment based on the targets in the target scene; the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the position of the target at a certain moment; the edge information set includes: spatial edges, temporal edges and implicit edges; the spatial edges represent the relative position relationship between different targets at the same moment; the temporal edges represent the motion change relationship of the same target between adjacent time steps; the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship.

[0142] The trajectory prediction module 302 is used to input the space-time graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment.

[0143] Among them, the trajectory prediction model is constructed based on a recurrent neural network with a spatiotemporal attention mechanism; the spatiotemporal attention recurrent neural network includes: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on a recurrent neural network; the state node transition layer is constructed based on a multi-head attention mechanism and a gated recurrent unit.

[0144] The edge coding layer is used to generate the hidden state at the current moment based on the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment based on the position of the target at the current moment, the edge information set at the current moment, and the hidden state at the current moment.

[0145] Furthermore, the trajectory prediction device based on spatiotemporal attention also includes: a spatiotemporal graph construction module, an edge encoding module, a node state update module and an intelligent optimization module, which are used to determine the trajectory prediction model.

[0146] The above device can accurately predict the future trajectory by analyzing and modeling the historical trajectory data of the target, and is suitable for scenarios such as autonomous driving and pedestrian prediction. The process of the trajectory prediction device is as follows Figure 9 shown.

[0147] The target's historical trajectory data is obtained according to the data flow graph, and the spatiotemporal graph construction module generates a spatiotemporal graph structure based on the target's historical trajectory. The target state at each time step is modeled as a node in the spatiotemporal graph, and the spatiotemporal dependencies between targets are modeled as edges in the spatiotemporal graph. The edge encoding module encodes the edges of the spatiotemporal graph to capture the spatiotemporal relationships between targets. It includes spatial edges, temporal edges, and implicit edges, which respectively represent the relative spatial position of the target at the same time step, the motion changes in adjacent time steps, and potential interactions. The node state update module dynamically updates the node states in the spatiotemporal graph through a multi-head attention mechanism and a gated recurrent unit (GRU). This module is responsible for iteratively updating the target state at each time step, combining historical trajectory information with edge features for state updates. The state update formula is:

[0148]

[0149] Among them, M i (t) is the output of the multi-head attention mechanism.

[0150] The trajectory prediction module predicts the target's position in the future time step based on the updated node states. This module generates trajectory prediction results and calculates the error with the true trajectory to evaluate the prediction accuracy.

[0151] The intelligent optimization module is responsible for optimizing the model parameters and evaluating the prediction performance. Based on error feedback, the intelligent optimization module adjusts the model parameters to improve the prediction accuracy.

[0152] This application is a trajectory prediction system based on the attention of the spatiotemporal graph. By constructing a spatiotemporal graph, the position information of each pedestrian at different time steps is represented as a node in the graph, and the interaction relationship between the time step and the space step is modeled as an edge in the graph. The edges in the spatiotemporal graph not only capture the spatial relationship between pedestrians and surrounding pedestrians, but also cover the motion changes in the time dimension, thus forming a dynamic spatiotemporal dependency model. The system uses a multi-head attention mechanism to process the spatiotemporal relationship of the target in parallel, and combines the gated recurrent neural network to update the state of the time series, which can achieve high-precision prediction of the future trajectory of the target. By introducing dynamic edges and node state update mechanisms, the system can adaptively adjust the interaction relationship between targets to adapt to a variety of complex scenarios. This method effectively improves the prediction accuracy, significantly reduces the error in high-dynamic scenes, and greatly improves the computational efficiency.

[0153] In summary, the trajectory prediction system is an advanced prediction method that integrates spatiotemporal graph modeling, a multi-head attention mechanism, and dynamic state updates. It can achieve efficient and high-precision trajectory prediction in complex multi-target dynamic scenarios, and provides a solid technical foundation for future applications in intelligent driving, pedestrian prediction, and other fields. Simulation results show that the algorithm proposed in this application outperforms all compared algorithms in five scenarios and also has good scalability.

[0154] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 10As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store a spatiotemporal graph. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a trajectory prediction method based on spatiotemporal attention is implemented.

[0155] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0156] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0157] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0158] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0159] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0160] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, and the like.

[0161] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A trajectory prediction method based on spatiotemporal attention, characterized in that: The trajectory prediction method based on spatiotemporal attention includes: A spatiotemporal graph is constructed at the current moment based on the targets in the target scene; the spatiotemporal graph includes a node information set and an edge information set; the nodes in the node information set represent the positions of the targets at a certain moment; the edge information set includes spatial edges, temporal edges, and implicit edges; the spatial edges represent the relative positional relationships between different targets at the same moment; the temporal edges represent the motion change relationship of the same target between adjacent time steps; and the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship. Input the current time-space graph into the trajectory prediction model to obtain the position of the target in the target scene at the next moment; The trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a state node conversion layer; the edge coding layer is constructed based on a recurrent neural network; the state node conversion layer is constructed based on a multi-head attention mechanism and a gated recurrent unit; The edge coding layer is used to generate the hidden state at the current moment based on the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency relationship between targets; the state node conversion layer is used to determine the position of the target at the next moment based on the position of the target at the current moment, the edge information set at the current moment, and the hidden state at the current moment; The method for determining the trajectory prediction model includes: Constructing an edge coding layer; the edge coding layer includes multiple embedding layers and multiple recurrent neural network layers; the embedding layers and the recurrent neural network layers are connected in a one-to-one correspondence; the number of the embedding layers and the recurrent neural network layers is determined according to the number of edge types in the edge information set; the embedding layer is used to embed a certain type of edge in the input edge information set to generate a feature vector of uniform dimension; the recurrent neural network layer is used to generate the hidden state of the edge of the type at the current moment based on the feature vector output by the corresponding embedding layer and the hidden state of the edge of the type at the previous moment; Construct a state node conversion layer; the state node conversion layer includes: an embedder, a multi-head attention mechanism layer, a gated recurrent unit and a decoder; the embedder is used to perform embedding processing based on the position of the target at the current moment to generate the embedded position at the current moment; the multi-head attention mechanism layer is used to process the hidden states of all types of edges output by all recurrent neural network layers and the embedded positions at the current moment to obtain the output results of the multi-head attention mechanism; the gated recurrent unit is used to obtain the hidden state at the next moment based on the output results of the multi-head attention mechanism and the hidden state at the current moment; the decoder is used to determine the offset at the current moment based on the hidden state at the next moment; the embedded position at the current moment and the offset at the current moment are added to obtain the position of the target at the next moment; Connecting the edge encoding layer and the state node conversion layer to obtain a spatiotemporal attention mechanism recurrent neural network; The spatiotemporal attention mechanism recurrent neural network is trained using training data, and the trained spatiotemporal attention mechanism recurrent neural network is determined as the trajectory prediction model.

2. The trajectory prediction method based on spatiotemporal attention according to claim 1, characterized in that The node information set is represented as: Among them, G V Represents a node information set; represents the position of target i at the current time t; The horizontal coordinate of the position of target i at the current time t; The ordinate represents the position of target i at the current time t; The spatial edge is represented as: in, Represents a spatial edge; represents the relative position relationship between target i and target j at the current time t; p represents the distance function; The time edge is expressed as: in, represents the time edge; Represents the motion change relationship between target i at the current time t and the previous time t-1; represents the position of target i at the previous time t-1; The implicit edges include: space-space edge factor, space-time edge factor, time-space edge factor and time-time edge factor; The space-space edge factor is expressed as: in, represents the space-space edge factor, which characterizes the position of target i relative to target j; represents the relative position relationship between target i and target k at the current time t; represents the relative position relationship between target k and target j at the current time t; ⊙ represents the element product; there is no connection relationship between target i and target j; The space-time edge factor is expressed as: in, represents the space-time edge factor, characterizing the position of target i relative to the past position of target j; Represents the motion change relationship between target j at the current time t and the previous time t-1; The time-space edge factor is expressed as: in, represents the time-space edge factor, which represents the position of target i relative to target j at the previous moment; represents the relative position relationship between target i and target j at the previous moment t-1; The time-time edge factor is expressed as: in, represents the time-time edge factor, which characterizes the position change of target i between the current time t and the previous time t-1; Represents the motion change relationship between target i at the previous moment t-1 and the previous moment t-2.

3. The trajectory prediction method based on spatiotemporal attention according to claim 1, characterized in that The multi-head attention mechanism layer includes: a first linear layer, a scaled dot product attention layer, a splicing layer, and a second linear layer; there are multiple first linear layers and multiple scaled dot product attention layers; the first linear layer is connected to the scaled dot product attention layer in a one-to-one correspondence; each scaled dot product attention layer is connected to the splicing layer; the splicing layer is connected to the second linear layer; the number of the first linear layers and the scaled dot product attention layers is greater than the number of recurrent neural network layers, the input of one of the first linear layers is the position after embedding at the current moment, the output of the recurrent neural network layer is connected to the input of the first linear layer in a one-to-one correspondence, and the inputs of the remaining first linear layers are zero vectors; The first linear layer is used to perform a linear transformation on the input data to generate a query vector, a key vector and a value vector; the scaled dot product attention layer is used to perform attention calculation on the output of the corresponding first linear layer; the splicing layer is used to splice the outputs of all scaled dot product attention layers to obtain a splicing result; the second linear layer is used to perform a linear transformation on the splicing result to obtain the output result of the multi-head attention mechanism.

4. The trajectory prediction method based on spatiotemporal attention according to claim 1, characterized in that The spatiotemporal attention mechanism recurrent neural network is trained using the training data, and the trained spatiotemporal attention mechanism recurrent neural network is determined as the trajectory prediction model, specifically including: Acquire training data; the training data includes: a spatiotemporal diagram of a target in a target scene at a historical moment and a real position at a next moment after the historical moment; The training data is input into the spatiotemporal attention mechanism recurrent neural network, and the Adam optimizer is used to perform training with the goal of minimizing the loss function to obtain a trained spatiotemporal attention mechanism recurrent neural network, and the trained spatiotemporal attention mechanism recurrent neural network is determined as the trajectory prediction model; the loss function represents the average deviation of the target trajectory.

5. The trajectory prediction method based on spatiotemporal attention according to claim 4, characterized in that The expression of the loss function is: Among them, MSE represents the loss function; represents the predicted position at the historical moment t1; p t1 represents the real position at the historical moment t1; T represents all historical moments.

6. The trajectory prediction method based on spatiotemporal attention according to claim 1, characterized in that The target is a pedestrian or an intelligent agent.

7. A trajectory prediction device based on spatiotemporal attention, used to implement the trajectory prediction method based on spatiotemporal attention according to any one of claims 1 to 6, characterized in that: The trajectory prediction device based on spatiotemporal attention includes: A spatiotemporal graph construction module is used to construct a spatiotemporal graph at the current moment based on the targets in the target scene; the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the positions of the targets at a certain moment; the edge information set includes: spatial edges, temporal edges, and implicit edges; the spatial edges represent the relative positional relationships between different targets at the same moment; the temporal edges represent the motion change relationship of the same target between adjacent time steps; and the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship; The trajectory prediction module is used to input the space-time graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment; The trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a state node conversion layer; the edge coding layer is constructed based on a recurrent neural network; the state node conversion layer is constructed based on a multi-head attention mechanism and a gated recurrent unit; The edge coding layer is used to generate the hidden state at the current moment based on the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment based on the position of the target at the current moment, the edge information set at the current moment, and the hidden state at the current moment.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the trajectory prediction method based on spatiotemporal attention according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the trajectory prediction method based on spatiotemporal attention according to any one of claims 1 to 6 is implemented.

Images