Track prediction method and device based on space-time attention, equipment and medium

Abstract

The invention discloses a trajectory prediction method and device based on space-time attention, equipment and a medium, and relates to the field of trajectory prediction, and the method comprises the steps: constructing a space-time diagram of a current moment according to a target in a target scene; inputting the time-space diagram at the current moment into a trajectory prediction model to obtain the position of a target in the target scene at the next moment; wherein the trajectory prediction model is constructed based on a space-time attention mechanism recurrent neural network; the space-time attention mechanism recurrent neural network comprises an edge coding layer and a node state conversion layer; the edge coding layer is constructed based on a recurrent neural network; and the state node conversion layer is constructed based on a multi-head attention mechanism and a gating circulation unit. According to the invention, the trajectory prediction precision of the target can be improved.

Description

Technical Field

[0001] The present application relates to the field of trajectory prediction, and in particular to a trajectory prediction method, device, equipment and medium based on spatiotemporal attention. Background Art

[0002] In recent years, spatiotemporal prediction systems have received increasing attention, especially in the fields of autonomous driving, intelligent monitoring, and pedestrian trajectory prediction. With the complexity and intelligence of various application scenarios, how to accurately predict the future trajectory of a target has become an important research topic. In these scenarios, the motion trajectory of a target is affected by many factors, including historical trajectories, dynamic changes in spatial positions, and interference from the surrounding environment.

[0003] Related trajectory prediction methods still have problems such as low accuracy and low computational efficiency when dealing with high-density scenes and dynamic interactions of multiple targets. For example, in an environment with dense pedestrians, the interactions between different targets are complex and changeable, and related trajectory prediction methods are difficult to capture such complex spatial dependencies, resulting in large prediction errors. At the same time, as the prediction time step increases, the prediction accuracy gradually decreases, and the model cannot effectively handle long-term trajectory prediction tasks. Summary of the invention

[0004] The purpose of this application is to provide a trajectory prediction method, device, equipment and medium based on spatiotemporal attention, which can improve the trajectory prediction accuracy of the target.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a trajectory prediction method based on spatiotemporal attention, comprising:

[0007] A spatiotemporal graph at the current moment is constructed according to the targets in the target scene; the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the position of the target at a certain moment; the edge information set includes: spatial edges, temporal edges and implicit edges; the spatial edges represent the relative position relationship between different targets at the same moment; the temporal edges represent the motion change relationship between the same target in adjacent time steps; the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship;

[0008] Input the current time-space graph into the trajectory prediction model to obtain the position of the target in the target scene at the next moment;

[0009] The trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state conversion layer; the edge coding layer is constructed based on a recurrent neural network; the state node conversion layer is constructed based on a multi-head attention mechanism and a gated recurrent unit;

[0010] The edge coding layer is used to generate the hidden state at the current moment according to the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment according to the position of the target at the current moment, the edge information set at the current moment and the hidden state at the current moment.

[0011] In a second aspect, the present application provides a trajectory prediction device based on spatiotemporal attention, comprising:

[0012] A spatiotemporal graph construction module is used to construct a spatiotemporal graph at the current moment according to the targets in the target scene; the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the position of the target at a certain moment; the edge information set includes: spatial edges, temporal edges and implicit edges; the spatial edges represent the relative position relationship between different targets at the same moment; the temporal edges represent the motion change relationship between the same target in adjacent time steps; the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship;

[0013] The trajectory prediction module is used to input the space-time graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment;

[0014] The trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state conversion layer; the edge coding layer is constructed based on a recurrent neural network; the state node conversion layer is constructed based on a multi-head attention mechanism and a gated recurrent unit;

[0015] The edge coding layer is used to generate the hidden state at the current moment according to the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment according to the position of the target at the current moment, the edge information set at the current moment and the hidden state at the current moment.

[0016] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described trajectory prediction methods based on spatiotemporal attention.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned trajectory prediction methods based on spatiotemporal attention.

[0018] According to the specific embodiments provided in this application, this application has the following technical effects:

[0019] The present application provides a trajectory prediction method, apparatus, device and medium based on spatio-temporal attention. According to the spatio-temporal graph (ST-Graph) at the current moment and the trajectory prediction model constructed based on the spatio-temporal attention recurrent neural network (STA-RNN), the position of the target in the target scene at the next moment is predicted. The present application adopts STA-RNN, and by combining spatio-temporal graph modeling, multi-edge coding scheme and multi-head attention mechanism, the trajectory prediction model can capture the complex spatio-temporal dependencies between targets and achieve high-precision trajectory prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 This is an application environment diagram of a trajectory prediction method based on spatiotemporal attention in one embodiment of the present application;

[0022] Figure 2 A schematic diagram of a flow chart of a trajectory prediction method based on spatiotemporal attention provided in one embodiment of the present application;

[0023] Figure 3 A schematic diagram of the structure of a time-space diagram provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of an edge coding layer provided in an embodiment of the present application;

[0025] Figure 5 An internal calculation flow chart of a multi-head attention mechanism provided in one embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of a node state conversion layer provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the entire prediction process of trajectory prediction based on spatiotemporal attention provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of functional modules of a trajectory prediction method and device based on spatiotemporal attention provided by another embodiment of the present application;

[0029] Fig. 9 A schematic diagram of a flow chart of a trajectory prediction device provided in one embodiment of the present application;

[0030] Fig.10 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0033] The trajectory prediction method based on spatiotemporal attention provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the spatiotemporal graph to the server 104. After the server 104 receives the spatiotemporal graph, for the spatiotemporal graph, the server 104 inputs the spatiotemporal graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment; wherein, the trajectory prediction model is constructed based on the spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on the recurrent neural network; the state node transition layer is constructed based on the multi-head attention mechanism and the gated recurrent unit.

[0034] The server 104 may feed back the obtained position of the target at the next moment to the terminal 102. In addition, in some embodiments, the trajectory prediction method based on spatiotemporal attention may also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 may directly process the spatiotemporal graph, or the server 104 may obtain the spatiotemporal graph from the data storage system and process the spatiotemporal graph.

[0035] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0036] In an exemplary embodiment, Figure 2 As shown, a trajectory prediction method based on spatiotemporal attention is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 201 to 208. Among them:

[0037] Step 201, constructing a spatiotemporal graph at the current moment according to the target in the target scene; wherein the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the position of the target at a certain moment; the edge information set includes: spatial edges, temporal edges and implicit edges.

[0038] The spatial edge represents the relative position relationship between different targets at the same moment; the temporal edge represents the motion change relationship of the same target between adjacent time steps; and the implicit edge represents the potential interaction relationship between targets that do not have a connection relationship.

[0039] Step 202, input the spatiotemporal graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment; wherein the trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on a recurrent neural network; the state node transition layer is constructed based on a multi-head attention mechanism and a gated recurrent unit.

[0040] The edge coding layer is used to generate the hidden state at the current moment according to the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment according to the position of the target at the current moment, the edge information set at the current moment and the hidden state at the current moment.

[0041] The target scene in the above embodiment can be application scenes such as pedestrian trajectory prediction, traffic flow prediction and robot navigation in crowded scenes. The target can be a pedestrian or an intelligent agent, etc. By implementing the above steps 201 to 202, high-precision trajectory prediction can be achieved.

[0042] In another exemplary embodiment of the present application, the space-time diagram is introduced in detail.

[0043] The multiple moving targets in the target scene are modeled as a spatiotemporal graph consisting of nodes and edges. The structure of the spatiotemporal graph is as follows: Figure 3 As shown in , each node represents the position and attributes of a target at a specific time step, while the edge is used to describe the relationship between different time steps and spatial positions. Each node represents the position and attributes of a target at a specific time step, while the edge is used to describe the relationship between different time steps and spatial positions. Based on the position of the target, the constructed spatiotemporal graph is represented as

[0044] The node information set is represented as:

[0045]

[0046] Among them, G V Represents a node information set; represents the position of target i at the current time t; The horizontal coordinate representing the position of target i at the current time t; Indicates the ordinate of the position of target i at the current time t; where is used to indicate the value range or condition of a specific variable or parameter, and is used here to indicate The value of .

[0047] The edge information set is used to connect the spatiotemporal relationships between different time steps or different targets. The edges are divided into two categories, and it is assumed that all edges are undirected.

[0048] The spatial edge connects the positions of two different targets at the same time step, indicating their proximity in space. The set of spatial edges can represent the spatial interaction of all targets at a specific time step. The spatial edge is represented as:

[0049]

[0050] in, Represents a spatial edge; represents the relative position relationship between target i and target j at the current time t; p represents the distance function.

[0051] The temporal edge connects different positions of the same target in adjacent time steps, describing the time series changes of the target. The temporal edge is used to capture the motion changes of the target in adjacent time steps. Assuming that all edges are undirected, the temporal edge is expressed as:

[0052]

[0053] in, represents the time edge; Represents the motion change relationship between target i at the current time t and the previous time t-1; Represents the position of target i at the previous time t-1.

[0054] The implicit edges include: Spatial-Spatial Edge Factor, Spatial-Temporal Edge Factor, Temporal-Spatial Edge Factor and Temporal-Temporal Edge Factor. Implicit edge factor is an analysis method for extracting indirect and complex relationships from graph structures. Traditional spatiotemporal graph methods usually only rely on spatial information or temporal information at adjacent moments, while ignoring more complex indirect relationships. For example, although two targets (such as pedestrians) are not close to each other, they may be affected by each other's movement during prediction. The embodiment is based on a feature extraction method of implicit edge factors, which can generate rich spatiotemporal dependency features by crossing different edge types in spatial and temporal dimensions. This embodiment introduces four implicit edge factors.

[0055] The space-space edge factor is expressed as:

[0056]

[0057] in, represents the space-space edge factor, which characterizes the position of target i relative to target j; represents the relative position relationship between target i and target k at the current time t; represents the relative position relationship between target k and target j at the current time t; ⊙ represents the element product, also known as the Hadamard product, which is an operator commonly used in graph computing and deep learning to multiply corresponding elements of two matrices; there is no connection relationship between target i and target j.

[0058] The space-time edge factor is expressed as:

[0059]

[0060] in, represents the space-time edge factor, characterizing the position of target i relative to the past position of target j; It represents the motion change relationship between the target j at the current time t and the previous time t-1, and represents the movement of the target j from the previous time t-1 to the current time t.

[0061] The time-space edge factor is expressed as:

[0062]

[0063] in, represents the time-space edge factor, which represents the position of target i relative to target j at the previous moment; Represents the relative position relationship between target i and target j at the previous time t-1.

[0064] The time-time edge factor is expressed as:

[0065]

[0066] Among them, F i TT (t) represents the time-time edge factor, which represents the position change of target i between the current time t and the previous time t-1; It represents the motion change relationship between target i at the previous moment t-1 and the previous moment t-2, and represents the movement of target j from the previous moment t-2 to the previous moment t-1.

[0067] By constructing a spatiotemporal graph, subsequent models can better capture dynamic changes in the scene, especially the interactions between multiple targets. These relationships are not limited to spatial interactions at the same time step, but also include temporal associations across time steps, thus providing important input information for subsequent trajectory prediction.

[0068] In another exemplary embodiment of the present application, the method for determining the trajectory prediction model includes:

[0069] (1) Constructing an edge coding layer; the edge coding layer includes multiple embedding layers and multiple recurrent neural network (RNN) layers; the embedding layers and the recurrent neural network layers are connected in a one-to-one correspondence; the number of the embedding layers and the recurrent neural network layers is determined according to the number of edge types in the edge information set; the embedding layer is used to embed a certain type of edge in the input edge information set to generate a feature vector of uniform dimension; the recurrent neural network layer is used to generate the hidden state of the edge of the type at the current moment according to the feature vector output by the corresponding embedding layer and the hidden state of the edge of the type at the previous moment. For example, the number of the embedding layers and the recurrent neural network layers is equal to the number of edge types in the edge information set, and the edge types in the edge information set include 1 spatial edge, 1 temporal edge, and 4 implicit edges, that is, the number of the embedding layers and the recurrent neural network layers can be 6.

[0070] Specifically, the role of the edge coding layer is to model and encode the relationship between objects. Since the interaction of objects in the scene can be represented by different types of edges (such as spatial edges, temporal edges, and implicit edges), the edge coding layer is specifically used to capture the information of these edges. The main task of this layer is to process different types of edge information through multiple independent RNNs. For example, spatial edges represent the relative position relationship between different objects at the same time, temporal edges represent the movement changes of objects between adjacent time steps, and implicit edges are used to capture potential interaction relationships that are not explicitly connected. Through these encodings, the model can extract more sophisticated and diverse spatiotemporal relationships from the spatiotemporal graph. The edge coding layer can accurately reflect the interactions and spatiotemporal dependencies between objects through a comprehensive analysis of different types of edge information.

[0071] The structure of the edge coding layer is as follows Figure 4 shown. Figure 4 In the example, we first select the same type of edge information connected to the same node and add them together to form a vector. This will generate six different types of vectors, corresponding to six different outputs. Then they are sent to the embedding layer for embedding. The role of the embedding layer is to convert edge information of different dimensions into a unified low-dimensional vector, which can represent So that the subsequent RNN layers can process this information uniformly.

[0072] Through the embedding operation, the original high-dimensional input features are mapped into a feature space that the model can process, and the spatiotemporal dependency information of the original edges is retained. It is input to the RNN layer for time series processing. The RNN layer is responsible for combining the edge information of each time step with the hidden state of the previous time step to generate a new hidden state This hidden state is used to represent the spatiotemporal dependencies of the target at the current time step.

[0073] Through the iterative processing of the RNN layer, the model can capture the dynamic characteristics of edge information changing over time, especially the interaction between targets in different time and space. The RNN layer also effectively alleviates the gradient vanishing problem in time series modeling, allowing the model to handle longer time dependencies.

[0074] The processed hidden state will be used as the input of the subsequent node state conversion layer to dynamically adjust the state of the target node. The hidden state of each node is not only determined by its own historical information, but also affected by the edge relations associated with it. This enables the model to flexibly simulate the spatiotemporal interactions between targets and predict future trajectories based on these interactions.

[0075] The edge encoding layer enables the model to capture the complex spatiotemporal dependencies between objects in the spatiotemporal graph through multi-type edge information input, embedding and RNN processing. It encodes and processes the interactive relationship between objects, provides key input information for updating node states, and thus enhances the model's ability to predict trajectories in dynamic scenes.

[0076] (2) Construct a state node conversion layer; the state node conversion layer includes: an embedder, a multi-head attention mechanism (Multi-HeadAttention, MHA) layer, a gated recurrent unit (GRU) and a decoder. The embedder is used to perform embedding processing according to the position of the target at the current moment to generate the embedded position at the current moment; the multi-head attention mechanism layer is used to process the hidden state of all types of edges output by all recurrent neural network layers and the embedded position at the current moment to obtain the multi-head attention mechanism output result; the gated recurrent unit is used to obtain the hidden state at the next moment according to the multi-head attention mechanism output result and the hidden state at the current moment; the decoder is used to determine the offset at the current moment according to the hidden state at the next moment; the embedded position at the current moment and the offset at the current moment are added to obtain the position of the target at the next moment.

[0077] (3) Connecting the edge coding layer and the state node conversion layer to obtain a spatiotemporal attention mechanism recurrent neural network (STA-RNN).

[0078] (4) Using training data to train the spatiotemporal attention mechanism recurrent neural network, and determining the trained spatiotemporal attention mechanism recurrent neural network as the trajectory prediction model.

[0079] This step specifically includes:

[0080] Acquire training data; the training data includes: a spatiotemporal graph of a target in a target scene at a historical moment and a real position at the next moment of the historical moment; input the training data into the spatiotemporal attention mechanism recurrent neural network, use an Adam optimizer to perform training with the goal of minimizing the loss function, obtain a trained spatiotemporal attention mechanism recurrent neural network, and determine the trained spatiotemporal attention mechanism recurrent neural network as the trajectory prediction model; the loss function represents the average deviation of the target trajectory.

[0081] The loss function adopts the mean square error function, which is expressed as follows:

[0082]

[0083] Among them, MSE represents the loss function; represents the predicted position at historical time t1; p t1 represents the real position at the historical moment t1; T represents all historical moments.

[0084] In another exemplary embodiment of the present application, the state node conversion layer is further introduced.

[0085] First, the multi-head attention mechanism layer includes: a first linear layer, a scaled dot product attention layer, a concatenation layer, and a second linear layer; the first linear layer and the scaled dot product attention layer are multiple; the first linear layer is connected to the scaled dot product attention layer in one-to-one correspondence; each of the scaled dot product attention layers is connected to the concatenation layer; the concatenation layer is connected to the second linear layer; the number of the first linear layers and the scaled dot product attention layers is greater than the number of recurrent neural network layers, the input of one of the first linear layers is the position after embedding at the current moment, the output of the recurrent neural network layer is connected to the input of the first linear layer in one-to-one correspondence, and the input of the remaining first linear layers is a zero vector. For example, the number of the first linear layers and the scaled dot product attention layers can be 8, the input of one of the first linear layers is the position after embedding at the current moment, the outputs of 6 recurrent neural network layers are connected to the inputs of 6 first linear layers in one-to-one correspondence, and the input of the remaining 1 first linear layer is a zero vector, where the zero vector is only used for filling, and it does not contain any meaningful feature information, so it will not interfere with model learning, nor will it affect the attention calculation of other input features.

[0086] The first linear layer is used to perform a linear transformation on the input data to generate a query vector, a key vector and a value vector; the scaled dot product attention layer is used to perform attention calculation on the output of the corresponding first linear layer; the splicing layer is used to splice the outputs of all scaled dot product attention layers to obtain a splicing result; the second linear layer is used to perform a linear transformation on the splicing result to obtain the output result of the multi-head attention mechanism.

[0087] Specifically, when the node state is updated, the node state conversion layer uses a multi-head attention mechanism (MHA) to process the hidden states encoded from different edges. MHA processes information from different edges in parallel at each time step, focusing on the most relevant spatiotemporal features, and strengthens attention to important information through a weight distribution mechanism. MHA enables the model to distinguish information at different time steps and spatial locations in the time series, avoiding information redundancy caused by large amounts of data.

[0088] In STA-RNN, MHA processes different spatiotemporal information from spatial edges, temporal edges, and implicit edges in parallel to better capture complex spatiotemporal dependencies. Through this multi-head mechanism, the model can simultaneously focus on the interactions between multiple targets, enhancing its ability to handle complex scenes. The internal calculation process of the multi-head attention mechanism is as follows: Figure 5 As shown, combined Figure 5 Explain the computational logic of the multi-head attention mechanism.

[0089] (1) Input and mapping of Q, K, and V.

[0090] In the multi-head attention mechanism network, the input feature vector is first (The hidden state from the RNN layer) generates Q, K, and V through different linear transformations. Each attention head has its own independent parameter matrix to generate Q, K, and V. The calculation formula is as follows:

[0091]

[0092] Among them, W Q , W K , W V It is the weight matrix that needs to be learned, which is used to calculate Query, Key, and Value respectively.

[0093] (2) Calculation of attention weight.

[0094] Next, the attention weight is obtained by calculating the similarity between the query Q and the key K. The commonly used similarity metric is the dot product, and the result is scaled and then normalized by the softmax function. The specific formula is:

[0095]

[0096] Where Q is the query vector. K T is the transpose of the key vector. k is the dimension of the query and key, used to scale the dot product to prevent the dot product value from becoming too large as the dimension increases. V is the value vector. Softmax is used for normalization so that the sum of the attention weights is 1.

[0097] (3) Calculation of multi-head attention.

[0098] In the multi-head attention mechanism, there will be multiple independent attention heads, calculating multiple different Q, K, V in parallel. Figure 5 The number of attention heads n can be 8. Each attention head will produce different weights and weighted values. For any attention head, the calculation formula is as follows:

[0099]

[0100] Finally, the outputs of all attention heads are concatenated and passed through a linear transformation W L The final multi-head attention output is calculated as follows:

[0101] MultiHead(Q,K,V)=Concat(head1,head2,...,head n )W L .

[0103] The multi-head attention mechanism can effectively improve the ability of the STA-RNN model to capture long-term and multi-dimensional spatial dependencies. By processing multiple spatiotemporal features in parallel, the model can avoid the limitations of relying on a single spatiotemporal relationship, especially when dealing with scenes with complex dynamic interactions (such as pedestrian interactions in a crowd). MHA can flexibly focus on different spatiotemporal levels, thereby generating more accurate results in trajectory prediction.

[0104] The node state conversion layer is a key component in the STA-RNN model for updating and adjusting node states. Its main function is to update the node state of the current time step based on the edge information of each time step in the spatiotemporal graph and the output of the multi-head attention mechanism, combined with the hidden state of the previous time step. The design of this layer enables the model to adapt to the trajectory changes of the target in complex dynamic scenes by continuously iteratively updating the node state. Figure 6 The network structure of the node state transition layer is shown.

[0105] Figure 6In each time step, the node state transition layer first updates the hidden state according to the output from the multi-head attention mechanism. The six hidden states in the figure represent the different hidden states obtained by the edge encoding layer after the spatial edge, temporal edge, space-space edge factor, space-time edge factor, time-space edge factor, and time-time edge factor connected to the same node from top to bottom.

[0106] This process combines the edge-level hidden state of the current time step with the node hidden state of the previous time step, and specifically uses a gated recurrent unit (GRU) to process this spatiotemporal information.

[0107] Multi-head attention output: The multi-head attention mechanism generates a set of weighted edge-level hidden states by processing inputs from different edge types. These states contain the spatiotemporal relationships between targets and are used to guide the update of node states.

[0108] GRU: It is an improved recurrent neural network that effectively controls the transmission of information through a gating mechanism, avoiding the gradient vanishing problem common in traditional RNNs. The specific functions of GRU in the node state transition layer are as follows:

[0109] Update gate: The update gate in GRU controls how much of the current hidden state comes from the information of the previous time step and how much comes from the multi-head attention result of the current time step. This ensures that the model captures short-term dependencies while not ignoring long-term historical information.

[0110] Reset gate: By controlling the reset gate, GRU can decide whether to completely rely on the current input information or retain part of the previous state. This can dynamically adjust the model's reliance on historical information, making the model adaptive at different time steps.

[0111] Updated hidden state It is the result after GRU processing, combining the historical trajectory information of the node and the current spatiotemporal edge relationship. This hidden state not only represents the state of the current time step, but also provides the necessary input for the prediction of the next time step. The current hidden state is passed through the decoder composed of the fully connected layer to obtain the offset of the node position ξ i (t), which is used to calculate the landing point at the next moment.

[0112] Since trajectory prediction is a dynamic time series task, the node state conversion layer updates the node state in each time step. Through the joint action of GRU and MHA, the node state of each time step is gradually updated iteratively, thus providing a more accurate basis for trajectory prediction in future time steps. Such iterative updates can not only capture the short-term motion trend of the target, but also capture long-term dependencies by accumulating historical information.

[0113] In general, the node state conversion layer in this application realizes the dynamic update of node state by combining the multi-head attention mechanism and GRU. By weighting the edge-level information in each time step, the node state can adapt to complex spatiotemporal changes and provide accurate input for the prediction of subsequent time steps. The introduction of GRU further enhances the model's ability to handle long-term dependencies and ensures the continuity and accuracy of trajectory prediction.

[0114] The entire prediction process of trajectory prediction based on spatiotemporal attention is as follows Figure 7 shown.

[0115] Based on STA-RNN, a trajectory prediction model combining spatiotemporal graph modeling and attention mechanism was constructed. STA-RNN accurately captures the complex spatiotemporal dependencies between targets by jointly processing spatial and temporal information. The key to this model lies in the design of the edge encoding layer and the node state transition layer. These two layers are important components that determine the performance of the model. Their synergy can effectively handle the interaction and dynamic changes of targets in the spatiotemporal dimension and improve the accuracy of trajectory prediction.

[0116] In another exemplary embodiment of the present application, pedestrian prediction is used as an example to introduce the optimization training of the trajectory prediction model. This embodiment uses the ETH and UCY dataset scenes, observes 3.2 seconds, and predicts the target trajectory of 4.8 seconds. These datasets are widely used for pedestrian trajectory prediction. There are many pedestrians in the scene, and the interactions are complex. The movement patterns between pedestrians are not only affected by the historical trajectory, but also by the dynamic behavior of the surrounding pedestrians.

[0117] Trajectory prediction optimization is a key part of STA-RNN to ensure that the model can accurately predict the future target trajectory. In order to improve the accuracy and robustness of trajectory prediction, a variety of optimization strategies are introduced in the prediction algorithm of this application, including loss function design, parameter optimization algorithm and model training method. These optimization mechanisms enable the model to perform well in complex trajectory prediction tasks and quickly converge to the ideal prediction results.

[0118] (1) Loss function design.

[0119] The mean square error (MSE) is the main loss function used to evaluate the gap between the predicted trajectory and the true trajectory. At each time step, STA-RNN measures the prediction accuracy of the model by calculating the Euclidean distance between the predicted position and the true position. Specifically, MSE is used to measure the average deviation of the target trajectory, and the calculation formula is not repeated here. By minimizing MSE, the model can continuously adjust parameters during training to improve the accuracy of trajectory prediction.

[0120] (2) Parameter optimization algorithm.

[0121] Adam optimizer: In order to speed up model convergence and ensure the stability of parameter updates, the prediction network uses the Adam optimizer. Adam combines the advantages of momentum and adaptive learning rate adjustment, allowing the model to find the global optimal solution faster during training. The learning rate during the prediction network training process of this application is set to 0.001.

[0122] (3) Model training method.

[0123] Dataset: Use the ETH and UCY datasets, which contain pedestrian motion trajectories in five scenarios.

[0124] Input data preprocessing: The trajectory data is first normalized and the coordinate range is scaled to [-1, 1] to ensure that the eigenvalues ​​of each dimension are at the same level during model training.

[0125] Observation period and prediction period: During training, the observed trajectories of the first 3.2 seconds are used as input, and the model predicts the trajectory of the next 4.8 seconds based on these trajectories.

[0126] The trajectory prediction method based on spatiotemporal attention proposed in the embodiment of the present application uses a spatiotemporal graph to represent the relationship between multiple targets (pedestrians or intelligent agents) in space and time, and on this basis encodes, updates and decodes the information of nodes and edges in the graph, uses a recurrent neural network to process time series data, and improves the prediction accuracy of the model through a variety of edge encoding mechanisms. This method is particularly suitable for trajectory prediction tasks in complex dynamic environments, such as pedestrian trajectory prediction in crowded scenes, traffic flow prediction, and robot navigation.

[0127] The embodiment of the present application focuses on the following two aspects: First, the spatiotemporal graph modeling and multi-edge encoding process: by modeling the navigation scene as a spatiotemporal graph, the dynamic changes of pedestrian positions and time are represented. The pedestrian positions in each time step are connected through spatial edges and temporal edges to establish a complex spatiotemporal relationship. Six different types of edge encoding schemes are introduced through the edge encoder layer, and multiple independent RNNs are used to process edge information, so as to learn the various connections between pedestrians and other nodes and capture implicit spatiotemporal dependencies. This scheme can effectively cope with trajectory prediction in dynamic scenes and provide accurate path calculation. The second is the trajectory prediction process based on the spatiotemporal attention mechanism: the spatiotemporal trajectory prediction is realized by combining the multi-head attention mechanism (MHA) with the gated recurrent unit (GRU). In the trajectory prediction process, the spatiotemporal edge information is first processed in parallel by MHA to capture the long-term spatiotemporal dependencies in the pedestrian trajectory, so that the model can more accurately distinguish the impact of different moments and spatial positions on future trajectories. Subsequently, the GRU is used to dynamically update the node state, and the node state is gradually updated iteratively by jointly calculating the attention output at the current moment and the node hidden state at the previous moment. Finally, the output of GRU is decoded through the fully connected layer to generate the pedestrian's future position prediction result. This method makes full use of spatiotemporal features to enhance the accuracy and robustness of trajectory prediction, and is particularly suitable for multi-step trajectory prediction tasks in complex and dynamic scenes.

[0128] The effectiveness of the trajectory prediction method based on spatiotemporal attention in this embodiment is verified through experiments below.

[0129] The trajectory prediction method based on spatiotemporal attention proposed in this application shows superior performance in multiple practical application scenarios, especially when dealing with complex dynamic scenes. The comparison algorithm is Social LSTM (SLSTM), which uses a social pooling layer combined with a long short-term memory network to receive the current position and predict the output position; Structural-RNN (SRNN) uses structured modeling to represent the scene as a bitmap, explicitly modeling the interaction between individuals and modular design for flexible deployment, and modeling different types of interactions separately; Trajectron++, which is a graph-based model that uses spatiotemporal graphs and variational autoencoder architectures to simulate multimodal trajectory distributions and predict future positions in dynamic environments; the model STA-RNN in the present invention combines spatiotemporal graph modeling with a multi-edge encoding scheme and a multi-head attention mechanism, so that the model can capture the complex spatiotemporal dependencies between targets and achieve high-precision trajectory prediction.

[0130] The simulation parameters are set as follows: This experiment uses the ETH / UCY dataset, which contains trajectory data of multiple pedestrians in different scenes. The data sampling frequency is 2.5 frames per second, with a total of 3837 scenes. The observation time is 3.2 seconds, the prediction time is 4.8 seconds, and all trajectory data are normalized to the range of [-1, 1]. The model uses the Adam optimizer with a learning rate of 0.001 and 10 training cycles. At each time step, the model predicts the trajectory through the edge encoding layer and the node state transition layer, and the dimensions of the edge embedding and node embedding are 64 and 128 respectively. The mean square error (MSE) is used as the loss function in training to ensure stable convergence.

[0131] ADE (Average Displacement Error) and FDE (Final Displacement Error) are common indicators for measuring the performance of trajectory prediction models. ADE is the average Euclidean distance between the predicted coordinates and the true historical coordinates at all time steps. FDE is the Euclidean distance between the final position of the predicted trajectory and the final position in the true trajectory. The calculation formula is as follows:

[0132]

[0133] in are the predicted coordinates and true coordinates at time t, are the predicted coordinates and true coordinates of the endpoint time T respectively.

[0134] Table 1 summarizes the performance of these four trajectory prediction models on five datasets, showing the ADE and FDE comparisons of different models.

[0135] Table 1 Performance comparison of different models on ETH / UCY dataset

[0136]

[0137] S-LSTM performs the worst on the five datasets, especially in the densely populated UCY dataset, where it lags behind the ETH dataset. The reason for this poor performance is that as the pedestrian density increases, the complexity of capturing social patterns between pedestrians also increases, and S-LSTM has difficulty in fully capturing social dynamics. SRNN is an improvement over S-LSTM, but its lack of an attention mechanism means that it cannot effectively integrate spatial relationships and time series information, nor can it focus on important states. Trajectron++ further improves trajectory prediction by using a spatiotemporal graph-based approach that can better capture spatiotemporal dependencies. However, without an explicit attention mechanism, it may still face the challenge of effectively prioritizing relevant interactions in complex environments.

[0138] In terms of ADE / FDE indicators, the model (STA-RNN) of this application is improved by 24.7% and 45.9% compared with S-LSTM, 12.7% and 25.6% compared with SRNN, and 2.11% and 4.92% compared with Trajectron++, indicating that the model of this application performs better in dense environments. Through the attention mechanism and more efficient spatial and temporal reasoning, STA-RNN can effectively utilize the edge encoding information inferred from the spatiotemporal graph, significantly outperforming previous models, thereby achieving better prediction performance.

[0139] The embodiment of the present application models multiple mobile targets in the target scene as a spatiotemporal graph structure composed of nodes and edges, wherein the node represents the state of the target at a specific time step, and the edge describes the relationship between different time steps and spatial positions. Through this modeling method, a complex spatiotemporal relationship graph is constructed to capture the spatiotemporal dependency between targets. After the modeling is completed, a multi-type edge coding scheme of spatial edges, temporal edges and implicit edges is adopted, and an independent recurrent neural network is used to process these edge information to capture the complex interactions between targets and improve the prediction accuracy. Finally, a multi-head attention mechanism is introduced in the node state conversion layer to process spatiotemporal edge information and capture the long spatiotemporal dependency in the pedestrian trajectory. The multi-head attention mechanism can process information from different edges in parallel, so that the model can predict future trajectories more accurately. At the same time, a gated recurrent unit is introduced to dynamically update the node state, combine the attention output at the current moment with the node hidden state at the previous moment, and gradually iterate to update the trajectory prediction.

[0140] Based on the same inventive concept, the embodiment of the present application also provides a trajectory prediction device based on spatiotemporal attention for implementing the trajectory prediction method based on spatiotemporal attention involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the trajectory prediction device based on spatiotemporal attention provided below can refer to the limitations of the trajectory prediction method based on spatiotemporal attention above, and will not be repeated here.

[0141] In an exemplary embodiment, Figure 8 As shown, a trajectory prediction device based on spatiotemporal attention is provided, comprising:

[0142] The spatiotemporal graph construction module 301 is used to construct a spatiotemporal graph at the current moment according to the targets in the target scene; the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the position of the target at a certain moment; the edge information set includes: spatial edges, temporal edges and implicit edges; the spatial edges represent the relative position relationship between different targets at the same moment; the temporal edges represent the motion change relationship of the same target between adjacent time steps; the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship.

[0143] The trajectory prediction module 302 is used to input the space-time graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment.

[0144] Among them, the trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state transition layer; the edge coding layer is constructed based on a recurrent neural network; the state node transition layer is constructed based on a multi-head attention mechanism and a gated recurrent unit.

[0145] The edge coding layer is used to generate the hidden state at the current moment according to the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment according to the position of the target at the current moment, the edge information set at the current moment and the hidden state at the current moment.

[0146] Furthermore, the trajectory prediction device based on spatiotemporal attention also includes: a spatiotemporal graph construction module, an edge encoding module, a node state update module and an intelligent optimization module, which are used to determine the trajectory prediction model.

[0147] The above device can accurately predict the future trajectory by analyzing and modeling the historical trajectory data of the target, and is suitable for scenarios such as autonomous driving and pedestrian prediction. The process of the trajectory prediction device is as follows Fig. 9 shown.

[0148] The historical trajectory data of the target is obtained according to the data flow graph, and the spatiotemporal graph construction module generates the spatiotemporal graph structure according to the historical trajectory of the target. The target state at each time step is modeled as a node in the space-time graph, and the spatiotemporal dependencies between targets are modeled as edges in the space-time graph. The edge encoding module encodes the edges of the space-time graph to capture the spatiotemporal relationships between targets. It includes spatial edges, temporal edges, and implicit edges, which respectively represent the relative spatial position of the target at the same time step, the motion changes in adjacent time steps, and the potential interaction relationship. The node state update module dynamically updates the node state in the spatiotemporal graph through a multi-head attention mechanism and a gated recurrent unit (GRU). This module is responsible for iteratively updating the state of the target at each time step, combining historical trajectory information with edge features for state update. The state update formula is:

[0149]

[0150] Among them, M i (t) is the output of the multi-head attention mechanism.

[0151] The trajectory prediction module predicts the location of the target in the future time step based on the updated node state. This module generates trajectory prediction results and calculates the error with the true trajectory to evaluate the prediction accuracy.

[0152] The intelligent optimization module is responsible for optimizing the model parameters and evaluating the prediction performance. Based on error feedback, the intelligent optimization module adjusts the model parameters to improve the prediction accuracy.

[0153] The present application is a trajectory prediction system based on the attention of the spatiotemporal graph. By constructing a spatiotemporal graph, the position information of each pedestrian at different time steps is represented as a node in the graph, and the interaction relationship between the time step and the space step is modeled as an edge in the graph. The edges in the spatiotemporal graph not only capture the spatial relationship between pedestrians and surrounding pedestrians, but also cover the motion changes in the time dimension, thus forming a dynamic spatiotemporal dependency model. The system uses a multi-head attention mechanism to process the spatiotemporal relationship of the target in parallel, and combines a gated recurrent neural network to update the state of the time series, which can achieve high-precision prediction of the future trajectory of the target. By introducing a dynamic edge and node state update mechanism, the system can adaptively adjust the interaction relationship between targets to adapt to a variety of complex scenarios. This method effectively improves the prediction accuracy, significantly reduces errors in high-dynamic scenes, and greatly improves computational efficiency.

[0154] In summary, the trajectory prediction system is an advanced prediction method that integrates spatiotemporal graph modeling, multi-head attention mechanism, and dynamic state update. It can achieve efficient and high-precision trajectory prediction in complex multi-target dynamic scenes, and provides a solid technical foundation for future intelligent driving, pedestrian prediction and other fields. The simulation results show that in five scenarios, the performance of the algorithm of this application is better than all the comparison algorithms, and the method has good scalability.

[0155] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.10As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a spatiotemporal graph. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a trajectory prediction method based on spatiotemporal attention is implemented.

[0156] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0157] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0158] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0159] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0160] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0161] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, etc., but is not limited thereto.

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A trajectory prediction method based on spatiotemporal attention, characterized in that: The trajectory prediction method based on spatiotemporal attention includes: A spatiotemporal graph at the current moment is constructed according to the targets in the target scene; the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the position of the target at a certain moment; the edge information set includes: spatial edges, temporal edges and implicit edges; the spatial edges represent the relative position relationship between different targets at the same moment; the temporal edges represent the motion change relationship between the same target in adjacent time steps; the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship; Input the current time-space graph into the trajectory prediction model to obtain the position of the target in the target scene at the next moment; The trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state conversion layer; the edge coding layer is constructed based on a recurrent neural network; the state node conversion layer is constructed based on a multi-head attention mechanism and a gated recurrent unit; The edge coding layer is used to generate the hidden state at the current moment according to the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment according to the position of the target at the current moment, the edge information set at the current moment and the hidden state at the current moment.

2. The trajectory prediction method based on spatiotemporal attention according to claim 1, characterized in that: The node information set is represented as: Among them, G V Represents a node information set; represents the position of target i at the current time t; The horizontal coordinate representing the position of target i at the current time t; Represents the ordinate of the position of target i at the current time t; The spatial edge is represented as: in, Represents a spatial edge; represents the relative position relationship between target i and target j at the current time t; p represents the distance function; The time edge is expressed as: in, represents the time edge; Represents the motion change relationship between target i at the current time t and the previous time t-1; represents the position of target i at the previous time t-1; The implicit edges include: space-space edge factor, space-time edge factor, time-space edge factor and time-time edge factor; The space-space edge factor is expressed as: in, represents the space-space edge factor, which characterizes the position of target i relative to target j; represents the relative position relationship between target i and target k at the current time t; represents the relative position relationship between target k and target j at the current time t; ⊙ represents the element product; there is no connection relationship between target i and target j; The space-time edge factor is expressed as: in, represents the space-time edge factor, characterizing the position of target i relative to the past position of target j; Represents the motion change relationship between target j at the current time t and the previous time t-1; The time-space edge factor is expressed as: in, represents the time-space edge factor, which represents the position of target i relative to target j at the previous moment; represents the relative position relationship between target i and target j at the previous time t-1; The time-time edge factor is expressed as: Among them, F i TT (t) represents the time-time edge factor, which represents the position change of target i between the current time t and the previous time t-1; Represents the motion change relationship between target i at the previous moment t-1 and the previous moment t-2.

3. The trajectory prediction method based on spatiotemporal attention according to claim 1, characterized in that: The method for determining the trajectory prediction model includes: Construct an edge coding layer; the edge coding layer includes multiple embedding layers and multiple recurrent neural network layers; the embedding layers and the recurrent neural network layers are connected in a one-to-one correspondence; the number of the embedding layers and the recurrent neural network layers is determined according to the number of edge types in the edge information set; the embedding layer is used to embed a certain type of edge in the input edge information set to generate a feature vector of uniform dimension; the recurrent neural network layer is used to generate the hidden state of the edge of the type at the current moment according to the feature vector output by the corresponding embedding layer and the hidden state of the edge of the type at the previous moment; Construct a state node conversion layer; the state node conversion layer includes: an embedder, a multi-head attention mechanism layer, a gated recurrent unit and a decoder; the embedder is used to perform embedding processing according to the position of the target at the current moment, and generate the embedded position at the current moment; the multi-head attention mechanism layer is used to process the hidden state of all types of edges output by all recurrent neural network layers and the embedded position at the current moment, and obtain the output result of the multi-head attention mechanism; the gated recurrent unit is used to obtain the hidden state at the next moment according to the output result of the multi-head attention mechanism and the hidden state at the current moment; the decoder is used to determine the offset at the current moment according to the hidden state at the next moment; the embedded position at the current moment and the offset at the current moment are added to obtain the position of the target at the next moment; Connecting the edge encoding layer and the state node conversion layer to obtain a spatiotemporal attention mechanism recurrent neural network; The spatiotemporal attention mechanism recurrent neural network is trained using training data, and the trained spatiotemporal attention mechanism recurrent neural network is determined as the trajectory prediction model.

4. The trajectory prediction method based on spatiotemporal attention according to claim 3, characterized in that: The multi-head attention mechanism layer includes: a first linear layer, a scaled dot product attention layer, a concatenation layer, and a second linear layer; the first linear layer and the scaled dot product attention layer are both multiple; the first linear layer is connected to the scaled dot product attention layer in a one-to-one correspondence; each of the scaled dot product attention layers is connected to the concatenation layer; the concatenation layer is connected to the second linear layer; the number of the first linear layers and the scaled dot product attention layers is greater than the number of recurrent neural network layers, the input of one of the first linear layers is the position after embedding at the current moment, the output of the recurrent neural network layer is connected to the input of the first linear layer in a one-to-one correspondence, and the inputs of the remaining first linear layers are zero vectors; The first linear layer is used to perform a linear transformation on the input data to generate a query vector, a key vector and a value vector; the scaled dot product attention layer is used to perform attention calculation on the output of the corresponding first linear layer; the splicing layer is used to splice the outputs of all scaled dot product attention layers to obtain a splicing result; the second linear layer is used to perform a linear transformation on the splicing result to obtain the output result of the multi-head attention mechanism.

5. The trajectory prediction method based on spatiotemporal attention according to claim 3, characterized in that: The spatiotemporal attention mechanism recurrent neural network is trained using training data, and the trained spatiotemporal attention mechanism recurrent neural network is determined as the trajectory prediction model, specifically including: Acquire training data; the training data includes: a spatiotemporal diagram of a target in a target scene at a historical moment and a real position at a next moment of the historical moment; The training data is input into the spatiotemporal attention mechanism recurrent neural network, and the Adam optimizer is used to perform training with the goal of minimizing the loss function to obtain a trained spatiotemporal attention mechanism recurrent neural network, and the trained spatiotemporal attention mechanism recurrent neural network is determined as the trajectory prediction model; the loss function represents the average deviation of the target trajectory.

6. The trajectory prediction method based on spatiotemporal attention according to claim 5, characterized in that: The expression of the loss function is: Among them, MSE represents the loss function; represents the predicted position at historical time t1; p t1 represents the real position at the historical moment t1; T represents all historical moments.

7. The trajectory prediction method based on spatiotemporal attention according to claim 1, characterized in that: The target is a pedestrian or an intelligent agent.

8. A trajectory prediction device based on spatiotemporal attention, characterized in that: The trajectory prediction device based on spatiotemporal attention comprises: A spatiotemporal graph construction module is used to construct a spatiotemporal graph at the current moment according to the targets in the target scene; the spatiotemporal graph includes: a node information set and an edge information set; the nodes in the node information set represent the position of the target at a certain moment; the edge information set includes: spatial edges, temporal edges and implicit edges; the spatial edges represent the relative position relationship between different targets at the same moment; the temporal edges represent the motion change relationship between the same target in adjacent time steps; the implicit edges represent the potential interaction relationship between targets that do not have a connection relationship; The trajectory prediction module is used to input the space-time graph at the current moment into the trajectory prediction model to obtain the position of the target in the target scene at the next moment; The trajectory prediction model is constructed based on a spatiotemporal attention mechanism recurrent neural network; the spatiotemporal attention mechanism recurrent neural network includes: an edge coding layer and a node state conversion layer; the edge coding layer is constructed based on a recurrent neural network; the state node conversion layer is constructed based on a multi-head attention mechanism and a gated recurrent unit; The edge coding layer is used to generate the hidden state at the current moment according to the input edge information set at the current moment and the hidden state at the previous moment; the hidden state represents the spatiotemporal dependency between targets; the state node conversion layer is used to determine the position of the target at the next moment according to the position of the target at the current moment, the edge information set at the current moment and the hidden state at the current moment.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the trajectory prediction method based on spatiotemporal attention as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the trajectory prediction method based on spatiotemporal attention described in any one of claims 1 to 7 is implemented.

Images