A trajectory prediction method based on spatiotemporal Transformer

Abstract

The present invention discloses a track prediction method based on spatiotemporal Transformer, comprising: mathematically modeling the track prediction problem, constructing a spatiotemporal graph and inputting it into an encoder based on a Transformer network to capture the current state of the target and information at historical moments, fusing the encoding features at the current moment, and putting them into an encoding feature list; modeling the interaction relationship between targets and extracting spatial interaction features; enhancing the embedding of spatial interaction relationships and characterizing temporal relationships, and extracting temporal attention features; constructing a random noise module, modeling the randomness of target motion, and outputting random Gaussian noise; inputting the spatial interaction features, temporal attention features, and random Gaussian noise into a decoder based on a fully connected layer to predict the target track, and outputting the position of the target at the future moment.

Description

Technical Field

[0001] The present invention relates to a track prediction method, in particular to a track prediction method based on spatiotemporal Transformer. Background Art

[0002] Track prediction is to predict the future motion state of targets such as ships and aircraft based on their historical motion trajectories.

[0003] Track prediction is widely used in both civil transportation and the military. In civil transportation, with the increasing number and volume of aircraft and ships, traffic congestion and navigation safety are becoming increasingly frequent. Predicting the future tracks of targets like ships and aircraft enables key technologies in air and maritime traffic management, such as traffic flow monitoring, track conflict prediction, and flight scheduling, thereby improving the efficiency and safety of traffic management. In the military, predicting the future tracks of targets like ships and aircraft allows for analysis of enemy tactical intentions and anticipation of their next move, thereby gaining an advantage on the battlefield.

[0004] Track prediction has attracted a large number of scholars both domestically and internationally to conduct research. Traditional track prediction methods are generally divided into linear models and nonlinear models. Typical linear model methods include Kalman filtering, but due to the randomness of target motion, linear models have significant limitations and cannot accurately fit the target's trajectory. Typical nonlinear model methods include particle filtering, but these methods are sensitive to parameters, lack feature expression capabilities, and do not consider the trajectory interactions between different targets. With the development of deep learning, many methods have adopted models based on convolutional neural networks or recurrent neural networks to predict target tracks in recent years. These methods learn target track patterns through data-driven approaches and have shown great superiority over traditional methods.

[0005] Although some deep learning methods for trajectory prediction have also achieved good performance, some problems in target trajectory prediction have not been fully considered, resulting in insufficient robustness and accuracy of the model: First, the decision-making process of the target is generally more active and random, and the target's motion trajectory often has multiple reasonable paths; Second, when making decisions, the target often needs to consider the motion trajectory and current position of other targets to avoid collisions or to perform a tactical task as a group, so the tracks of different targets will affect each other; Third, the motion state of the same target is different at different times, and the motion state of different targets is also different. This difference needs to be considered in the trajectory prediction process. Summary of the Invention

[0006] Purpose of the invention: The technical problem to be solved by the present invention is to provide a track prediction method based on spatiotemporal Transformer in response to the shortcomings of the existing technology.

[0007] In order to solve the above technical problems, the present invention discloses a track prediction method based on spatiotemporal Transformer, comprising the following steps:

[0008] Step 1: mathematically model the trajectory prediction problem, obtain the motion trajectories of all targets at different times, and construct a spatiotemporal graph of the target trajectory according to the appearance time and spatial position of each target;

[0009] Step 2: Construct a list of encoded features and input the spatiotemporal graph into the encoder based on the Transformer network to capture the target’s current state Fc and historical information Fh, and fuse them to obtain the current state t obs The encoding features And the encoding features of the current moment Add to coded feature list middle;

[0010] Step 3: Encode the features at the current moment Input into the spatial interaction module to model the interaction relationship between targets and extract the spatial interaction feature Fs;

[0011] Step 4: List the spatial interaction features Fs and the coded features Connect and input into the temporal attention module to enhance the embedding of spatial interaction relationships and represent temporal relationships, and extract the temporal attention feature Ft;

[0012] Step 5: Construct a random noise module to model the randomness of target motion and output random Gaussian noise Fn;

[0013] In step 6, the spatial interaction feature Fs, the temporal attention feature Ft, and the random Gaussian noise Fn are input into the decoder based on the fully connected layer to perform target track prediction, obtain the position of the target at the future moment, and complete the track prediction based on the spatiotemporal Transformer.

[0014] Beneficial effects:

[0015] This paper discloses a track prediction method based on spatiotemporal Transformer. Taking into account the mutual influence of the trajectories of different targets, the spatial interaction module is used to model the spatial interactivity of the target trajectories. Taking into account the differences in the motion states of the relationship between targets at different times, the temporal variability of the target trajectories is modeled through the temporal attention module. Furthermore, considering that the decision-making process of the target is generally subjective and random, and the target's motion trajectory often has multiple reasonable paths, the random noise module is used to model the randomness of the target trajectory. By fusing spatiotemporal features and random noise, a more robust and reasonable target track prediction is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0017] Figure 1 Schematic diagram of the steps for implementing the present invention.

[0018] Figure 2 A schematic diagram of a network structure according to an embodiment of the present invention.

[0019] Figure 3 Schematic diagram of a self-attention network layer according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention provides a track prediction method based on spatiotemporal Transformer. Taking into account the mutual influence of the tracks between different targets, the spatial interaction of the target track is modeled through the spatial interaction module; taking into account the differences in the motion states of the relationship between targets at different times, the temporal differences of the target track are modeled through the temporal attention module; and taking into account that the decision-making process of the target is generally subjective and random, and that there are often multiple reasonable paths for the target's motion track, the randomness of the target track is modeled through the random noise module. By fusing spatiotemporal features and random noise, a more robust and reasonable target track prediction is achieved, such as Figure 1 As shown, the specific steps include:

[0021] Step 1: Mathematically model the trajectory prediction problem, obtain the motion trajectories of all targets at different times, and construct a spatiotemporal graph of the target trajectory according to the appearance time and spatial position of each target. The spatiotemporal graph consists of nodes and edges, and mainly contains the following three types of values: the value of node v in the spatiotemporal graph at time step t Represents the spatial position of the target v at the tth moment; the value of the edge connecting the node u and the node v in the spatiotemporal graph at time step t Indicates the relative spatial position of target u and target v at the tth moment, if If the value is greater than the threshold thres, the edge is considered to be non-existent; in the spatiotemporal graph, the value of the edge connecting the node v at time step t-1 and the node v at time step t is Represents the relative distance and relative direction of the target v moving from the t-1th moment to the tth moment.

[0022] Step 2: Input the spatiotemporal graph into the encoder based on the Transformer network to capture the target current state Fc and historical information Fh, and fuse this information to obtain the current state t obs The encoding features And the encoding features of the current moment Add to coded feature list

[0023] Step 3: Encode the features at the current moment Input into the spatial interaction module to model the interaction relationship between targets and extract the spatial interaction feature Fs;

[0024] Step 4: List the spatial interaction features Fs and the coded features The concatenation is then input into the temporal attention module to enhance the embedding of spatial interaction relationships and characterize temporal relationships, thereby extracting the temporal attention feature Ft.

[0025] Step 5: Build a random noise module (generate a Gaussian random noise), model the randomness of the target motion, and output random Gaussian noise Fn;

[0026] In step 6, the spatial interaction feature Fs, the temporal attention feature Ft, and the random Gaussian noise Fn are input into the decoder based on the fully connected layer to predict the target track and output the position of the target at the future moment.

[0027] In one implementation, step 2 includes:

[0028] Step 2.1: The current time t in the space-time graph obs The node position set of the target n=1,2,…,N under As input, the size of the input position set vector is N*2, and the position set vector is mapped to N*N through the embedding layer. c Dimensional vector collection The formula is:

[0029]

[0030] where φ(·) represents the embedding layer function;

[0031] Step 2.2, consider the current position state relationship and capture feature information. Each eigenvector in The corresponding query matrix can be obtained Bond Matrix Sum Matrix where f Q , f K and f V They are the query function, key function and value function shared by all N targets. We define obs The relationship matrix between the nth target node and the vth target node connected by an edge in the time-space graph for The relationship matrix is ​​used to model the correlation between two nodes. obs The set of all adjacent nodes of the nth target node in the time-space graph is Nb(n), then the attention function of the information encoding at the current moment is It can be expressed as:

[0032]

[0033] Softmax is an activation function used to normalize the attention coefficient of the spatial dimension information encoding at the current moment. Qc represents the dimension of the query matrix Qc, The significance here is to decouple the distribution of attention coefficients from the dimension of the query matrix Qc so that the gradient value remains stable during training. n Expressed as:

[0034]

[0035] where f cout (n) is the output function of the information encoding at the current moment;

[0036] Step 2.3: In the space-time graph, historical moments m=1,2,…,t obs The node position set of target n=1,2,…,N under -1 As input, the size of the input position set vector is (t obs -1)*N*2, and the position set vector is mapped to (t obs -1)*N*N h Dimensional vector collection The formula is:

[0037]

[0038] where φ(·) represents the embedding layer function;

[0039] Step 2.4, model the historical dynamic motion of each target separately, without including spatial interactions. The feature vector of each target n in The corresponding query matrix can be obtained Bond Matrix Sum Matrix where f Q , f K and f V They are the query function, key function and value function shared by all N targets. Then for each target n historical moments, the multi-head attention function Att encoding the time domain information h (n) can be expressed as:

[0040]

[0041] Softmax is an activation function used to normalize the attention coefficient of the time dimension information encoding of historical moments. Qh represents the dimension of the query matrix Qh, The significance here is to decouple the distribution of attention coefficients from the dimension of the query matrix Qh so that the gradient value remains stable during training. n Expressed as:

[0042]

[0043] where f hout (n) is the output function of the historical moment information encoding, k represents the number of multi-head attention heads, Att h (n) j represents the j-th attention module for the n-th target;

[0044] Step 2.5: fuse the current state code Fc and the historical information code Fh of all targets to obtain the current time t obs The encoding features The specific fusion method is to connect the feature vector Fc and the feature vector Fh in the channel dimension, and then extract the fusion feature vector through the fully connected layer network. The expression formula is as follows:

[0045]

[0046] Among them, concat channel Represents the function of connection in the channel dimension, FClayer represents the fully connected layer network;

[0047] Step 2.6, the encoding features of the current moment Add to coded feature list

[0048] In one implementation, step 3 includes:

[0049] Step 3.1, for the current time t obs The set of encoded feature vectors The feature vector of each target in The corresponding query matrix can be obtained Bond Matrix Sum Matrix where f Q , f K and f V They are the query function, key function, and value function shared by all N targets.

[0050] Step 3.2, the spatial interaction relationship matrix Ms between the nth target node and the vth target node nv Qs n T Ks v , the relationship matrix is ​​used to model the degree of correlation between two target nodes. Define the set of all adjacent nodes to the nth target node in all N nodes in the spatiotemporal graph as Nb(n), then the attention function Att that represents the spatial interaction relationship is s (n) can be expressed as:

[0051]

[0052] Softmax is an activation function used to normalize the attention coefficient of spatial domain interaction modeling. Qs represents the dimension of the query matrix Qs, The significance here is to decouple the distribution of attention coefficients from the dimension of the query matrix Qs, so that the gradient value remains stable during training;

[0053] Step 3.3, the spatial interaction feature Fs of the nth target n Expressed as:

[0054] Fs n =f sout (Att s (n))

[0055] where f sout (n) is the output function for modeling airspace interaction relationships, specifically in the form of a fully connected layer network. The airspace interaction module models the airspace interaction relationships of targets, determines the degree of impact of target track interactions, and improves the accuracy and stability of track prediction.

[0056] In one implementation, step 4 includes:

[0057] Step 4.1, list the spatial interaction features Fs and the coded features The input Fse of the time domain attention module is obtained by connecting in the channel dimension, and the expression formula is as follows:

[0058]

[0059] Among them, concat channel Represents a function that performs concatenation in the channel dimension.

[0060] Step 4.2, for each feature vector of target n in the feature vector Fse The corresponding query matrix can be obtained Bond Matrix Sum Matrix where f Q , f K and f V are the query function, key function and value function shared by all N targets respectively. Then for each target n, the multi-head attention function Att that represents the temporal difference is t (n) can be expressed as:

[0061]

[0062] Softmax is an activation function used to normalize the attention coefficient of temporal difference modeling. Qt Represents the dimension of the query matrix Qt, The significance here is to decouple the distribution of attention coefficients from the dimension of the query matrix Qt, so that the gradient value remains stable during training.

[0063] Step 4.3, the nth target temporal attention feature Ft n Expressed as:

[0064]

[0065] where f tout (n) is the output function of the time domain attention feature capture, specifically in the form of a fully connected layer network. k represents the number of multi-head attention heads, Att t (n) j Represents the temporal difference attention module for the jth number of heads of the nth target; the temporal attention module is used to model the differences in target motion, enhance the embeddedness of spatial interaction relationships, and represent temporal relationships.

[0066] In one implementation, step 6 includes:

[0067] In step 6.1, the spatial interaction feature Fs, the temporal attention feature Ft, and the random Gaussian noise Fn are concatenated in the channel dimension to obtain the decoder input feature vector Fd, which is expressed as follows:

[0068] Fd=concat channel (Fs,Ft,Fn)

[0069] Among them, concat channel Represents a function that performs concatenation in the channel dimension.

[0070] Step 6.2: Input the feature vector Fd into the decoder based on the fully connected layer to predict the target track, and output the target n=1,2,…,N at the future time t pred The node location set The expression formula is as follows:

[0071]

[0072] Where FClayer represents the fully connected layer network.

[0073] Example:

[0074] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0075] like Figure 2 As shown, the implementation process of the present invention includes the following steps:

[0076] Step 1: Mathematically model the trajectory prediction problem, obtain the motion trajectories of all targets at different times, and construct a spatiotemporal graph of the target trajectory according to the appearance time and spatial position of each target. The spatiotemporal graph consists of nodes and edges, and mainly contains the following three types of values: the value of node v in the spatiotemporal graph at time step t Represents the spatial position of the target v at the tth moment; the value of the edge connecting the node u and the node v in the spatiotemporal graph at time step t Indicates the relative spatial position of target u and target v at the tth moment, if If the value is greater than the threshold thres, the edge is considered to be non-existent. In this embodiment, the threshold thres is set to 10. In the spatiotemporal graph, the value of the edge connected by the node v at time step t-1 and the node v at time step t is Represents the relative distance and relative direction of the target v moving from the t-1th moment to the tth moment.

[0077] Step 2: Input the spatiotemporal graph into the encoder based on the Transformer network to capture the target current state Fc and historical information Fh, and fuse this information to obtain the current state tobs The encoding features And the encoding features of the current moment Add to coded feature list

[0078] Step 3: Encode the features at the current moment Input into the spatial interaction module to model the interaction relationship between targets and extract the spatial interaction feature Fs;

[0079] Step 4: List the spatial interaction features Fs and the coded features The concatenation is then input into the temporal attention module to enhance the embedding of spatial interaction relationships and characterize temporal relationships, thereby extracting the temporal attention feature Ft.

[0080] Step 5: Construct a random noise module to model the randomness of target motion and output random Gaussian noise Fn;

[0081] In step 6, the spatial interaction feature Fs, the temporal attention feature Ft, and the random Gaussian noise Fn are input into the decoder based on the fully connected layer to predict the target track and output the position of the target at the future moment.

[0082] The step 2 includes:

[0083] Step 2.1: The current time t in the space-time graph obs The node position set of the target n=1,2,…,N under As input, the size of the input position set vector is N*2, and the position set vector is mapped to N*N through the embedding layer. c Dimensional vector collection The formula is:

[0084]

[0085] where φ(·) represents the embedding layer function; in this embodiment, N is 32, N c is 32.

[0086] Step 2.2, consider the current position state relationship and capture feature information. Figure 3 As shown, for the vector set Each eigenvector in The corresponding query matrix can be obtained Bond Matrix Sum Matrix where f Q , f K and f V They are the query function, key function and value function shared by all N targets. We defineobs The relationship matrix between the nth target node and the vth target node connected by an edge in the time-space graph for The relationship matrix is ​​used to model the correlation between two nodes. obs The set of all adjacent nodes of the nth target node in the time-space graph is Nb(n), then the attention function of the information encoding at the current moment is It can be expressed as:

[0087]

[0088] Softmax is an activation function used to normalize the attention coefficient of the spatial dimension information encoding at the current moment. Qc Indicates the dimension of the query matrix Qc. In this embodiment, N Qc is 2048-dimensional, The significance here is to decouple the distribution of attention coefficients from the dimension of the query matrix Qc so that the gradient value remains stable during training. n Expressed as:

[0089]

[0090] where f count (n) is the output function of the current moment information encoding, in this embodiment, f cout (n) is the fully connected layer network;

[0091] Step 2.3: In the space-time graph, historical moments m=1,2,…,t obs The node position set of target n=1,2,…,N under -1 As input, the size of the input position set vector is (t obs -1)*N*2, and the position set vector is mapped to (t obs -1)*N*N h Dimensional vector collection The formula is:

[0092]

[0093] where φ(·) represents the embedding layer function; in this embodiment, N h is 32.

[0094] Step 2.4, model the historical dynamic motion of each target separately, without including spatial interactions. The feature vector of each target n in The corresponding query matrix can be obtained Bond Matrix Sum Matrix where f Q , f K and f V They are the query function, key function and value function shared by all N targets. Then for each target n historical moments, the multi-head attention function Att encoding the time domain information h (u) can be expressed as:

[0095]

[0096] Softmax is an activation function used to normalize the attention coefficient of the time dimension information encoding of historical moments. Qh Indicates the dimension of the query matrix Qh. In this embodiment, N Qh is 2048-dimensional, The significance here is to decouple the distribution of attention coefficients from the dimension of the query matrix Qh so that the gradient value remains stable during training. n Expressed as:

[0097]

[0098] where f hout (n) is the output function of the historical moment information encoding, k represents the number of multi-head attention heads, in this embodiment k is 8, f hout (n) is the fully connected layer network, Att h (n) j represents the j-th attention module for the n-th target;

[0099] Step 2.5: fuse the current state code Fc and the historical information code Fh of all targets to obtain the current time t obs The encoding features The specific fusion method is to connect the feature vector Fc and the feature vector Fh in the channel dimension, and then extract the fusion feature vector through the fully connected layer network. The expression formula is as follows:

[0100]

[0101] Among them, concat channel Represents the function of connection in the channel dimension, FClayer represents the fully connected layer network;

[0102] Step 2.6, the encoding features of the current moment Add to coded feature list

[0103] In this embodiment, step 3 includes:

[0104] Step 3.1, for the current time t obs The set of encoded feature vectors The feature vector of each target in The corresponding query matrix can be obtained Bond Matrix Sum Matrix where f Q , f K and f V They are the query function, key function, and value function shared by all N targets.

[0105] Step 3.2, the spatial interaction relationship matrix Ms between the nth target node and the vth target node nv Qs n T Ks v , the relationship matrix is ​​used to model the degree of correlation between two target nodes. Define the set of all adjacent nodes to the nth target node in all N nodes in the spatiotemporal graph as Nb(n), then the attention function Att that represents the spatial interaction relationship is s (n) can be expressed as:

[0106]

[0107] Softmax is an activation function used to normalize the attention coefficient of spatial domain interaction modeling. Qs Indicates the dimension of the query matrix Qs. In this embodiment, N Qs is 2048-dimensional, The significance here is to decouple the distribution of attention coefficients from the dimension of the query matrix Qs, so that the gradient value remains stable during training;

[0108] Step 3.3, the spatial interaction feature Fs of the nth target n Expressed as:

[0109] Fs n =f sout (Att s (n))

[0110] where f sout (n) is the output function for modeling airspace interaction relationships, specifically in the form of a fully connected layer network. The airspace interaction module models the airspace interaction relationships of targets, determines the degree of impact of target track interactions, and improves the accuracy and stability of track prediction.

[0111] In this embodiment, step 4 includes:

[0112] Step 4.1, list the spatial interaction features Fs and the coded features The input Fse of the time domain attention module is obtained by connecting in the channel dimension, and the expression formula is as follows:

[0113]

[0114] Among them, concat channel Represents a function that performs concatenation in the channel dimension.

[0115] Step 4.2, for each feature vector of target n in the feature vector Fse The corresponding query matrix can be obtained Bond Matrix Sum Matrix where f Q , f K and f V are the query function, key function and value function shared by all N targets respectively. Then for each target n, the multi-head attention function Att that represents the temporal difference is t (n) can be expressed as:

[0116]

[0117] Softmax is an activation function used to normalize the attention coefficient of temporal difference modeling. Qt Indicates the dimension of the query matrix Qt. In this embodiment, N Qt is 2048-dimensional, The significance here is to decouple the distribution of attention coefficients from the dimension of the query matrix Qt, so that the gradient value remains stable during training.

[0118] Step 4.3, the nth target temporal attention feature Ft n Expressed as:

[0119]

[0120] where f tout (n) is the output function of the time domain attention feature capture, specifically in the form of a fully connected layer network. k represents the number of multi-head attention heads. In this embodiment, k is 8. Att t (n) j Represents the temporal difference attention module for the jth number of heads of the nth target; the temporal attention module is used to model the differences in target motion, enhance the embeddedness of spatial interaction relationships, and represent temporal relationships.

[0121] In this embodiment, step 6 includes:

[0122] In step 6.1, the spatial interaction feature Fs, the temporal attention feature Ft, and the random Gaussian noise Fn are concatenated in the channel dimension to obtain the decoder input feature vector Fd, which is expressed as follows:

[0123] Fd=concat channel (Fs,Ft,Fn)

[0124] Among them, concat channel Represents a function that performs concatenation in the channel dimension.

[0125] Step 6.2: Input the feature vector Fd into the decoder based on the fully connected layer to predict the target track, and output the target n=1,2,…,N at the future time t pred The node location set The expression formula is as follows:

[0126]

[0127] Where FClayer represents the fully connected layer network.

[0128] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium is capable of storing a computer program that, when executed by the data processing unit, can execute the invention content of the spatiotemporal Transformer-based trajectory prediction method provided by the present invention and some or all of the steps in each embodiment. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0129] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a computer program, i.e., a software product. The computer program software product can be stored in a storage medium and includes a number of instructions for enabling a device including a data processing unit (which can be a personal computer, server, single-chip microcomputer, MUU or network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0130] This invention provides a concept and method for track prediction based on a spatiotemporal transformer. There are numerous methods and approaches for implementing this technical solution. The above is merely a preferred embodiment of the invention. It should be noted that those skilled in the art may make improvements and modifications without departing from the principles of the invention, and such improvements and modifications are also within the scope of protection of the invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A track prediction method based on spatiotemporal Transformer, characterized in that: The steps include: Step 1: mathematically model the trajectory prediction problem, obtain the motion trajectories of all targets at different times, and construct a spatiotemporal graph of the target trajectory according to the appearance time and spatial position of each target; Step 2: Build a list of encoded features and input the spatiotemporal graph into the encoder based on the Transformer network to capture the current state of the target. and information about historical moments , and fuse to get the current moment The encoding features , and the encoding features of the current moment Add to the list of coded features [ ]middle; Step 3: Encode the features at the current moment Input into the spatial interaction module to model the interaction relationship between targets and extract the spatial interaction features ; Step 4: Then use the spatial interaction features and a list of coded features[ ] and input it into the temporal attention module to enhance the embedding of spatial interaction relationships and represent temporal relationships, and extract temporal attention features. ; Step 5: Build a random noise module to model the randomness of target motion and output random Gaussian noise. ; Step 6: Transform the spatial interaction features , temporal attention features and random Gaussian noise The input is sent to the decoder based on the fully connected layer to predict the target's trajectory, obtain the target's position at the future moment, and complete the trajectory prediction based on the spatiotemporal Transformer; The spatiotemporal graph in step 1 is composed of nodes and edges, and contains three types of values: Node Value , the space-time graph at time step Node and nodes The value of the connected edge And the space-time graph at time step Node and time steps Node The value of the connected edge ; Among them, the space-time graph is in time step Node Value Indicates the target In the The spatial position of the time step; the space-time diagram is in the time step Node and nodes The value of the connected edge Indicates the target and goals In the The relative position in space at a moment, if Greater than threshold It is considered that this edge does not exist; the space-time graph is in the time step Node and time steps Node The value of the connected edge Indicates the target From Move to the The relative distance and relative direction at a moment.

2. A track prediction method based on spatiotemporal Transformer according to claim 1, characterized in that: The method for constructing the encoding feature list described in step 2 includes: Step 2-1, mapping the node position set of the target at the current moment in the space-time graph; using the same method to map the node position set of the target at the historical moment in the space-time graph; Step 2-2, consider the current position state relationship, capture feature information, and obtain the current state of each target. The current state of the target express; Step 2-3: Model the dynamic motion of the target at each historical moment, without spatial interaction, and obtain the historical moment information of each target. Target historical moment information express; Step 2-4: Encode the current state of all targets and historical moment information encoding Fusion to get the current moment The encoding features ; Step 2-5, the encoding features of the current moment Add to the list of coded features at the previous moment [ ], get the encoding feature list [ ].

3. The track prediction method based on spatiotemporal Transformer according to claim 2, characterized in that: The method for mapping the node position set of the target at the current moment in the space-time graph in step 2-1 includes: The current moment in the space-time diagram The goal The node location set As the input position set vector, the size of the input position set vector is ,in Indicates the current time The goal The node position is mapped to the position set vector through the embedding layer in the encoder based on the Transformer network. Dimensional vector collection , which means the formula is: ； in, represents the embedding layer function; The same method is used to map the node position set of the target at the historical moment in the space-time graph, and the method includes: Transform historical moments into space-time diagrams The goal The node location set As the input position set vector, the size of the input position set vector is , the position set vector is mapped to Dimensional vector collection , which means the formula is: 。 4. The track prediction method based on spatiotemporal Transformer according to claim 3, characterized in that: The method for capturing feature information described in step 2-2 includes: For vector collections Each eigenvector in , the corresponding query matrix , bond matrix Sum Matrix ,in , and All The query function, key function, and value function shared by all targets; Defined in At this moment, the space-time graph has an edge connected The target node and The relationship matrix between target nodes for , used to model the correlation between two nodes; defined in moment, the first The set of all adjacent nodes of the target node is , then the attention function of the information encoding at the current moment is It is expressed as follows: ； in, is an activation function used to normalize the attention coefficient encoding the spatial dimension information at the current moment; Represents the query matrix Dimensions, To combine the distribution of attention coefficients with the query matrix Dimension decoupling, The current state of the target Expressed as: ； in, Output function that encodes the information at the current moment.

5. The track prediction method based on spatiotemporal Transformer according to claim 4, characterized in that: The method for obtaining the historical moment information of each target described in steps 2-3 includes: For vector collections Each target in The eigenvector of , the corresponding query matrix , bond matrix Sum Matrix , then for each target Multi-head attention function for encoding temporal information at historical moments It is expressed as follows: ； in, is an activation function used to normalize the attention coefficient encoding the time dimension information of historical moments, Represents the query matrix Dimensions, To combine the distribution of attention coefficients with the query matrix Dimension decoupling, Target historical moment information Expressed as: ； in, The output function for encoding historical moment information, Indicates the number of heads of multi-head attention, Indicates that for The first target The number of attention modules.

6. The track prediction method based on spatiotemporal Transformer according to claim 5, characterized in that: Encode the current state of all targets as described in steps 2-4 and historical moment information encoding Fusion to get the current moment The encoding features , the specific fusion method is: The current state code is the feature vector and historical moment information encoding, i.e., feature vector Connect in the channel dimension, and then extract the fusion feature vector through the fully connected layer network. The expression formula is as follows: ； in, represents the function for connection in the channel dimension, Represents a fully connected layer network.

7. The track prediction method based on spatiotemporal Transformer according to claim 6, characterized in that: The airspace interaction module described in step 3 specifically includes: Step 3-1, for the current moment The set of encoded feature vectors The feature vector of each target in , the corresponding query matrix , bond matrix Sum Matrix ; Step 3-2, The target node and The spatial interaction relationship matrix between target nodes for , used to model the correlation between two target nodes; Define all in the space-time graph The node with The set of all adjacent nodes of a target node is , then the attention function that characterizes the spatial interaction relationship It is expressed as follows: ； in, is the activation function used to normalize the attention coefficient for modeling spatial interaction relationships. Represents the query matrix Dimensions, To combine the distribution of attention coefficients with the query matrix Dimension decoupling; Step 3-3, Spatial interaction characteristics of targets Expressed as: ； in, The output function for modeling spatial interaction relationships is a fully connected layer network; the set of spatial interaction features of all targets is the spatial interaction feature .

8. The track prediction method based on spatiotemporal Transformer according to claim 7, characterized in that: The temporal attention module described in step 4 specifically includes: Step 4-1: Spatial interaction features and the encoding feature list of the previous moment[ ] Connect in the channel dimension to get the input of the time domain attention module , which is expressed as follows: ； in, represents the function of connection in the channel dimension; Step 4-2, for the feature vector Each target in The eigenvector of , the corresponding query matrix , bond matrix Sum Matrix , then for each target Multi-head attention function representing temporal differences Expressed as: ； in, is the activation function used to normalize the attention coefficient for modeling temporal differences; Represents the query matrix Dimensions, To combine the distribution of attention coefficients with the query matrix Dimension decoupling; Step 4-3, target temporal attention features Expressed as: ； in, is the output function captured by the time domain attention feature, specifically in the form of a fully connected layer network. Indicates the number of heads of multi-head attention, Indicates that for The first target The temporal difference attention module of the number of targets; the set of temporal attention features of all targets is the temporal attention feature .

9. The track prediction method based on spatiotemporal Transformer according to claim 8, characterized in that: The specific method for obtaining the position of the target at the future time in step 6 includes: Step 6-1, the spatial interaction features , temporal attention features and random Gaussian noise Perform channel dimension connection to obtain the input feature vector of the decoder , expressed as follows: ； in, represents the function of connection in the channel dimension; Step 6.2: The decoder’s input feature vector Input to the decoder based on the fully connected layer network to predict the target track and output the target In the future The node location set , expressed as follows: ； in, Represents a fully connected layer network.

Images