Vehicle trajectory prediction method based on time-space fusion attention

Abstract

The invention discloses a vehicle trajectory prediction method based on time-space fusion attention. The method comprises a data preprocessing module DPM, a data preprocessing module DPM, a time feature extraction module TEM, a spatial feature extraction module SEM and a trajectory prediction module TPM. According to the method, the dynamic undirected graph is used for representation, the vehicles serve as nodes, the edges represent the interaction relation between the vehicles, the weights are distributed for the edges, the random noise obeying normal distribution is sampled, the state vectors are added, the robustness of the model can be enhanced, and accurate vehicle track prediction is achieved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle trajectory prediction, and in particular to a vehicle trajectory prediction method based on spatiotemporal fusion attention. Background Art

[0002] In modern traffic systems, the development of vehicle trajectory prediction technology is crucial to improving traffic safety, optimizing traffic flow, and promoting the development of autonomous driving technology. Especially in the context of the continuous evolution of autonomous driving technology, accurate prediction of vehicle trajectories has become increasingly important. Vehicle trajectory prediction aims to predict the future driving trajectory of a vehicle by analyzing the vehicle's historical driving data and current environmental information, so as to take necessary measures in advance to ensure traffic safety and smoothness. In most autonomous driving technologies, the vehicle trajectory prediction module is located between the vehicle perception module and the decision-making module. By analyzing the driving information collected by the vehicle perception module, the future trajectory and behavioral intentions of traffic participants can be predicted, and then the decision-making module will decide the future behavior of the vehicle based on the prediction results. The accuracy of the vehicle's future trajectory prediction directly determines the driving safety of autonomous vehicles. Summary of the invention

[0003] The purpose of the present invention is to propose a vehicle trajectory prediction method based on spatiotemporal fusion attention in order to solve or at least partially solve the technical problem of insufficient prediction accuracy in current vehicle trajectory prediction research in view of the above-mentioned problems existing in the prior art.

[0004] The above-mentioned purpose of the present invention is achieved by the following technical means:

[0005] A vehicle trajectory prediction method based on spatiotemporal fusion attention includes the following steps:

[0006] Step 1: Use the data preprocessing module DPM to generate the historical time step t of the environment vehicle corresponding to each moment h The trajectory data sequence and the historical time step t of the target vehicle h The trajectory data sequence of the target vehicle is generated by the historical time step t h The trajectory data sequence corresponding to t f The trajectory data of the time step is used as a label to construct the historical time step t h The dynamic weighted graph at each moment in ;

[0007] Step 2: Use the time feature extraction module TEM to extract the historical time step t of the target vehicle. h The trajectory data sequence is embedded and encoded, and the outputs of the embedding and position encoding are directly added to obtain the embedded sequence E, which is input to the multi-head self-attention layer. The output of the multi-head self-attention layer is passed through the residual structure and the historical time step t of the target vehicle.h After the trajectory data sequence is connected and normalized, it is input into the feedforward neural network for nonlinear transformation through the ReLU activation function, and residual connection and normalization are performed to obtain the semantic code h that represents the temporal characteristics of the target vehicle trajectory data. temp ;

[0008] Step 3: Use the spatial feature extraction module SEM to extract the historical time step t h The dynamic weighted graph at each moment in is input into the graph attention neural network GAT, which outputs the hidden vector S, which is input into the gated recurrent unit GRU, and the gated recurrent unit GRU outputs the semantic code h that represents the spatial features. spa ;

[0009] Step 4: Use the trajectory prediction module TPM to convert the semantic code h temp With semantic encoding h spa Fusion is performed to obtain the fusion feature h fusion , random noise R and fusion feature h fusion After concatenation, the state vector H is obtained. The gated recurrent unit GRU converts the state vector H into a semantic code as the input of the multi-layer perceptron MLP. The multi-layer perceptron MLP outputs the future t f Target vehicle trajectory within the time step

[0010] As described above, in the data preprocessing module DPM, the time window is continuously moved on the trajectory data sorted by global time, and the entire trajectory data sequence of the target vehicle is divided into time lengths t h +t f The trajectory data unit, the first t of the trajectory data unit h The trajectory data sequence of time steps is used as the input trajectory data for prediction. The trajectory data unit after t f The trajectory data of the time step is used as the label, and the t of the surrounding vehicle h The trajectory data sequence of the time step is related to the target vehicle t h The time steps correspond to each moment of the trajectory data sequence.

[0011] As mentioned above, in the data preprocessing module DPM, the historical time length t is constructed h From the first moment to the tth moment h The undirected graph G at time 1 ,···, The environment vehicles and the target vehicle build edges is the node formed by the target vehicle tar at time t, is the node formed by the environment vehicle j at time t, and the edge Add to edge set E, and represents the lane number of the target vehicle tar and the environment vehicle j at time t, represents the Euclidean distance between the target vehicle tar and the environment vehicle j at time t, and the edge The corresponding weight w tar,j Based on the following formula:

[0012]

[0013] The weighted adjacency matrix A at time t t It is expressed as:

[0014]

[0015] The trajectory data corresponding to the p-th vehicle at the t-th time is taken as the feature vector The feature vector After embedding is mapped to a high dimension, the weighted adjacency matrix A is used as the attribute value of the node of the corresponding undirected graph. t and contains the eigenvector The nodes together constitute the historical time step t h The dynamic weighted graph at each moment in .

[0016] Embed the sequence as above e 1 ,···, 1 to t respectively h The output of the embedding code and position code corresponding to the time is added. In the multi-head self-attention layer, the embedding sequence E is respectively combined with the weight matrix of the Transformer encoder Weight Matrix Weight Matrix Multiply them together to generate Q k Matrix, K k Matrix, V k Matrix, k represents the number of the attention head,

[0017]

[0018] head k =Attention(Q k ,K k ,V k )

[0019] Among them, softmax() is the softmax function, Attention() is the attention head, MultiHead is the output of the multi-head attention layer, head k is the kth attention head, W Ois the additional weight matrix, T is the transpose, and d represents the dimension of the trajectory data after embedding encoding.

[0020] As mentioned above, in the graph attention neural network GAT, the correlation coefficient e between the target node and its neighbor nodes in the dynamic weighted graph is calculated tar,j

[0021]

[0022] Among them, e tar,j is the correlation coefficient, LeakyReLU() is the LeakyReLU activation function, It is a single-layer feedforward neural network of the graph attention neural network GAT, W and W e is the learnable parameter matrix, is the attribute value of the target node formed by the target vehicle at time t, is the attribute value of the neighboring node composed of the environment vehicles at time t, w tar,j is the weight of the edge between the target node and its neighbor node,

[0023] Correlation coefficient e tar,j Normalize and get the normalized attention coefficient α tar,j ,in represents the first-order neighbor nodes of the target node,

[0024]

[0025] Compute splicing output

[0026] Among them, || represents the splicing operation, represents the attention coefficient of the kth attention head, W k is the parameter matrix corresponding to the kth attention head, σ is the sigmoid activation function, M is the number of attention heads,

[0027] Splicing output As s 1 ,···, s 1 ,···, 1 to t respectively h The hidden vector element at the moment, obtain the hidden vector

[0028] As mentioned above, the fusion feature h fusion Calculated based on the following formula:

[0029] h fusion =z⊙h spa +(1-z)⊙h temp

[0030] z=σ(W s h spa +W t h temp +b)

[0031] Among them, W s , W t and b are model parameters, ⊙ is the Hadamard product, σ is the sigmoid activation function, and z is the fusion gate.

[0032] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements each step of the above prediction method when executing the computer program.

[0033] A computer-readable storage medium stores a computer program, which implements the various steps of the above prediction method when executed by a processor.

[0034] A computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above prediction method are implemented.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. Spatial interaction relationship of joint modeling of distance and lane variables: In order to accurately characterize the spatial interaction relationship between different vehicles, the present invention uses a dynamic undirected graph for representation, in which vehicles are nodes and edges represent the interaction relationship between vehicles. In order to characterize the differences in interaction relationships, the key step is to assign weights to edges. Existing studies only considered calculating edge weights based on distance variables, while the present invention takes lane variables into consideration on this basis and involves them in weight calculation together with distance variables.

[0037] 2. Use random noise to characterize random factors: During vehicle driving, there are many random factors such as random events in the environment and random intentions of the driver. The present invention samples random noise that obeys normal distribution and adds it to the state vector, which can enhance the robustness of the model and achieve accurate vehicle trajectory prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is the principle structure diagram of the present invention;

[0039] Figure 2 This is an example diagram of sliding window data segmentation in an embodiment of the present invention;

[0040] Figure 3 A diagram of a method for selecting an environmental vehicle in an embodiment of the present invention;

[0041] Figure 4The process diagram of constructing a dynamic weighted adjacency matrix in the embodiment of the present invention is shown in FIG. 0 is the weighted adjacency matrix corresponding to the instance time;

[0042] Figure 5 Schematic diagram of a gated recurrent unit in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0044] A vehicle trajectory prediction method based on spatiotemporal fusion attention includes the following steps:

[0045] Step 1: Use the data preprocessing module DPM to generate the historical time step t of the environment vehicle corresponding to each moment h The trajectory data sequence and the historical time step t of the target vehicle h The trajectory data sequence of the target vehicle is generated by the historical time step t h The trajectory data sequence corresponding to t f The trajectory data of the time step is used as a label to construct the historical time step t h The dynamic weighted graph at each moment in ;

[0046] Trajectory data segmentation based on sliding windows. Sliding windows are a common data processing method in time series prediction tasks, which is used to convert time series data into samples that can be used to train supervised learning models. The present invention continuously moves the time window on the trajectory data sorted by global time, thereby segmenting the entire trajectory data sequence of the target vehicle into a series of time lengths t. h +t f The trajectory data unit, the first t of the trajectory data unit h The trajectory data sequence of time steps is used as the input trajectory data for prediction. The trajectory data unit after t f The trajectory data of the time step is used as the label. The environment vehicles are divided into t h The trajectory data sequence of time steps is used as the input trajectory data for prediction, and the t h The trajectory data sequence of the time step is related to the target vehicle t h The time step of the trajectory data sequence corresponds to each moment. That is, the time step through the target vehicle and the environment vehicle is t h The input trajectory data of the same time is used to predict the target vehicle's subsequent time step t f Trajectory data sequence. Figure 2 shown.

[0047] (1) Building a Dynamic Weighted Graph

[0048] The dynamic graph-based method can capture complex vehicle interaction relationships. The present invention uses a dynamic graph G = {V, E} to simulate the effects of environmental vehicles on target vehicles (the nodes in the node set V and the edges in the edge set E represent the spatial interaction relationships (distance relationships) between vehicles and vehicles, respectively). The dynamic graph is used as the input of the spatial feature extraction module. h At each moment in the , an undirected graph is constructed, expressed as G 1 ,···, The time length t h From the first moment to the tth moment h Undirected graph at the moment. But it should be noted that It is not fixed, because the number of nodes at different times may be different. The present invention assumes that all undirected graphs are constructed on a node set with n+1 nodes, that is, there is 1 target vehicle and n environment vehicles. Existing vehicles are regarded as actual nodes, and non-existent vehicles are regarded as virtual nodes. A virtual node refers to a symbolic vehicle representation that does not have any characteristics and is introduced only for the integrity of the undirected graph. Since the present invention models the interaction between vehicles as an undirected graph, the structure of the undirected graph will significantly affect the performance and efficiency of the model. If the undirected graph is a complete graph (each node is connected to other nodes), there will be redundant connections, which will grow quadratically with the increase in the number of nodes, thereby affecting the model performance. Therefore, the present invention constructs the undirected graph G as a star-shaped undirected graph centered on the target vehicle node.

[0049] In a vehicle driving scenario, not all environmental vehicles on the road will have an impact on the target vehicle. Spatial interaction only occurs when the distance between the environmental vehicle and the target vehicle is close. Therefore, in order to reduce the interference of irrelevant information and increase the model's attention to effective features, it is necessary to adopt a reasonable selection strategy to screen the environmental vehicles, that is, to select the neighbors of the target vehicle in the undirected graph. The present invention limits the selection range to: the first n environmental vehicles that are within a circle with a radius of L with the target vehicle as the center and the closest Euclidean metric. The present invention selects L=30 as the neighbor vehicle selection range.

[0050] Examples of environmental vehicle selections include Figure 3As shown in the figure, the vehicles within the yellow circle are considered as neighbors of the target vehicle. If n=5, the vehicles a, b, c, d, and e in the figure are used to build the map (the black line at the tail of each vehicle in the figure represents its past trajectory). If there are no n environmental vehicles in the circle, virtual nodes are used instead in the figure. The virtual nodes do not have trajectory data, and the trajectory data of the virtual nodes is 0. The environmental vehicles that meet the selection conditions build edges with the target vehicle. is the node formed by the target vehicle tar at time t, is the node formed by the environment vehicle j at time t, and is added to the edge set E, indicating that it is a neighbor of the target vehicle.

[0051] In driving scenarios, the impact of vehicles on other vehicles is different. For example, if a vehicle suddenly slows down, it may cause vehicles that are closer to it to slow down or change lanes, but it will have little effect on vehicles that are farther away. At the same time, even if the distance between vehicles is the same, the behavior of a vehicle may have a greater impact on vehicles in the same lane than on vehicles in adjacent lanes. Therefore, in order to distinguish the intensity of interaction between vehicles, it is necessary to characterize this difference by assigning different weights to the edges in the undirected graph. The present invention analyzes the driving behavior of vehicles in real scenarios and finds that lane factors also play an important role in differentiated characterization. The present invention believes that even when the distance between vehicles is the same, different lanes will lead to differences in the intensity of interaction. Specifically, there will be a stronger interaction relationship between vehicles in the same lane. Therefore, the weight calculation formula defined by the present invention is as shown in Formula 1, where and represents the lane number of the target vehicle tar and the environment vehicle j at time t, represents the Euclidean distance between the target vehicle tar and the environment vehicle j at time t, and the edge The corresponding weight w tar,j Based on the following formula:

[0052]

[0053] By assigning weights to the edges of the undirected graph, we can obtain the weighted adjacency matrix at each moment. The weighted adjacency matrix at moment t is It is expressed as:

[0054]

[0055] A simple example of constructing a weighted adjacency matrix is Figure 4 ,Assume that a, b, c, d, and e in the figure are selected environment vehicles, and they are all neighbors of the target vehicle in the dynamic weighted graph.

[0056] The weighted adjacency matrix A is defined tThen, the trajectory data corresponding to the p-th vehicle at the t-th moment is taken as the feature vector The feature vector After embedding is mapped to a high dimension, the weighted adjacency matrix A is used as the attribute value of the node of the corresponding undirected graph. t and contains the eigenvector (The feature vector of each node constitute Figure 1 The nodes of the feature matrix in ) together constitute the historical time step t h The dynamic weighted graph at each moment in is used as the input of the spatial feature extraction module.

[0057] Step 2: Use the time feature extraction module TEM to extract the historical time step t of the target vehicle. h The trajectory data sequence is embedded and encoded, and the outputs of the embedding and position encoding are directly added to obtain the embedded sequence E, which is input to the multi-head self-attention layer. The output of the multi-head self-attention layer is passed through the residual structure and the historical time step t of the target vehicle. h After the trajectory data sequence is connected and normalized, it is input into the feedforward neural network for nonlinear transformation through the ReLU activation function, and residual connection and normalization are performed to obtain the semantic code h that represents the temporal characteristics of the target vehicle trajectory data. temp ;

[0058] In the time dimension, the future trajectory of a vehicle is highly dependent on its historical trajectory. For example, a vehicle that is accelerating is likely to continue to accelerate in the future. Therefore, the present invention introduces a temporal feature extraction module to extract the key motion patterns and temporal feature information of the target vehicle. The temporal feature extraction module TEM uses a 6-layer Transformer encoder, whose input is the historical time step t of the target vehicle. h The input trajectory data is the semantic code h representing the temporal features. temp .

[0059] First, we need to calculate the historical time step t of the target vehicle. hThe trajectory data sequence is embedded and positionally encoded. Embedded coding refers to mapping the trajectory data sequence into a high-dimensional vector so that the model can better capture the features in the trajectory. In the trajectory prediction task, each trajectory data in the trajectory data sequence corresponds to a moment, and the model needs to understand the relationship between these moments. Unlike the serial processing input of the recurrent neural network model, the self-attention mechanism in the Transformer encoder processes the input in parallel, so the relative position relationship of the trajectory data in the trajectory data sequence cannot be taken into account. For this reason, position encoding is introduced to assign a unique vector to each trajectory data in the trajectory data sequence so that the model can distinguish inputs at different moments. The present invention uses formulas 3 and 4 for position encoding, which respectively represent the encoding methods for even moments and odd moments, where p represents the position of the moment in the trajectory data sequence, f represents the index of the dimension (starting from 0 and increasing), and d represents the dimension of the trajectory data after embedded encoding.

[0060]

[0061] PE(p,2f) is the position code corresponding to an even number, PE(p,2f+1) is the position code corresponding to an odd number, and the historical time step t of the target vehicle h The input trajectory data of has the same data dimension after embedding coding and position coding respectively. The outputs of embedding coding and position coding are directly added to obtain the embedding sequence for multi-head self-attention layer calculation. e 1 ,···, 1 to t respectively h The output of the embedding code and position code corresponding to the time is added, and the elements in the embedding sequence E are concatenated for parallel calculation. Then the multi-head self-attention layer is input to capture the dependency of trajectory data at different times. The embedding sequence E is respectively combined with the weight matrix of the Transformer encoder Multiply them together to generate Q k , K k 、V k Matrix, k represents the number of attention heads. Q k , K k The matrix is ​​multiplied by the dot to generate the score matrix, which is used to determine the correlation between moments. The score matrix is ​​scaled and processed by the softmax function to obtain the attention weight matrix. Finally, the attention weight matrix is ​​added to V k The matrix is ​​multiplied to obtain the output of the current kth attention head, as shown in Formula 5. In order to enrich the expressive power of the temporal feature extraction module, a multi-head attention mechanism is used, that is, h independent attention heads are used to perform calculations simultaneously, and the outputs of all attention heads are concatenated before the final linear layer, as shown in Formula 6, where || is the concatenation operation, and W Ois the additional weight matrix.

[0062]

[0063] head k =Attention(Q k ,K k ,V k ) (6)

[0064] Among them, softmax() is the softmax function, Attention() is the attention head, MultiHead is the output of the multi-head attention layer, head k is the kth attention head, T is the transpose,

[0065] Finally, the output of the multi-head self-attention layer is passed through the residual structure and the historical time step t of the target vehicle. h After the trajectory data sequence is connected and normalized, it is input into the feedforward neural network for nonlinear transformation through the ReLU activation function, and then residual connection and normalization are performed again, and finally the semantic code h that represents the temporal characteristics of the target vehicle trajectory data is obtained. temp .

[0066] Step 3: Use the spatial feature extraction module SEM to extract the historical time step t h The dynamic weighted graph at each moment in is input into the graph attention neural network GAT, which outputs the hidden vector S, which is input into the gated recurrent unit GRU, and the gated recurrent unit GRU outputs the semantic code h that represents the spatial features. spa ;

[0067] In highly dynamic driving scenarios, modeling the spatial relationship between vehicles is crucial to improving trajectory prediction accuracy. This is because in the spatial dimension, the future trajectory of a vehicle depends on the movement of the vehicles around it. And because the graph attention neural network GAT can effectively utilize the graph structure and capture the differential interaction relationship between nodes in space, the present invention uses it to extract spatial features. At the same time, there is also a temporal dependency between the spatial interaction relationships at different moments in the historical time step, which needs to be further extracted. Therefore, the present invention proposes a spatial feature extraction module based on the graph attention neural network GAT and the gated recurrent unit GRU to capture the spatial dependency between vehicles. The spatial feature extraction module first uses the graph attention neural network GAT to extract the spatial relationship features between the target vehicle and the environment vehicles at each moment, and its input is the historical time step t h The dynamic weighted graph at each moment in the output is the hidden vector that represents the interaction relationship at each moment s 1 ,···, 1 to t respectively h Then, the gated recurrent unit GRU is used to extract the historical time step t h The temporal dependency of the spatial interaction relationship in the CNN is represented by the hidden vector S, and the output is the semantic code h that represents the spatial features. spa .

[0068] Firstly, a graph attention neural network (GAT) is used to capture the spatial interaction relationship between the target vehicle and the environment vehicles at each moment, and a multi-head attention mechanism is added to enhance the generalization ability of the model.

[0069] The first step is to calculate the attention coefficient α between the target node (target vehicle) and its neighbor nodes (environmental vehicles) in the dynamic weighted graph. tar,j .

[0070] First, use Formula 7 to calculate the correlation coefficient e between the target node (target vehicle) and its neighbor nodes (environmental vehicles) in the dynamic weighted graph tar,j , where W and W e is a learnable parameter matrix, || represents the concatenation operation, It is a single-layer feedforward neural network of the graph attention neural network GAT, which is used to map the concatenated high-dimensional features to a real number. Finally, the LeakyReLU activation function is used to implement nonlinear mapping. The edge weights w in the weighted adjacency matrix tar,j It participates in the attention coefficient calculation by concatenating with the node features. It is worth noting that in different attention head calculations, W and W e are different.

[0071]

[0072] Among them, e tar,j is the correlation coefficient, LeakyReLU() is the LeakyReLU activation function, It is a single-layer feedforward neural network of the graph attention neural network GAT, W and W e is the learnable parameter matrix, is the attribute value of the target node formed by the target vehicle at time t (in the data preprocessing module DPM, the trajectory data corresponding to the p-th vehicle at the t-th time is used as the feature vector The feature vector After embedding is mapped to a high dimension, it is used as the attribute value of the node of the corresponding undirected graph). is the attribute value of the neighboring node composed of the environment vehicles at time t, w tar,j is the weight of the edge between the target node and its neighbor node.

[0073] Then the correlation coefficient e tar,j Normalize to make the weight distribution more effective and get the normalized attention coefficient α tar,j , as shown in Formula 8, where Represents the first-order neighbor nodes of the target node.

[0074]

[0075] In the second step, each attention head performs a weighted summation on the attribute value of the target node, and then concatenates the outputs of M independent attention heads, as shown in Formula 9. Where || represents the concatenation operation, represents the attention coefficient obtained by the kth attention head, W k is the parameter matrix corresponding to the kth attention head, and σ is the sigmoid activation function.

[0076]

[0077] For splicing output,

[0078] The above process is executed repeatedly to obtain the h The spatial interaction characteristics between the target vehicle and the environment vehicle, that is, the hidden vector s 1 ,···, 1 to t respectively h Hidden vector elements of the moment, As s 1 ,···, The hidden vector is then used as the input of the gated recurrent unit GRU to extract the temporal dependency between the spatial interaction features at different times. The calculation process of the gated recurrent unit GRU is shown in formula 10, where r g 、z g and They are reset gate, update gate and candidate hidden state, h g is the hidden state at the current moment, x g Represents the hidden vector element at the current moment, h g-1 Represents the hidden vector element of the previous moment, W r , U r , W z , U z , W h and U h is the learnable parameter matrix, b r , b z and b his the learnable bias, ⊙ is the Hadamard product, and σ represents the sigmoid activation function. Through formula 10, all hidden vector elements and output vector elements can be cyclically updated to obtain all output hidden vector elements h g After concatenation, we get the semantic code h spa , and finally outputs the semantic code h representing the spatial features spa .

[0079]

[0080] Step 4: Use the trajectory prediction module TPM to convert the semantic code h temp With semantic encoding h spa Fusion is performed to obtain the fusion feature h fusion , random noise R and fusion feature h fusion After concatenation, the state vector H is obtained. The gated recurrent unit GRU converts the state vector H into a semantic code as the input of the multi-layer perceptron MLP. The multi-layer perceptron MLP outputs the future t f Target vehicle trajectory within the time step

[0081] The temporal feature extraction module and the spatial feature extraction module are used to capture the semantic encoding h that characterizes the temporal features of the target vehicle trajectory data. temp and the semantic encoding h that represents the spatial features spa After that, the semantic code h temp and semantic encoding h spa The feature fusion is performed to generate the predicted vehicle trajectory distribution. This is a typical sequence generation problem. For this reason, the present invention designs a trajectory prediction module to generate the target vehicle’s future t f The trajectory prediction module first uses the gated feature fusion module to transform the semantic code h temp With semantic encoding h spa Perform adaptive feature fusion to obtain the fusion feature h fusion Then random noise sampling is performed to combine the random noise with the fusion feature h fusion After splicing, the input is decoded by the decoder composed of the gated recurrent unit GRU and the multi-layer perceptron MLP to obtain the final predicted trajectory.

[0082] First, the gated feature fusion module is used to transform the semantic code h temp With semantic encoding h spa To merge, the structure is as follows Figure 5 The gated feature fusion module introduces a learnable gate unit to dynamically control the semantic encoding h temp With semantic encoding h spa The weight of , thereby realizing the semantic encoding h tempWith semantic encoding h spa The feature fusion process is as shown in formula 11, the fusion feature h fusion is obtained by fusion gate z, and the calculation formula is shown in 12, where W s , W t and b are learnable model parameters, ⊙ is the Hadamard product, and σ is the sigmoid activation function.

[0083] h fusion =z⊙h spa +(1-z)⊙h temp (11)

[0084] z=σ(W s h spa +W t h temp +b) (12)

[0085] Then, random noise is added. In actual vehicle driving scenarios, there are many random factors, such as the driver's random intentions and random changes in the traffic environment. Therefore, the present invention introduces random noise R~N(0,1) that obeys the standard normal distribution to characterize the random factors in the vehicle driving process, so as to improve the robustness of the model. fusion After concatenation, the state vector H of the input decoder is obtained.

[0086] H=h fusion ||R (13)

[0087] Finally, the predicted trajectory is generated. The decoder includes a gated recurrent unit GRU and a multi-layer perceptron MLP. In order to stabilize the training process and prevent the gradient from disappearing or exploding, and to further extract features, the present invention first uses a gated recurrent unit GRU in the decoder to convert the state vector H into a semantic code and then uses the semantic code as the input of the multi-layer perceptron MLP. The calculation process of the gated recurrent unit GRU is the same as formula 10. Finally, the multi-layer perceptron MLP is used to generate the future t f Target vehicle trajectory within the time step

[0088]

[0089] Among them, D_GRU() is the gated recurrent unit GRU, W d is the learnable parameter matrix of the gated recurrent unit GRU (including the learnable parameter matrix W r , U r , W z , U z , W h and U h ).

[0090] In this implementation method, the present invention realizes a vehicle trajectory prediction model based on spatiotemporal fusion attention, which achieves accurate prediction of the vehicle's future trajectory by extracting the temporal and spatial features from the historical trajectories of the target vehicle and the environment vehicles and adaptively fusing the two.

[0091] The total loss function of the present invention adopts the MSELoss function.

[0092] After the vehicle trajectory prediction model is trained, the data preprocessing module DPM generates the historical time step t of the environment vehicle corresponding to each moment. h The trajectory data sequence and the historical time step t of the target vehicle h The trajectory data sequence (not used to generate the target vehicle historical time step t h The trajectory data sequence corresponding to t f The trajectory data of the time step is used as a label) to construct the historical time step t h The dynamic weighted graph at each moment in the future is further predicted using the time feature extraction module TEM, the space feature extraction module SEM, and the trajectory prediction module TPM. f Target vehicle trajectory within a time step.

[0093] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0094] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements each step in the above-mentioned prediction methods when executing the computer program.

[0095] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, each step of the above prediction method is implemented.

[0096] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the above prediction method when executed by a processor.

[0097] It should be noted that the embodiments described in the present invention are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the described embodiments or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the attached claims.

Claims

1. A vehicle trajectory prediction method based on spatiotemporal fusion attention, characterized in that: The following steps are involved: Step 1: Use the data preprocessing module DPM to generate the historical time step t of the environment vehicle corresponding to each moment h The trajectory data sequence and the historical time step t of the target vehicle h The trajectory data sequence of the target vehicle is generated by the historical time step t h The trajectory data sequence corresponding to t f The trajectory data of the time step is used as a label to construct the historical time step t h The dynamic weighted graph at each moment in ; Step 2: Use the time feature extraction module TEM to extract the historical time step t of the target vehicle. h The trajectory data sequence is embedded and encoded, and the outputs of the embedding and position encoding are directly added to obtain the embedded sequence E, which is input to the multi-head self-attention layer. The output of the multi-head self-attention layer is passed through the residual structure and the historical time step t of the target vehicle. h After the trajectory data sequence is connected and normalized, it is input into the feedforward neural network for nonlinear transformation through the ReLU activation function, and residual connection and normalization are performed to obtain the semantic code h that represents the temporal characteristics of the target vehicle trajectory data. temp ; Step 3: Use the spatial feature extraction module SEM to extract the historical time step t h The dynamic weighted graph at each moment in is input into the graph attention neural network GAT, which outputs the hidden vector S, which is input into the gated recurrent unit GRU, and the gated recurrent unit GRU outputs the semantic code h that represents the spatial features. spa ; Step 4: Use the trajectory prediction module TPM to convert the semantic code h temp With semantic encoding h spa Fusion is performed to obtain the fusion feature h fusion , random noise R and fusion feature h fusion After concatenation, the state vector H is obtained. The gated recurrent unit GRU converts the state vector H into a semantic code as the input of the multi-layer perceptron MLP. The multi-layer perceptron MLP outputs the future t f Target vehicle trajectory within the time step 2. According to claim 1, a vehicle trajectory prediction method based on spatiotemporal fusion attention is characterized in that: In the data preprocessing module DPM, the time window is continuously moved on the trajectory data sorted by global time, and the entire trajectory data sequence of the target vehicle is divided into time lengths t h +t f The trajectory data unit, the first t of the trajectory data unit h The trajectory data sequence of time steps is used as the input trajectory data for prediction. The trajectory data unit after t f The trajectory data of the time step is used as the label, and the t of the surrounding vehicle h The trajectory data sequence of the time step is related to the target vehicle t h The time steps correspond to each moment of the trajectory data sequence.

3. The vehicle trajectory prediction method based on spatiotemporal fusion attention according to claim 1 is characterized in that: In the data preprocessing module DPM, the historical time length t is constructed h From the first moment to the tth moment h Undirected graph at time The environment vehicles and the target vehicle build edges is the node formed by the target vehicle tar at time t, is the node formed by the environment vehicle j at time t, and the edge Add to edge set E, and represents the lane number of the target vehicle tar and the environment vehicle j at time t, represents the Euclidean distance between the target vehicle tar and the environment vehicle j at time t, and the edge The corresponding weight w tar,j Based on the following formula: The weighted adjacency matrix A at time t t It is expressed as: The trajectory data corresponding to the p-th vehicle at the t-th time is taken as the feature vector p=1,···,t h , the feature vector After embedding is mapped to a high dimension, the weighted adjacency matrix A is used as the attribute value of the node of the corresponding undirected graph. t and contains the eigenvector The nodes together constitute the historical time step t h The dynamic weighted graph at each moment in .

4. The vehicle trajectory prediction method based on spatiotemporal fusion attention according to claim 3 is characterized in that: The embedded sequence 1 to t respectively h The output of the embedding code and position code corresponding to the time is added. In the multi-head self-attention layer, the embedding sequence E is respectively combined with the weight matrix of the Transformer encoder Weight Matrix Weight Matrix Multiply them together to generate Q k Matrix, K k Matrix, V k Matrix, k represents the number of the attention head, head k =Attention(Q k ,K k ,V k ) Among them, softmax() is the softmax function, Attention() is the attention head, MultiHead is the output of the multi-head attention layer, head k is the kth attention head, W O is the additional weight matrix, T is the transpose, and d represents the dimension of the trajectory data after embedding encoding.

5. The vehicle trajectory prediction method based on spatiotemporal fusion attention according to claim 4 is characterized in that: In the graph attention neural network GAT, the correlation coefficient e between the target node and its neighbor nodes in the dynamic weighted graph is calculated. tar,j Among them, e tar,j is the correlation coefficient, LeakyReLU() is the LeakyReLU activation function, It is a single-layer feedforward neural network of the graph attention neural network GAT, W and W e is the learnable parameter matrix, is the attribute value of the target node formed by the target vehicle at time t, is the attribute value of the neighboring node composed of the environment vehicles at time t, w tar,j is the weight of the edge between the target node and its neighbor node, Correlation coefficient e tar,j Normalize and get the normalized attention coefficient α tar,j ,in represents the first-order neighbor nodes of the target node, Compute splicing output t=1,2,···,t h Among them, || represents the splicing operation, represents the attention coefficient of the kth attention head, W k is the parameter matrix corresponding to the kth attention head, σ is the sigmoid activation function, M is the number of attention heads, Splicing output As 1 to t respectively h The hidden vector element at the moment, obtain the hidden vector 6. The vehicle trajectory prediction method based on spatiotemporal fusion attention according to claim 5 is characterized in that: The fusion feature h fusion Calculated based on the following formula: h fusion =z⊙h spa +(1-z)⊙h temp z=σ(W s h spa +W t h temp +b) Among them, W s , W t and b are model parameters, ⊙ is the Hadamard product, σ is the sigmoid activation function, and z is the fusion gate.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, each step of the prediction method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the prediction method according to any one of claims 1 to 6 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, each step of the prediction method according to any one of claims 1 to 6 is implemented.

Images