A vehicle trajectory prediction method based on STGNN-Net

Through the STGNN-Net method, the spatiotemporal features are extracted using TGCN-Net and STG-Net and combined with the Informer network and BiGRU model, the problems of low trajectory prediction accuracy and efficiency in the existing technology are solved, and high-precision and fast vehicle trajectory are achieved. predict.

CN118468223BActive Publication Date: 2024-11-22HUAIYIN INSTITUTE OF TECHNOLOGY
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410614595.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-17
Publication Date
2024-11-22
Estimated Expiration
2044-05-17

AI Technical Summary

Technical Problem

The existing trajectory prediction methods have shortcomings in accuracy and efficiency, making it difficult to achieve high-precision and fast vehicle trajectory prediction.

Method used

A vehicle trajectory prediction method based on STGNN-Net is proposed. By constructing feature extraction network TGCN-Net and spatiotemporal feature extraction network STG-Net, combining the Informer network and the BiGRU network with seq2seq model architecture, it integrates time and space relationships to extract spatiotemporal features And feature fusion is performed to predict vehicle trajectory.

Benefits of technology

It achieves higher accuracy and robustness, and can predict vehicle trajectories more accurately and quickly, reducing information loss in the feature extraction stage, and improving prediction efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118468223B_ABST
    Figure CN118468223B_ABST
Patent Text Reader

Abstract

The present invention discloses a vehicle trajectory prediction method based on STGNN-Net. First, the pre-acquired trajectory data set is divided and features are constructed; secondly, a vehicle trajectory prediction network based on STGNN-Net is constructed, one branch uses TGCN-Net to extract the time features of the input trajectory data, and the other branch uses STG-Net to obtain the spatiotemporal features of the trajectory, so that the model can consider the influence of different time and space ranges, and then the residual network is used to fuse the features of TGCN-Net and STG-Net respectively; finally, adaptive learning is used to fuse the Informer features and then sent to the BiGRU network to predict the vehicle's trajectory. The present invention makes full use of the time features in the trajectory data, combines the spatial features of different times, extracts features through TGCN-Net and STG-Net to reduce the loss of information, and realizes more accurate and robust trajectory prediction; through the graph neural network and attention mechanism, the network can consider the interactive information, thereby optimizing the features and achieving higher trajectory prediction accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of trajectory prediction, and in particular relates to a vehicle trajectory prediction method based on STGNN-Net. Background Art

[0002] At present, the rapid development of the field of autonomous driving has gradually attracted widespread attention to trajectory prediction. As the core module of autonomous driving, vehicle trajectory prediction is generally located at the back end of the perception module. It predicts the intention of the vehicle and obtains trajectory prediction for a period of time in the future through comprehensive consideration of road information, traffic information, environmental information, etc. perceived by the perception module. It is a connecting module in autonomous driving and has become a hot topic of research in recent years. It is also a very challenging topic at present. Vehicle trajectory prediction refers to the computer analyzing the historical trajectory data of the vehicle and using various deep learning algorithms and models to predict the possible driving trajectory of the vehicle in the future. According to the prediction of the target vehicle, the vehicle can be warned, braked, and route planned according to the predicted trajectory range to avoid traffic accidents such as collisions and illegal lane changes due to unhurried reactions.

[0003] In the task of vehicle trajectory prediction, high-precision maps are mainly used to abstract the characteristics of vehicles and the environment. The mainstream representation methods are rasterization and vectorization. Rasterization represents the characteristics of the map in a two-dimensional matrix, mainly abstracting the spatial characteristics of the high-precision map. A unit in each matrix is ​​called a grid cell (Cell), or a pixel (Pixel). This type of representation method is characterized by strong expressiveness and can abstract the overall characteristics, but is limited by the number of pixels. When the image is enlarged or stretched, the original clarity of the image will change. Vectorization uses points, lines, and surfaces to record the shape of geometric objects in the image. By representing the image by vectorization, the amount of data storage can be reduced. It is more suitable for recording some regular images, such as lane lines, sidewalks, parking areas, etc. The vectorized map representation can ensure that no matter how it is scaled or stretched, its clarity will not change. At the same time, after vectorization, it can be calculated quickly to improve the running speed.

[0004] With the rapid development of computers, the algorithms for predicting trajectories have gradually transitioned from machine learning to deep learning. Currently, deep learning methods mainly include Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM), Graph Convolution Network (GCN), etc. It is necessary to choose the appropriate method for trajectory prediction according to the specific application scenario and data characteristics.

[0005] Convolutional neural network is a variant of neural network, which usually extracts features of topological data (such as images, audio, etc.). Compared with ordinary networks, it mainly uses convolution operations instead of full connection operations, and parameters can be shared, which greatly reduces the number of network products and the complexity of network models. Generally, in trajectory prediction tasks based on rasterization, CNN networks can be used to extract environmental features around moving vehicles from map data to help predict future trajectories.

[0006] Recurrent neural networks have powerful information processing capabilities and deep feature representation capabilities in processing time series tasks. Therefore, some scholars use recurrent neural networks to extract trajectory features in trajectory prediction tasks. RNN stores historical information of the current time step, and its output is determined by both hidden storage units and inputs. The parameters of the model can be shared, and the number of parameters will not increase with time. However, when the time is too long, its gradient will have "gradient vanishing" and "gradient exploding" problems in back propagation. LSTM is often used to solve this problem in trajectory prediction tasks.

[0007] In trajectory prediction research, graph neural networks represent traffic participants in driving scenarios as nodes, and the relationship between traffic participants as an edge. All points and edges form a graph. When extracting features through graph structures, graph convolutional neural networks extend traditional CNNs to graph data convolution operations, and can obtain data feature information by mapping node features in the graph through the adjacency matrix. Considering the different values ​​of different nodes, the GCN network was subsequently improved, and the Graph Attention Neural Network (GAT) was proposed. It introduces an attention mechanism in the feature extraction process of the graph structure, assigns different weights to the feature information transmitted by different neighbor nodes, and these weights can be continuously updated through learning.

[0008] Compared with the RNN and CNN methods in existing research, graph neural networks can extract non-Euclidean spatial data features. In complex driving scenarios, each scene can be regarded as an irregular graph, and the nodes of each graph are represented by the motion state characteristics of traffic participants or static environmental information. There are interconnected edges between the nodes of each graph, which can be used to represent the interdependence of traffic participants in the scene. Therefore, graph neural networks are suitable for solving interaction-based vehicle trajectory prediction problems. Summary of the invention

[0009] Purpose of the invention: In view of the low accuracy and slow efficiency of current trajectory prediction for vehicle trajectory prediction, a vehicle trajectory prediction method based on STGNN-Net is invented, which integrates time and space relationships to achieve accurate prediction of vehicle trajectory with high accuracy and strong robustness.

[0010] Technical solution: The present invention proposes a vehicle trajectory prediction method based on STGNN-Net, comprising the following steps:

[0011] (1) Divide and construct features of the pre-acquired trajectory dataset;

[0012] (2) Construct a vehicle trajectory prediction network based on STGNN-Net, including a feature extraction network and a prediction network; the feature extraction network includes a temporal feature extraction network TGCN-Net and a spatiotemporal feature extraction network STG-Net; TGCN-Net uses a dual-branch structure to focus on global and local temporal features, and fuses features to extract temporal features; STG-Net first uses the GAT of the first branch to enable the network to consider interactive information, then downsamples the nodes after interaction so that the network can extract node features in a direction, and finally expands the dimension to keep the output feature size unchanged; the second branch is used to extract temporal features; the features of the two branches are combined to achieve the fusion of spatiotemporal features;

[0013] (3) The temporal and spatiotemporal features in the trajectory are extracted through TGCN-Net and STG-Net, the features are fused through the Informer network through adaptive learning, and the motion trajectory of the target vehicle is predicted through the BiGRU network of the seq2seq model architecture.

[0014] Furthermore, the implementation process of step (1) is as follows: every twelve frames of data are constructed into a group, and the first six frames of data in a group of data are used as training samples, and the last six frames of data are used as true label values.

[0015] Furthermore, in step (2), the TGCN-Net includes a Batch Normalization layer and two TE-Nets; the TE-Net includes a time domain module TM and a graph convolutional network GCN; the TM is composed of a gated recurrent unit GRU and a time domain convolutional network TCN; the TGCN-Net input is only the feature of the node in the last time frame, and the feature data first exchanges the information of the neighboring nodes through the GCN, and then extracts the time information of the node through the TM; when extracting the time feature of the last time frame, the GRU is used to focus on the global time information, and the TCN is used to focus on the local features in the sequence; after feature fusion, the TE-Net can perceive the global and local time features.

[0016] Furthermore, the TM divides the TE-Net backbone network into two branches, and the specific implementation process is as follows:

[0017] First, the data of R0 is divided into the data of the last time frame R1 and the data of the first six frames R2, and then R1 is sent to TGCN-Net, first input to the Batch Normalization layer, and then to the first TE-Net; TE-Net first passes through a graph convolution layer with a step size of 1, a convolution kernel size of 1×1, an input channel of 4, and an output channel of 192, and then performs a matrix multiplication operation with the adjacency matrix to obtain the output R11 of the GCN network;

[0018] The way to update the GCN network node features is as follows:

[0019]

[0020] Among them, A and I N are the adjacency matrix and the N-dimensional identity matrix, respectively. It is the adjacency matrix of its own node connected to it:

[0021]

[0022] in, yes The degree matrix of The matrix is ​​normalized;

[0023] Construct a two-layer GCN with ReLU and softmax as activation functions respectively. The overall forward propagation formula is:

[0024]

[0025] Among them, X is the input node feature, W 0 Represents the weight matrix of the first layer of GCN, W 1 represents the weight matrix of the second layer GCN, and Z is the updated node feature;

[0026] The first TE-Net contains a residual structure, which sends R1 to a convolution layer with a stride of 1, a convolution kernel size of 1×1, 4 input channels, and 64 output channels, and then passes through a BatchNormalization layer to obtain R12; the feature fusion output R13 is obtained by performing matrix and operation on R11 and R12;

[0027] TM divides the main network of TE-Net into two branches. The first branch takes R13 as input, first passes through a Batch Normalization layer and a ReLU activation layer, and then sends it to two convolutional layers with a step size of 1, padding of (2, 0), a convolution kernel size of 5×1, an input channel of 64, and an output channel of 64. Finally, it passes through a Batch Normalization layer and a Dropout operation to obtain R131; the second branch inputs R13 into a 2-layer GRU network with an input feature dimension of 64 and a hidden state dimension of 60, and then passes through a fully connected layer to obtain R132; R131 and R132 are fused to obtain the output R14 of TM, which is the output of the first TE-Net;

[0028] The second TE-Net first sends R14 to the GCN network consistent with the first TE-Net to obtain R15, and then inputs it into TM to obtain R16, which is the output of TGCN-Net.

[0029] Furthermore, in step (2), the STG-Net includes two fully connected networks and two STE-Nets; the STE-Net is a dual-branch network composed of a graph attention network GAT, a spatial convolutional network SCN, a graph convolutional network GCN, and a bidirectional gated recurrent unit BiGRU; the data is sent to the STE-Net, and the data dimension is first expanded using the fully connected network, and then the features are extracted; the input is all the data of the previous six time frames; the first branch takes all the node features of the previous six time frames as input, constructs an adjacency matrix, extracts spatial features through the GAT network, and finally uses the SCN to adjust the number of nodes in combination with the data characteristics, fuses the features, and keeps the output feature size unchanged; the second branch constructs six adjacency matrices, each of which represents the connectivity relationship of the nodes in a time frame, uses the GCN network to exchange the node information of each time frame, and extracts the time features of each node in the previous six time frames through the BiGRU network.

[0030] Furthermore, the STG-Net in step (3) extracts spatiotemporal features as follows:

[0031] Take R2 as input, pass through a fully connected layer with an input dimension of 4 and an output dimension of 16, and then perform ReLU activation before sending it to a fully connected layer with an input dimension of 16 and an output dimension of 32, and then send it to the first branch of STE-Net. First, pass through a GAT network to get the network output R21. The specific parameters are n_head is 4, the input feature dimension is 32, and the output feature dimension is set to 16;

[0032] The way to update features using the GAT network is as follows:

[0033] e ij =a[wh i ||wh j ],j∈N i (4)

[0034] Among them, e ij is to calculate the attention coefficient between two nodes, h i and h j are the features of the i-th and j-th nodes respectively, N i is the set of nodes around node i, w is a shared parameter that can linearly change the node features, [.||.] represents the concatenation of the results of the changes of nodes i and j; a is the mapping of the concatenated high-dimensional features to a real number:

[0035]

[0036] Among them, LeakyReLU is an activation function, exp is an exponential function, first of all, the attention coefficient e ij Normalize to get a ij , after weighted summation and softmax, we get the updated feature h′ of the i-th node i ;

[0037] After passing through the GAT network, R21 is used as input and sent to the SCN. The SCN contains two convolutional layers. The first convolutional layer has a step size of 1×4, a convolution kernel size of 5×1, 64 input channels, and 64 output channels. The second convolutional layer has a step size of 1, a convolution kernel size of 1×1, 30 input channels, and 120 output channels. The number of objects in each frame is downsampled by the first convolution, and the second convolution is used to increase the dimension to obtain the output R22 of the SCN.

[0038] Another branch of STE-Net is the temporal feature extraction module. After R2 is used as input and dimensioned through the fully connected layer, it first passes through a GCN network with a step size of 1, a convolution kernel size of 1×1, an input channel of 32, and an output channel of 64, and then passes through a bidirectional gated recurrent unit to obtain R23. This network is a 2-layer BiGRU with an input feature dimension of 64 and a hidden state dimension of 60. The output R24 of the first STE-Net is obtained by performing matrix and operations on R22 and R23.

[0039] The second STE-Net first sends R24 to the first branch, which consists of a GAT network and the same SCN as the first STE-Net. R25 is obtained through GAT and R26 is obtained through SCN, where the n_head of the GAT network is 1, the input feature dimension is 64, and the output feature dimension is 32; the other branch takes R24 as input and sends it to a GCN network with a stride of 1, a convolution kernel size of 1×1, 64 input channels, and 64 output channels, and then passes through a 2-layer bidirectional gated recurrent unit network with 64 features and 60 hidden state dimensions to obtain R27; the output R28 of the second STE-Net is obtained by performing matrix and operations on R26 and R27.

[0040] Furthermore, the process of implementing feature fusion by using the Informer network through adaptive learning in step (3) is as follows:

[0041] The Informer network has four input features, namely T1, T2, T3, and T4; T1 is equal to the output R14 of the first TE-Net multiplied by the parameter Ra; T2 is equal to the output R16 of the second TE-Net multiplied by the parameter Rb; T3 is equal to the output R24 of the first STE-Net multiplied by the parameter Rc; T4 is equal to the output R28 of the second STE-Net multiplied by the parameter Rd; T1, T2, T3, and T4 are matrixed and operated to obtain feature T, focusing on multi-faceted features.

[0042] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: the STGNN-Net model proposed in the present invention extracts spatiotemporal features by two parallel branches, one of which only uses the information of the last time frame to extract temporal features, and the other branch uses the information of the first six time frames to extract spatiotemporal information, making full use of the features of different time and space ranges to achieve more accurate and robust trajectory prediction; TGCN-Net uses a dual-branch structure to focus on global and local temporal features, and fuses features to extract temporal features; in STG-Net, GAT is first used to enable the network to consider interaction information, and then the nodes after interaction are downsampled to 30, so that the network can extract node features in a direction, and finally the dimension is expanded to keep the output feature size unchanged; the features of TGCN-Net and STG-Net are fused respectively through the residual network, and the backbone network uses the Informer network to fuse features of different depths after adaptive learning to reduce the loss of underlying information in the feature extraction stage and achieve rapid convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the vehicle trajectory prediction network structure based on STGNN-Net constructed by the present invention;

[0044] Figure 2 This is a schematic diagram of the TE-Net structure proposed in the present invention;

[0045] Figure 3 This is a schematic diagram of the TM structure proposed in the present invention;

[0046] Figure 4 This is a schematic diagram of the STE-Net structure proposed in the present invention; DETAILED DESCRIPTION

[0047] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0048] This embodiment involves a large number of variables, and each variable is described as follows, as shown in Table 1.

[0049] Table 1 Variable description table

[0050]

[0051]

[0052] The present invention proposes a vehicle trajectory prediction method based on STGNN-Net (Spatiotemporal and Temporal Graph Neural Network, spatiotemporal fusion graph neural network). The method extracts the time features of data within the last time frame through a temporal graph convolutional neural network (Temporal Graph Convolutional Network, TGCN-Net), and uses a spatial and temporal graph neural network (Spatial and Temporal Graph Neural Network, STG-Net) to extract the spatiotemporal features in the vehicle trajectory data of the first six time frames; then the temporal and spatial features are sent to an informer network for feature fusion through adaptive learning, and finally a Bidirectional Gated Recurrent Unit (Bidirectional Gated Recurrent Unit) network of a seq2seq model architecture is used to predict the motion trajectory of the target vehicle.

[0053] like Figure 1 As shown, a vehicle trajectory prediction network based on STGNN-Net is constructed, including a feature extraction network and a prediction module; wherein the feature extraction network includes TGCN-Net and STG-Net.

[0054] TGCN-Net stacks two TE-Nets (Temporal Extraction Networks). The TM (Temporal Module) in TE-Net is composed of GRU (Gated Recurrent Unit) and TCN (Temporal Convolutional Network). TGCN-Net inputs only the features of the nodes in the last time frame. The feature data first exchanges the information of neighboring nodes through GCN (Graph Convolutional Network), and then extracts the time information of the nodes through TM. When extracting the time features of the last time frame, GRU is used to focus on the global time information, and TCN is used to focus on the local features in the sequence. After feature fusion, TE-Net can perceive the global and local time features.

[0055] TM divides the TE-Net backbone network into two branches. The implementation process is as follows:

[0056] The first TE-Net is as follows Figure 2 As shown in the figure, the data of R0 is first divided into the data R1 of the last time frame and the data R2 of the first six frames, and then R1 is sent to TGCN-Net, first input into the BatchNormalization layer, and then sent to the first TE-Net. TE-Net first passes through a graph convolution layer with a step size of 1, a convolution kernel size of 1×1, an input channel of 4, and an output channel of 192. Then, a matrix multiplication operation is performed with the adjacency matrix to obtain the output R11 of the GCN network.

[0057] The way to update the GCN network node features is as follows:

[0058]

[0059] Among them, A and I N are the adjacency matrix and the N-dimensional identity matrix, respectively. It is the adjacency matrix of its own nodes connected to it.

[0060]

[0061] in, yes The degree matrix of Normalize the matrix.

[0062] Here we construct a two-layer GCN, with ReLU and softmax as activation functions. The overall forward propagation formula is:

[0063]

[0064] Among them, X is the input node feature, W 0 Represents the weight matrix of the first layer of GCN, W 1 represents the weight matrix of the second layer GCN, and Z is the updated node feature.

[0065] The first TE-Net contains a residual structure, which feeds R1 into a convolution layer with a stride of 1, a convolution kernel size of 1×1, 4 input channels, and 64 output channels, and then passes through the BatchNormalization layer to obtain R12. The feature fusion output R13 is obtained by performing matrix and operation on R11 and R12.

[0066] like Figure 3 As shown in the figure, TM divides the backbone network of TE-Net into two branches. The first branch takes R13 as input, first passes through a Batch Normalization layer and a ReLU activation layer, then enters two convolutional layers with a step size of 1, padding of 0, a convolution kernel size of 5×1, 64 input channels, and 64 output channels, and finally passes through a BatchNormalization layer and a Dropout operation to obtain R131. The second branch inputs R13 into a 2-layer GRU network with an input feature dimension of 64 and a hidden state dimension of 60, and then passes through a fully connected layer to obtain R132. The output R14 of TM is obtained by fusing R131 and R132, which is the output of the first TE-Net.

[0067] The second TE-Net first sends R14 to the GCN network consistent with the first TE-Net to obtain R15, and then inputs it into TM to obtain R16, which is the output of TGCN-Net.

[0068] like Figure 1 As shown in the figure, STG-Net includes two fully connected networks and two STE-Nets (Spatial and Temporal Extraction Network). STE-Net is a dual-branch network composed of GAT (Graph Attention Network), SCN (Spatial Convolutional Network), GCN (Graph Convolutional Network), and BiGRU (Bidirectional Gated Recurrent Unit).

[0069] After being sent to STE-Net, the data dimension is first expanded using a fully connected network, and then features are extracted; the input is all the data of the first six time frames; the first branch takes all the node features of the first six time frames as input, constructs an adjacency matrix, extracts spatial features through the GAT network, and finally uses SCN to adjust the number of nodes and fuse features based on the data characteristics, while keeping the output feature size unchanged; the second branch constructs six adjacency matrices, each of which represents the connectivity of nodes in a time frame, uses the GCN network to interact with the node information of each time frame, and extracts the time features of each node in the first six time frames through the BiGRU network. The implementation process of STG-Net extracting underlying spatial features is as follows:

[0070] Take R2 as input, pass through a fully connected layer with an input dimension of 4 and an output dimension of 16, and then perform ReLU activation before sending it to a fully connected layer with an input dimension of 16 and an output dimension of 32, and then send it to the first branch of STE-Net. First, pass through a GAT network to get the network output R21. The specific parameters are n_head is 4, the input feature dimension is 32, and the output feature dimension is set to 16;

[0071] The way to update features using the GAT network is as follows:

[0072] e ij =a[wh i ||wh j ],j∈N i (4)

[0073] Among them, e ij is to calculate the attention coefficient between two nodes, h i and h j are the features of the i-th and j-th nodes respectively, N i is the set of nodes around node i, w is a shared parameter that can linearly change the node features, [.||.] represents the concatenation of the results of the changes of nodes i and j. a is the mapping of the concatenated high-dimensional features to a real number.

[0074]

[0075] In the period, LeakyReLU is an activation function, exp is an exponential function, first of all, the attention coefficient e ij Normalize to get a ij , after weighted summation and softmax, we get the updated feature h′ of the i-th node i .

[0076] like Figure 4As shown in the figure, after passing through the GAT network, R21 is used as input and sent to SCN. From the observation data, it can be seen that the number of objects in each frame is mostly less than 30, so SCN is used to reduce the number of objects when constructing data. The number of nodes is mainly reduced through the first layer of convolution, and the main node features are extracted. After exchanging dimensions, the node features are fused through the second layer of convolution to ensure the consistency of data dimensions. The specific network is as follows:

[0077] SCN mainly consists of two convolutional layers. The first convolutional layer has a step size of 1×4, a convolution kernel size of 5×1, 64 input channels, and 64 output channels. The second convolutional layer has a step size of 1, a convolution kernel size of 1×1, 30 input channels, and 120 output channels. The first convolution downsamples the number of objects in each frame, and the second convolution is used to increase the dimension to obtain the output R22 of SCN.

[0078] Another branch of STE-Net is the temporal feature extraction module. After R2 is used as input and upgraded through the fully connected layer, it first passes through a GCN network with a step size of 1, a convolution kernel size of 1×1, an input channel of 32, and an output channel of 64, and then passes through a 2-layer BiGRU network with an input feature dimension of 64 and a hidden state dimension of 60 to obtain R23; this module can be used to focus on the bidirectional temporal feature dependency. The output R24 of the first STE-Net is obtained by performing matrix and operations on R22 and R23.

[0079] like Figure 1 As shown, the second STE-Net first sends R24 to the spatial feature extraction branch, which consists of a GAT network and the same SCN as the first STE-Net. R25 is obtained through the GAT network, and R26 is obtained through the SCN. The GAT network n_head is 1, the input feature dimension is 64, and the output feature dimension is 32. The second branch takes R24 as input and sends it to a GCN network with a stride of 1, a convolution kernel size of 1×1, 64 input channels, and 64 output channels. R27 is obtained through a 2-layer BiGRU network with 64 features and a hidden state dimension of 60. The output R28 of the second STE-Net is obtained by performing matrix and operations on R26 and R27.

[0080] The outputs of TGCN-Net and STG-Net are further fused, and the process of implementing Informer feature fusion using adaptive learning is as follows:

[0081] The Informer network has four input features, namely T1, T2, T3, and T4. T1 is equal to the output R14 of the first TE-Net multiplied by the parameter Ra, T2 is equal to the output R16 of the second TE-Net multiplied by the parameter Rb, T3 is equal to the output R24 of the first STE-Net multiplied by the parameter Rc, and T4 is equal to the output R28 of the second STE-Net multiplied by the parameter Rd. T1, T2, T3, and T4 are matrixed and operated to obtain feature T, focusing on more underlying features. Then, the extracted trajectory feature T is sent to the BiGRU network of the seq2seq model architecture to predict the motion trajectory of the target vehicle.

[0082] In this embodiment, the ApolloScape trajectory dataset is first obtained from the official website, and divided and feature constructed. Specifically, every twelve frames of data are constructed into a group. In a group of data, the first six frames of data are used as training samples, and the last six frames of data are used as true label values. Finally, the data is divided into 8:2. The data storage folder is Appollo / datasets, which includes two subfolders, prediction_test and prediction_train, where prediction_test saves a test set file, and prediction_train stores a batch of training set data files. The vehicle trajectory dataset is input into STGNN-Net for training, with an initial learning rate of 0.001, an optimizer using Adam, batch_size of 32, and epochs of 300. Model training is started to obtain the trained STGNN-Net weight model, which is finally verified on the test dataset.

[0083] The above description is only an example of the present invention and is not intended to limit the present invention. Any equivalent substitution made within the principle of the present invention should be included in the protection scope of the present invention. The content not elaborated in detail in the present invention belongs to the existing technology known to the technicians in this professional field.

Claims

1. A vehicle trajectory prediction method based on STGNN-Net, It is characterized in that The following steps are involved: (1) Divide and construct features of the pre-acquired trajectory dataset; (2) Construct a vehicle trajectory prediction network based on STGNN-Net, including a feature extraction network and a prediction network; the feature extraction network includes a temporal feature extraction network TGCN-Net and a spatiotemporal feature extraction network STG-Net; TGCN-Net uses a dual-branch structure to focus on global and local temporal features, and fuses features to extract temporal features; STG-Net first uses the GAT of the first branch to enable the network to consider interactive information, then downsamples the nodes after interaction so that the network can extract node features in a direction, and finally expands the dimension to keep the output feature size unchanged; the second branch is used to extract temporal features; Combining the features of the two branches to achieve the fusion of spatiotemporal features; (3) The temporal and spatial features in the trajectory are extracted through TGCN-Net and STG-Net, and the features are fused through the Informer network through adaptive learning, and the motion trajectory of the target vehicle is predicted through the BiGRU network of the seq2seq model architecture; Step (2) The TGCN-Net includes a BatchNormalization layer and two TE-Nets; the TE-Net includes a time domain module TM and a graph convolutional network GCN; the TM is composed of a gated recurrent unit GRU and a time domain convolutional network TCN; the TGCN-Net input is only the feature of the node in the last time frame, and the feature data first exchanges the information of the neighboring nodes through the GCN, and then extracts the time information of the node through the TM; when extracting the time feature of the last time frame, the GRU is used to focus on the global time information, and the TCN is used to focus on the local features in the sequence; after feature fusion, the TE-Net can perceive the global and local time features.

2. A vehicle trajectory prediction method based on STGNN-Net according to claim 1, It is characterized in that The implementation process of step (1) is as follows: every twelve frames of data are constructed into a group, and the first six frames of data in a group of data are used as training samples, and the last six frames of data are used as true label values.

3. A vehicle trajectory prediction method based on STGNN-Net according to claim 1, It is characterized in that The TM divides the TE-Net backbone network into two branches. The specific implementation process is as follows: First, the data of R0 is divided into the data of the last time frame R1 and the data of the first six frames R2, and then R1 is sent to TGCN-Net, first input to the Batch Normalization layer, and then to the first TE-Net; TE-Net first passes through a graph convolution layer with a step size of 1, a convolution kernel size of 1×1, an input channel of 4, and an output channel of 192, and then performs a matrix multiplication operation with the adjacency matrix to obtain the output R11 of the GCN network; The way to update the GCN network node features is as follows: Among them, A and I N are the adjacency matrix and the N-dimensional identity matrix, respectively. It is the adjacency matrix of its own node connected to it: in, yes The degree matrix of The matrix is ​​normalized; Construct a two-layer GCN with ReLU and softmax as activation functions respectively. The overall forward propagation formula is: Among them, X is the input node feature, W 0 Represents the weight matrix of the first layer of GCN, W 1 represents the weight matrix of the second layer GCN, and Z is the updated node feature; The first TE-Net includes a residual structure, which feeds R1 into a convolution layer with a stride of 1, a convolution kernel size of 1×1, 4 input channels, and 64 output channels, and then passes through a Batch Normalization layer to obtain R12; the feature fusion output R13 is obtained by performing matrix and operation on R11 and R12; TM divides the backbone network of TE-Net into two branches. The first branch takes R13 as input, first passes through a BatchNormalization layer and a ReLU activation layer, and then is sent to two convolutional layers with a step size of 1, padding of (2, 0), a convolution kernel size of 5×1, an input channel of 64, and an output channel of 64. Finally, it passes through a Batch Normalization layer and a Dropout operation to obtain R131. The second branch inputs R13 into a 2-layer GRU network with an input feature dimension of 64 and a hidden state dimension of 60, and then passes through a fully connected layer to obtain R132. The output R14 of TM is obtained by fusing R131 and R132, which is the output of the first TE-Net. The second TE-Net first sends R14 to the GCN network consistent with the first TE-Net to obtain R15, and then inputs it into TM to obtain R16, which is the output of TGCN-Net.

4. A vehicle trajectory prediction method based on STGNN-Net according to claim 1, It is characterized in that Step (2) The STG-Net includes two fully connected networks and two STE-Nets; the STE-Net is a dual-branch network consisting of a graph attention network GAT, a spatial convolutional network SCN, a graph convolutional network GCN, and a bidirectional gated recurrent unit BiGRU; sent to the STE-Net, the data dimension is first expanded using the fully connected network, and then the features are extracted; The input is all the data of the first six time frames; the first branch takes all the node features of the first six time frames as input, constructs an adjacency matrix, extracts spatial features through the GAT network, and finally uses SCN to adjust the number of nodes and fuse features based on the data characteristics, while keeping the output feature size unchanged; The second branch constructs six adjacency matrices, each of which represents the connectivity of nodes within a time frame. The GCN network is used to exchange node information for each time frame, and the BiGRU network is used to extract the temporal features of each node in the previous six time frames.

5. A vehicle trajectory prediction method based on STGNN-Net according to claim 1, It is characterized in that The STG-Net in step (3) extracts temporal and spatial features from the trajectory to achieve the following process: Take R2 as input, pass through a fully connected layer with an input dimension of 4 and an output dimension of 16, and then perform ReLU activation before sending it to a fully connected layer with an input dimension of 16 and an output dimension of 32, and then send it to the first branch of STE-Net. First, pass through a GAT network to get the network output R21. The specific parameters are n_head is 4, the input feature dimension is 32, and the output feature dimension is set to 16; The way to update features using the GAT network is as follows: e ij =a[wh i ||wh j ],j∈N i (4) Among them, e ij is to calculate the attention coefficient between two nodes, h i and h j are the features of the i-th and j-th nodes respectively, N i is the set of nodes around node i, w is a shared parameter that can linearly change the node features, [.||.] represents the concatenation of the results of the changes of nodes i and j; a is the mapping of the concatenated high-dimensional features to a real number: Among them, LeakyReLU is an activation function, exp is an exponential function, first of all, the attention coefficient e ij Normalize to get a ij , after weighted summation and normalized exponential function, the updated feature h' of the i-th node is obtained i ; After passing through the GAT network, R21 is used as input and sent to the SCN. The SCN contains two convolutional layers. The first convolutional layer has a step size of 1×4, a convolution kernel size of 5×1, 64 input channels, and 64 output channels. The second convolutional layer has a step size of 1, a convolution kernel size of 1×1, 30 input channels, and 120 output channels. The number of objects in each frame is downsampled by the first convolution, and the second convolution is used to increase the dimension to obtain the output R22 of the SCN. Another branch of STE-Net is the temporal feature extraction module. After R2 is used as input and upgraded through the fully connected layer, it first passes through a GCN network with a step size of 1, a convolution kernel size of 1×1, an input channel of 32, and an output channel of 64, and then passes through a 2-layer BiGRU with an input feature dimension of 64 and a hidden state dimension of 60 to obtain R23; the output R24 of the first STE-Net is obtained by performing matrix and operations on R22 and R23; The second STE-Net first sends R24 to the first branch, which consists of a GAT network and the same SCN as the first STE-Net. R25 is obtained through GAT and R26 is obtained through SCN, where the n_head of the GAT network is 1, the input feature dimension is 64, and the output feature dimension is 32; the other branch takes R24 as input and sends it to a GCN network with a stride of 1, a convolution kernel size of 1×1, 64 input channels, and 64 output channels, and then passes through a 2-layer bidirectional gated recurrent unit network with 64 features and 60 hidden state dimensions to obtain R27; the output R28 of the second STE-Net is obtained by performing matrix and operations on R26 and R27.

6. A vehicle trajectory prediction method based on STGNN-Net according to claim 1, It is characterized in that The process of implementing feature fusion by using the Informer network through adaptive learning in step (3) is as follows: The Informer network has four input features, namely T1, T2, T3, and T4; T1 is equal to the output R14 of the first TE-Net multiplied by the parameter Ra; T2 is equal to the output R16 of the second TE-Net multiplied by the parameter Rb; T3 is equal to the output R24 of the first STE-Net multiplied by the parameter Rc; T4 is equal to the output R28 of the second STE-Net multiplied by the parameter Rd; T1, T2, T3, and T4 are matrixed and operated to obtain feature T, focusing on multiple aspects of features.

Citation Information

Cited By

  • Individual risk dynamic assessment method and system based on space-time causal graph network

    CN120746796A

  • Individual risk dynamic assessment method and system based on spatiotemporal causal diagram network

    CN120746796B