A vehicle trajectory prediction method based on multi-vehicle space-time interaction relationship

Abstract

The application discloses a vehicle trajectory prediction method based on multi-vehicle space-time interaction relationship. First, according to the highway driving data set, the speed relationship graph, the acceleration relationship graph and the driving intention relationship graph of the target vehicle and the surrounding vehicles are generated, and then a new environment feature graph aggregating neighbor features is generated through SuperGATConv fusion; secondly, the new environment feature graph is input into an environment intention encoder constructed by a dilated convolution network to generate a spatial coding feature; the vehicle data information of the target vehicle to be predicted is input into an LSTM as an input feature to obtain a time coding feature. Then, the spatial coding feature and the time coding feature are spliced and input into a space-time decoder to obtain a future state feature. Finally, the future state feature is input into a prediction module constructed by an LSTM and an MPL to generate a predicted future trajectory. The application obtains rich interaction information between vehicles, ensures that local spatial information is not lost, and improves the prediction performance of the vehicle trajectory.

Description

Technical Field

[0001] This invention belongs to the field of trajectory prediction, specifically relating to a vehicle trajectory prediction method based on the spatiotemporal interaction relationship of multiple vehicles. Background Technology

[0002] Extensive research has been conducted in the field of trajectory prediction, resulting in numerous methods broadly categorized into three types: physics-based methods, statistical methods, and deep learning-based methods. Physics-based methods primarily utilize dynamic and kinematic models, leveraging physical and mechanical principles to predict trajectories based on the vehicle's current state. Statistical methods employ predefined maneuver distributions to describe predicted trajectories, such as Hidden Markov Models, Dynamic Bayesian Networks, and Support Vector Machines. These methods often provide more refined and complex model structures, resulting in better predictive performance compared to physics-based methods. Deep learning-based methods mainly include Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and Transformers. RNNs, such as LSTMs, are commonly used to process time-series trajectory data. CNNs excel at extracting spatial features from inputs such as bird's-eye views or raster images. Transformers have achieved significant success in many fields and have also demonstrated superior performance in trajectory prediction, particularly for tasks requiring long-term predictions.

[0003] While physics-based methods offer interpretability and efficiency, they typically exhibit lower prediction accuracy compared to state-of-the-art (SOTA) techniques. Statistical methods, though generally performing better than physics-based methods, often fall short of deep learning-based methods. Among deep learning methods, CNNs often struggle with temporal tasks, RNNs cannot explore spatial relationships between vehicles, and Transformers suffer from large model size, numerous parameters, and long training times compared to other models. In this context, introducing graph attention networks and multi-layer dilated convolutions can improve spatial feature extraction, and combining them with Transformer variants can further enhance predictive performance. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a vehicle trajectory prediction method based on the spatiotemporal interaction relationships of multiple vehicles on highways. The method aims to significantly improve the extraction quality of multi-vehicle interaction information through graph attention and multi-layer dilated convolution. Furthermore, it combines a Transformer variant model to enhance prediction capabilities. The specific design follows these steps:

[0005] Step 1: Obtain the target vehicle to be predicted, the vehicle data information of the target vehicle, and the driving data of surrounding vehicles from the highway driving dataset (open source public dataset). The driving data includes the lane information, position coordinate information, speed and acceleration information of all vehicles in the same time period. Further process the driving data to obtain the speed relationship graph, acceleration relationship graph, and driving intention relationship graph of the target vehicle and surrounding vehicles.

[0006] The velocity relationship map, acceleration relationship map, and driving intention relationship map are input into the graph attention network SuperGATConv to generate a new environmental feature map that aggregates the features of neighbors. As a variant of the graph attention network, SuperGATConv can better focus on the local interaction feature information between surrounding vehicles and the target vehicle.

[0007] Step 2: Input the new environmental feature map into the environmental intent encoder to generate spatial encoded features. The environmental intent encoder uses a multi-layer dilated convolutional network.

[0008] Step 3: Use the vehicle data information of the target vehicle to be predicted as input features. The input features include: historical trajectory, speed information, acceleration information, lane category information, and vehicle category information. Input these five types of information into LSTM to obtain time-coded features.

[0009] Step 4: Concatenate the spatial coding features and temporal coding features, and then input them into the spatiotemporal decoder to obtain the future state features. The spatiotemporal decoder uses PatchTST, which is a Transformer-based temporal prediction model.

[0010] Step 5: Based on the future state features, generate the predicted future trajectory through the prediction module built by LSTM and MLP, and perform reverse training.

[0011] The prediction module specifically involves inputting the future state features output by the spatiotemporal decoder into a network composed of LSTM and the activation function ELU. The resulting outputs are then fed into two parallel MLP networks to generate lateral driving intentions and longitudinal driving intentions.

[0012] Finally, the lateral driving intention, longitudinal driving intention, and future state features are concatenated and then input into a decoder composed of LSTM and MLP in sequence to generate the predicted future trajectory.

[0013] Compared with the prior art, the advantages of the present invention are as follows:

[0014] For three different information relationship graphs, SuperGATConv can better aggregate neighbor information. SuperGATConv is a GAT model that uses attention weights compared to the traditional GCN, meaning that the relationship between two nodes can be optimized into continuous values, and rich interaction information between vehicles can be obtained.

[0015] For graph information aggregated by graph networks, the environment intent encoder based on dilated convolution can ensure that the receptive field of the convolution kernel is expanded while keeping the number of parameters unchanged, so that each convolution output contains a larger range of information and captures richer driving intentions of nearby vehicles. At the same time, combining convolutional layers with dilated convolutional layers ensures that local spatial information is not lost.

[0016] A PatchTST-based spatiotemporal decoder is used to split the feature sequence into short sequences of the same size, learn and extract local features of the short sequences, alleviate the problem of information loss in long sequences, and compared with the traditional Transformer model, it has fewer model parameters and can generate more effective future state information, thus improving the prediction performance of vehicle trajectory. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the method described in this invention;

[0019] Figure 2 This is a schematic diagram of the environmental encoder described in this invention;

[0020] Figure 3 This is a schematic diagram of the PatchTST model algorithm used in this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] A vehicle trajectory prediction method based on the spatiotemporal interaction relationship of multiple vehicles, the prediction model framework is as follows: Figure 1 As shown, it includes the following steps:

[0023] Step 1: The driving data of the predicted vehicle and surrounding vehicles can be obtained from the NGSIM dataset. The coordinate information of the predicted vehicle and surrounding vehicles is put into a 3x13 bird's-eye view G. This bird's-eye view is specifically represented as the predicted vehicle as the center, containing the driving data of three lanes and up to 39 vehicles driving in the lanes. [2,7] is used as the center of the grid and the coordinates of the predicted vehicle and driving information are filled in. The driving data in the other grids are filled in according to the lanes of the surrounding vehicles and the distance between the surrounding vehicles and the predicted vehicle. The driving data includes the lane information, position coordinate information, speed and acceleration information, and vehicle category information of all vehicles in the same time period.

[0024] This invention considers the relative position information between the target vehicle and surrounding vehicles, which can reduce the range of data changes compared to the original position information. This helps the subsequent prediction model learn relevant information more quickly, reduces training time, and accelerates model convergence. This invention also considers driving intention as relevant information, which can enrich the types of model inputs and improve the accuracy of subsequent model predictions. Driving intention is defined in two dimensions: vertical and horizontal, with five categories: acceleration, deceleration, constant speed, left turn, and right turn. The driving intention of the same vehicle in each frame is defined and obtained based on the speed changes and horizontal and vertical coordinate changes of the same vehicle in consecutive frames. Combining the above two points, a bird's-eye view G is used to further process the driving data to obtain the speed relationship graph, acceleration relationship graph, and driving intention relationship graph of the target vehicle and surrounding vehicles. These graphs are then input into SuperGATConv to obtain a new relationship graph that aggregates neighbor features. Compared to the original relationship graph, the new relationship graph obtained by SuperGATConv aggregates information from surrounding nodes for each node, resulting in richer information for each node. The three relationship graph inputs are represented as follows:

[0025]

[0026] Since training a model requires absolute positional information, this invention addresses this challenge by using relative positional information to improve training efficiency. and T represents the velocity and acceleration of vehicle i after relative position processing. his For the first frame of the history, T fut t0 is the last frame of the predicted frame, and t0 is the last frame of the historical frames. and Let V[i][j] be the velocity and acceleration of vehicle j after relative position processing, and let A[i][j] and V[i][j] be the velocity relationship diagram and acceleration relationship diagram, respectively. The driving intention relationship diagram MAN[i][j] is designed according to the lateral and longitudinal driving intentions of each vehicle.i This represents the longitudinal driving intention of vehicle i, including acceleration, deceleration, and constant speed driving intentions, lon. i This represents the lateral driving intention of vehicle i, including left and right turns. If both lateral and longitudinal driving intentions are the same, the value is 2; if there is only one identical driving intention, the value is 1; and if there are no identical driving intentions, the value is 0.

[0027] SuperGATConv, as a variant of GAT, is represented as:

[0028]

[0029] e i,j =a T [x i ||x j ]·σ((x i ) T x j )

[0030] Where, x' i Vehicle i aggregates new environmental feature maps from neighboring vehicles, x i Let x be a feature of vehicle i. j Let α be a feature of vehicle j. i,j Let α be the attention weight for the interaction between vehicle i and vehicle j. i,i Let i be the self-attention weights for vehicle i. Let e ​​be the set of information about vehicle i's neighboring vehicles, LeakyReLU be the activation function, and e be the set of information about vehicle i's neighboring vehicles. i,j Let σ represent the relational attention between vehicle i and vehicle j, where a is a parameter in a single-layer feedforward network, and σ is the activation function.

[0031] Step 2: Environmental feature map x' obtained by SuperGATConv i The input is fed into the environment intent encoder, such as... Figure 2As shown, the environment intent encoder consists of five convolutional modules: Conv2d_1, Conv2d_2, Conv2d_3, Conv2d_4, and Conv2d_5. Conv2d_1, the first convolutional module, comprises convolutional layers with 1x1 and 3x3 kernels, a normalization layer (BatchNorm2d), and the ReLU activation function. The remaining four convolutional modules consist of a 1x1 convolutional layer, a 3x3 extended convolutional layer, a normalization layer (BatchNorm2d), and the ReLU activation function. The feature maps are used as inputs to Conv2d_1 and Conv2d_2 respectively. The input to Conv2d_3 is the concatenation of the outputs of Conv2d_1 and Conv2d_2. The input to Conv2d_4 is the concatenation of the outputs of Conv2d_1, Conv2d_2, and Conv2d_3. The input to the Conv2d_5 module is the concatenation of the outputs of the first four modules. This residual connection method can reduce the information loss caused by convolution. At the same time, dilated convolution can expand the receptive field of the convolution kernel and collect richer information about neighboring vehicles.

[0032] Step 3: Concatenate the five different types of information from the driving data and input them into the LSTM network to obtain time-coded features, including the target vehicle's lane information, position coordinates, speed information, acceleration information, and vehicle category information. The rich multi-input features can help LSTM extract historical driving states.

[0033] Step 4: Concatenate the spatial and temporal encoded features and input them into the spatiotemporal decoder. The spatiotemporal decoder uses the PatchTST model, such as... Figure 3 As shown, the input is first normalized and slicing to divide the long sequence into smaller sequence blocks. Then, the smaller sequence blocks are mapped and positionally encoded before being input into a two-layer Transformer encoder. Finally, the small sequence states are restored to the initial sequence length and future state features are obtained through flattening and linear layers. Compared with traditional Transformer encoders and decoders, the PatchTST model has fewer parameters and can handle longer input sequences, which further demonstrates that it can learn the knowledge and information of the sequence and make better predictions.

[0034] Step 5: Input the future state features obtained from the spatiotemporal decoder into the decoder composed of LSTM and activation function ELU to generate the state output. The outputs are then fed into two parallel MLP networks to generate the lateral driving intention Lon and the longitudinal driving intention Lat.

[0035] This driving intention, as an additional output, can better help the model learn the trend of future trajectories. Finally, the two driving intentions are concatenated with the future state obtained from the spatiotemporal decoder and input into the decoder composed of LSTM and MLP to obtain the predicted future driving trajectory of the vehicle.

[0036] Table 1

[0037]

[0038]

[0039] Table 1 shows the experimental results. The root mean square error (RMSE) is used as the indicator to measure the accuracy of trajectory prediction. 1-5s represent the RMSE for predictions from 1s to 5s. Trajectory prediction models from recent years were collected. All models were compared using the same NGSIM dataset and the same evaluation indicators. The average indicator of the present invention is 42.7% higher than that of the state-of-the-art (SOTA) model. The comparison also shows that the model implemented by the present invention is better than other models in all six indicators, which proves the true effectiveness of the present invention.

Claims

1. A vehicle trajectory prediction method based on multi-vehicle spatio-temporal interaction relationship, characterized in that, Includes the following steps: Step 1: Based on the highway driving dataset, generate speed relationship map, acceleration relationship map and driving intention relationship map between the target vehicle and surrounding vehicles, and then use SuperGATConv to fuse and generate a new environmental feature map that aggregates the neighbor features. Step 2: Input the new environmental feature map into the environmental intent encoder to generate spatial encoded features. The environmental intent encoder uses a multi-layer dilated convolutional network. Step 3: Use the vehicle data information of the target vehicle to be predicted as the input feature and input it into the LSTM to obtain the time-encoded feature; Step 4: Concatenate the spatial coding features and the temporal coding features, and then input them into the spatiotemporal decoder to obtain the future state features; Step 5: Based on the features of the future state, generate the predicted future trajectory through the prediction module built by LSTM and MLP, and perform reverse training; The specific process for generating the speed relationship diagram, acceleration relationship diagram, and driving intention relationship diagram between the target vehicle and surrounding vehicles is as follows: The target vehicle to be predicted, its vehicle data, and the driving data of surrounding vehicles are obtained from the highway driving dataset. The driving data includes lane information, position coordinates, speed, and acceleration information for all vehicles within the same time period. Speed ​​relationship graphs, acceleration relationship graphs, and driving intention relationship graphs are input, represented as follows: ； ； ； in, and Let be the velocity and acceleration of vehicle i after relative position processing. The first frame of the history. The last frame of the predicted frame; and Let be the velocity and acceleration of vehicle j after relative position processing. and These are velocity relationship diagrams, acceleration relationship diagrams, and driving intention relationship diagrams. Designed according to the lateral and longitudinal driving intentions of each vehicle. Represented as a vehicle The longitudinal driving intention, including acceleration, deceleration, and constant speed driving intention. Represented as a vehicle The lateral driving intention, including left turn and right turn, is 2 if both lateral and longitudinal driving intentions are the same, 1 if there is only one identical driving intention, and 0 if there are no identical driving intentions. The new environmental feature map is specifically represented as follows: ； ； ； in, It is a vehicle This aggregates new environmental feature maps of neighboring vehicles. Represented as a vehicle Features Represented as a vehicle Features For vehicles With vehicles Attention weights in the interaction between them For vehicles Self-attention weights For vehicles A collection of information about neighboring vehicles. For activation function, For vehicles With vehicles Relationship attention between them These are the parameters in a single-layer feedforward network. For activation functions; The environment intent encoder consists of five convolutional modules: Conv2d_1, Conv2d_2, Conv2d_3, Conv2d_4, and Conv2d_5. Conv2d_1, the first convolutional module, consists of convolutional layers with 1x1 and 3x3 kernels, a normalization layer BatchNorm2d, and a ReLU activation function. The remaining four convolutional modules consist of a 1x1 convolutional layer, a 3x3 extended convolutional layer, a normalization layer BatchNorm2d, and a ReLU activation function. The environment feature maps serve as the inputs to Conv2d_1 and Conv2d_2, respectively. The input to Conv2d_3 is the concatenation of the outputs of Conv2d_1 and Conv2d_2. The input to Conv2d_4 is the concatenation of the outputs of Conv2d_1, Conv2d_2, and Conv2d_3. The input to Conv2d_5 is the concatenation of the outputs of the first four modules. The prediction module is implemented as follows: The future state features obtained from the spatiotemporal decoder are input into the decoder composed of LSTM and the activation function ELU to generate the state output. The outputs are then fed into two parallel MLP networks to generate the lateral driving intention and the longitudinal driving intention. Finally, the generated lateral driving intention, longitudinal driving intention, and future state features are concatenated and then input into a decoder composed of LSTM and MLP in sequence to generate the predicted future trajectory.

2. The vehicle trajectory prediction method based on multi-vehicle spatiotemporal interaction relationship according to claim 1, characterized in that, The vehicle's speed and acceleration are calculated as follows: ； ； ； ； in, This is the last frame in the history.

3. The vehicle trajectory prediction method based on multi-vehicle spatiotemporal interaction relationship according to claim 1, characterized in that... The vehicle data information includes the target vehicle's lane information, location coordinates, speed information, acceleration information, and vehicle category information. The rich multi-input features can help LSTM extract historical driving states.

4. The vehicle trajectory prediction method based on multi-vehicle spatiotemporal interaction relationship according to claim 1, characterized in that, The spatiotemporal decoder uses PatchTST, a temporal prediction model based on Transformer.

Images