A vehicle trajectory prediction method based on graph attention interaction mechanism

Abstract

The present invention belongs to the technical field of autonomous driving, and specifically relates to a vehicle trajectory prediction method based on a graph attention interaction mechanism; the method comprises: obtaining a historical vehicle trajectory, preprocessing the historical vehicle trajectory, using a spatiotemporal pyramid network to process the preprocessed historical vehicle trajectory data, and obtaining scene features of the vehicle trajectory; using a spatiotemporal transformer network to process the preprocessed historical vehicle trajectory data, and obtaining interaction features of the vehicle trajectory; using a feature fusion network to process the scene features of the vehicle trajectory and the interaction features of the vehicle trajectory, and obtaining fusion features; analyzing and processing the fusion features, and obtaining a vehicle historical trajectory prediction result, and the autonomous driving vehicle travels according to the trajectory prediction result; the present invention has high prediction accuracy for vehicle trajectories and has broad application prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a vehicle trajectory prediction method based on a graph attention interaction mechanism. Background Art

[0002] With the rapid development of highway networks and the rapid increase in the number of vehicles, traffic congestion and traffic accidents continue to increase due to factors such as drivers' inability to respond quickly to surrounding information. At the same time, with the rapid development of artificial intelligence, autonomous vehicles will inevitably be the core of the highly intelligent transportation system in the future. In order for autonomous vehicles to drive safely and efficiently on the road, it is crucial to predict the future trajectory of nearby traffic participants based on the past state of the surrounding environment. The existing trajectory prediction method uses a recurrent neural network, which captures the operating status of traffic participants through potential states, treats the interactions of traffic participants equally, and uses an attention mechanism to weight traffic participants according to the learned function. However, the method using the attention mechanism cannot fully simulate the interaction between traffic participants, and the recurrent neural network has difficulty modeling the long-term dependency of input data.

[0003] In summary, providing a trajectory prediction method that can analyze the historical trajectory data of autonomous driving and effectively predict the future trajectories of traffic participants around the autonomous driving, so as to better plan future paths for the autonomous driving, avoid traffic accidents, and improve the accuracy of trajectory prediction has certain practical significance. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a vehicle trajectory prediction method based on a graph attention interaction mechanism, the method comprising: obtaining vehicle historical trajectory data, preprocessing the historical vehicle trajectory data, processing the preprocessed historical vehicle trajectory data, obtaining a vehicle trajectory prediction result, and the autonomous driving vehicle travels according to the trajectory prediction result;

[0005] The process of processing the preprocessed historical vehicle trajectory data includes:

[0006] S1: Use the spatiotemporal pyramid network to process the preprocessed historical vehicle trajectory data to obtain the scene characteristics of the vehicle trajectory;

[0007] S2: Use the spatiotemporal transformer network to process the preprocessed historical vehicle trajectory data to obtain the interactive features of the vehicle trajectory;

[0008] S3: Use a feature fusion network to process the scene features of the vehicle trajectory and the interactive features of the vehicle trajectory to obtain fused features;

[0009] S4: Analyze and process the fused features to obtain the vehicle historical trajectory prediction result.

[0010] Preferably, the process of preprocessing the historical vehicle trajectory data includes:

[0011] Perform a cleaning operation on the historical vehicle trajectory data to obtain the historical vehicle trajectory data after removing abnormal data and supplementing missing data;

[0012] Data extraction is performed on the cleaned historical vehicle trajectory data to obtain the BEV map and the coordinate data of the vehicle trajectory.

[0013] Preferably, the process of processing the preprocessed historical vehicle trajectory data using the spatiotemporal pyramid network includes:

[0014] S11: Extracting multi-scale spatial features of vehicle trajectories using 2D convolution in spatiotemporal pyramid network;

[0015] S12: Extract multi-scale temporal features of vehicle trajectories using one-dimensional convolution in spatiotemporal pyramid networks;

[0016] S13: Transversely connect the multi-scale spatial features of the vehicle trajectory and the multi-scale temporal features of the vehicle trajectory to obtain scene features of different scales of the vehicle trajectory.

[0017] Preferably, the process of using a spatiotemporal transformer network to process the preprocessed historical vehicle trajectory data includes: the spatiotemporal transformer network includes a first encoder, a second encoder and a decoder;

[0018] S21: using the first encoder to process the preprocessed historical vehicle trajectory data to obtain spatiotemporal coding features;

[0019] S22: using a second encoder to process the spatiotemporal coding features to obtain new spatiotemporal coding features;

[0020] S23: Use a decoder to decode the new spatiotemporal encoding features to obtain interactive features of the vehicle trajectory.

[0021] Further, the process of the first encoder processing the preprocessed historical vehicle trajectory data includes: the first encoder includes three connection layers, a time transformer network and a space transformer network;

[0022] Two fully connected layers are used to process the preprocessed historical vehicle trajectory data to obtain the embedded representation of the historical vehicle trajectory;

[0023] The embedded representation of historical vehicle trajectories is input into the temporal transformer network and the spatial transformer network respectively to obtain temporal features and spatial features;

[0024] The temporal features and spatial features are input into the fully connected layer to obtain the spatiotemporal encoding features.

[0025] Furthermore, the output of a node in the spatial transformer network is expressed as:

[0026]

[0027] h' i =f out (Att(i))+Att(i)

[0028] Among them, Att(i) represents the output of the self-attention mechanism of vehicle node i, m j→i represents the interaction between vehicle node j and vehicle node i, M represents the sum of the number of neighboring nodes of vehicle node i and vehicle node i, represents the dimension of the query matrix of the attention mechanism, v j represents the weight matrix of vehicle node j, h i represents the feature vector of the current vehicle node i, f out represents the output function, h' i Represents the feature vector of vehicle node i after updating.

[0029] Furthermore, the process of using the second encoder to process the spatiotemporal coding features includes: the second encoder includes a temporal transformer network and a spatial transformer network;

[0030] Input the spatiotemporal coding features into the spatial transformer network to obtain the spatiotemporal coding features containing spatial interaction information;

[0031] The spatiotemporal coding features containing spatial interaction information are input into the temporal transformer network to obtain the spatiotemporal coding features that enhance temporal attention, namely the new spatiotemporal coding features.

[0032] Preferably, the formula for processing the scene features of the vehicle trajectory and the interactive features of the vehicle trajectory using the feature fusion network is:

[0033]

[0034] Among them, F i represents the fusion feature, X i Represents scene features, h' i represents the interaction feature, ω1 represents the first weight factor, and ω2 represents the second weight factor.

[0035] The beneficial effects of the present invention are as follows: the present invention adopts a graph attention interaction mechanism, that is, the attention mechanism is used on graph convolution to extract the spatial features of vehicle trajectories. Compared with the traditional graph convolution that uses the same matrix for different neighbor nodes of the same node during convolution operations, the present invention uses the attention mechanism on graph convolution to learn different weights for different neighbor nodes, and can capture more complex social interaction information of vehicles; better extract vehicle interaction features; use a spatiotemporal pyramid network to extract scene information of vehicle trajectories, and fuse scene features and interaction features through a fusion network to better extract environmental information around the vehicle trajectory; the present invention has high prediction accuracy for vehicle trajectories and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the vehicle trajectory prediction method based on the graph attention interaction mechanism in the present invention;

[0037] Figure 2 It is a schematic diagram of the structure of the spatiotemporal pyramid network in the present invention;

[0038] Figure 3 It is a schematic diagram of the structure of the temporal transformer network in the present invention;

[0039] Figure 4 It is a schematic diagram of the structure of the spatial transformer network in the present invention;

[0040] Figure 5 It is a schematic diagram of the structure of the spatiotemporal transformer network in the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] This paper proposes a vehicle trajectory prediction method based on graph attention interaction mechanism. Figure 1 As shown, the method includes:

[0043] Acquire historical vehicle trajectory data, preprocess the historical vehicle trajectory data, process the preprocessed historical vehicle trajectory data, obtain vehicle trajectory prediction results, and the autonomous driving vehicle drives according to the trajectory prediction results;

[0044] The process of processing the preprocessed historical vehicle trajectory data includes:

[0045] S1: Use the spatiotemporal pyramid network to process the preprocessed historical vehicle trajectory data to obtain the scene characteristics of the vehicle trajectory;

[0046] S2: Use the spatiotemporal transformer network to process the preprocessed historical vehicle trajectory data to obtain the interactive features of the vehicle trajectory;

[0047] S3: Use a feature fusion network to process the scene features of the vehicle trajectory and the interactive features of the vehicle trajectory to obtain fused features;

[0048] S4: Analyze and process the fused features to obtain the vehicle historical trajectory prediction result.

[0049] The specific process of preprocessing historical vehicle trajectory data is as follows:

[0050] Performing a cleaning operation on the historical vehicle trajectory data, that is, removing incomplete trajectory data and supplementing the missing values ​​of the trajectory data, to obtain the historical vehicle trajectory data after removing abnormal data and supplementing the missing data;

[0051] The cleaned historical vehicle trajectory data is extracted. The specific process of data extraction is as follows:

[0052] The original 3D point cloud data in the historical vehicle trajectory data is projected into a 2D space, i.e., converted into a 2D pseudo image. The operation is to quantize the 3D point cloud into voxels and map the 3D height information into channels of the image to obtain a BEV map.

[0053] The coordinate data of the vehicle trajectory is extracted from the historical vehicle trajectory data.

[0054] The specific process of processing the preprocessed historical vehicle trajectory data is as follows:

[0055] The spatiotemporal pyramid network is used to process the preprocessed historical vehicle trajectory data to obtain the scene characteristics of the vehicle trajectory; the specific process is as follows:

[0056] like Figure 2 As shown in the figure, the spatiotemporal pyramid network includes: spatiotemporal convolution blocks, temporal pooling layers and feature fusion layers.

[0057] The spatiotemporal pyramid network mainly uses convolution blocks to extract features from historical vehicle trajectories. A convolution block contains a two-dimensional convolution and a three-dimensional convolution with a kernel function of (k, 1, 1), which is a real one-dimensional convolution. k is the time dimension corresponding to the three-dimensional convolution. For example, k is 5. The BEV graph obtained by preprocessing the historical vehicle trajectory data is input into the spatiotemporal pyramid network. In the spatial dimension, the two-dimensional convolution in the spatiotemporal pyramid network is used to extract spatial features of the BEV graph with each increment of 2 times the initial sample to obtain the multi-scale spatial features of the vehicle trajectory; in the temporal dimension, the one-dimensional convolution in the spatiotemporal pyramid network is used to extract temporal features, and the multi-scale temporal features of the vehicle trajectory are obtained by gradually reducing the temporal resolution. The multi-scale spatial features and multi-scale temporal features of the vehicle trajectory are both temporally pooled to obtain high-level semantic features of the temporal pooling. The high-level semantic features of the temporal pooling are fused, that is, the multi-scale spatial features of the vehicle trajectory and the multi-scale temporal features of the vehicle trajectory are horizontally connected to obtain spatiotemporal features of different scales, namely scene features.

[0058] Construct a temporal transformer network and a spatial transformer network; the core idea of ​​the Transformer is to completely replace recursion with multi-head attention. For the embedding of the input Transformer’s self-attention first learns all embedding query matrices from time t=1 to T key matrix And the corresponding value matrix The output of the self-attention mechanism in the temporal transformer network is calculated based on the query matrix, key matrix, and value matrix. The calculation formula is:

[0059]

[0060] Among them, Att(Q,K,V) represents the output of the self-attention mechanism in the temporal transformer network, d k Represents the dimension of the Query matrix or key matrix, and Softmax() represents the Softmax function.

[0061] Build a temporal transformer network, including:

[0062] Considering that at time steps 1 to T obs The historical vehicle trajectories observed in the process are used to predict the number of N vehicles in a scene from time step T. obs +1 to T future driving trajectory. At each time step t, there are N vehicles in Indicates the position of the vehicle mapped in the historical trajectory top view, represents the horizontal coordinate of the i-th vehicle at time t, represents the vertical coordinate of the i-th car at time t. Suppose two cars At time t, there is an undirected edge (i, j). At time t, an undirected graph is constructed based on the vehicle position, represented as G t =(V t ,E t ),in E t ={(i,j)|i,j are connected at time t}.

[0063] like Figure 3 As shown, a set of vehicle trajectories are used to embed As input, the temporal transformer network outputs a set of new embedding representations with temporal dependencies. Considering each car independently, the self-attention module first learns the Query matrix given the input key matrix and value matrix For a car, the matrix learned by the self-attention module is represented as:

[0064]

[0065]

[0066]

[0067] Among them, f Q , f K and f V They represent the functions of the corresponding calculation matrices Query, key and value of vehicle sharing respectively.

[0068] Calculate the attention of each vehicle, that is, the output of the self-attention module, and the calculation formula is:

[0069]

[0070] The output of the final temporal transformer network is generated by the output of the self-attention module through two skip connections and a final fully connected layer.

[0071] Build a spatial transformer network, including:

[0072] This paper proposes a transformer-based graph convolution method to extract spatial interaction information between vehicles. The self-attention mechanism can be regarded as message passing on an undirected fully connected graph. A vector h ini , the corresponding Query vector can be expressed as q i =f Q (h i ), the key vector is represented by k i =f K (h i ), the value vector is represented by v i =f V (h i ); define the interaction from node j to i in the fully connected graph as:

[0073]

[0074] Rewrite the formula for calculating attention as:

[0075]

[0076] like Figure 4 As shown, for any graph G t =(V t ,E t ), assuming that each node is associated with an embedding h i Associated with its neighbor node set, the graph convolution operation of vehicle node i can be written as:

[0077]

[0078] h' i =f out (Att(i))+Att(i)

[0079] Among them, Att(i) represents the self-attention mechanism output of node i, m j→i represents the interaction between vehicle node j and vehicle node i, M represents the sum of the number of vehicle node i and its neighboring nodes, M = Nb(i)∪{i}, Nb(i) represents the neighboring nodes of vehicle node i, {i} represents the current vehicle node, d k represents the dimension of the query matrix of the attention mechanism, v j represents the weight matrix of vehicle node j, h i represents the feature vector of the current vehicle node i, f out is the output function, which represents the fully connected layer in this invention, h' i It is the updated feature vector of vehicle node i, i.e., the node output.

[0080] The spatiotemporal transformer network is used to process the coordinate data of the historical vehicle trajectory data after preprocessing to obtain the interactive features of the vehicle trajectory; the specific process is as follows:

[0081] like Figure 5 As shown, the spatiotemporal transformer network includes a first encoder, a second encoder and a decoder;

[0082] The first encoder is used to process the preprocessed historical vehicle trajectory data to obtain spatiotemporal coding features; the first encoder includes three connection layers, a time transformer network and a space transformer network; the specific process of processing the preprocessed historical vehicle trajectory data is as follows: the position information at time t obtained by preprocessing the historical trajectory of the surrounding vehicles of the vehicle The layers are input into two parallel fully connected layers for processing to obtain the embedded representation of the historical vehicle trajectory; the embedded representation of the historical vehicle trajectory is input into the temporal transformer network and the spatial transformer network to obtain the temporal features and spatial features; the temporal features and spatial features are simply directly connected and the connected temporal features and spatial features are input into the full connection to obtain the spatiotemporal coding features.

[0083] A second encoder is used to process the spatiotemporal coding features to obtain new spatiotemporal coding features; the second encoder includes a temporal transformer network and a spatial transformer network; the specific process of processing the spatiotemporal coding features is: inputting the spatiotemporal coding features into the spatial transformer network, using temporal information to model spatial interactions, and obtaining spatiotemporal coding features containing spatial interaction information; inputting the spatiotemporal coding features containing spatial interaction information into the temporal transformer network, using spatial information to enhance temporal attention, and obtaining spatiotemporal coding features with enhanced temporal attention, namely, new spatiotemporal coding features.

[0084] The decoder is used to decode the new spatiotemporal coding features to obtain the interactive features of the vehicle trajectory; the decoder is a fully connected layer, and the decoding process is the inverse process of the encoding process. The spatiotemporal coding features are passed through the fully connected layer to obtain the interactive features of the vehicle trajectory. By using the mutual influence between nodes modeled by graph convolution and using the attention mechanism to calculate the mutual influence between nodes, more complex social interaction information of vehicles can be captured.

[0085] The feature fusion network is used to process the scene features of the vehicle trajectory and the interactive features of the vehicle trajectory to obtain the fusion features; the specific process is as follows:

[0086] The scene features obtained by the spatiotemporal pyramid network and the interactive features obtained by the spatiotemporal transformer are fused at a single scale; the formula for feature fusion is:

[0087]

[0088] Among them, F i represents the fusion feature, X i Represents scene features, h' i represents the interaction feature, ω1 represents the first weight factor, and ω2 represents the second weight factor. Different weight factors are obtained according to the actual traffic scene. For example, if there are more interactions in a scene, the interaction feature weight factor is larger. Preferably, the weight factor can be calculated according to the entropy method.

[0089] The fused features are analyzed and processed to obtain the vehicle historical trajectory prediction results.

[0090] The present invention adopts a graph attention interaction mechanism, that is, an attention mechanism is used on a graph convolution to extract the spatial features of a vehicle trajectory. Compared with the traditional graph convolution that uses the same matrix for different neighbor nodes of the same node during convolution operations, the present invention uses an attention mechanism on a graph convolution to learn different weights for different neighbor nodes, and can capture more complex social interaction information of vehicles; better extract vehicle interaction features; use a spatiotemporal pyramid network to extract scene information of vehicle trajectories, and fuse scene features and interaction features through a fusion network to better extract environmental information around the vehicle trajectory; the present invention has high prediction accuracy for vehicle trajectories and has broad application prospects.

[0091] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle trajectory prediction method based on graph attention interaction mechanism, characterized in that: include: Acquire historical vehicle trajectory data, preprocess the historical vehicle trajectory data, process the preprocessed historical vehicle trajectory data, obtain vehicle trajectory prediction results, and the autonomous driving vehicle drives according to the trajectory prediction results; The process of preprocessing the historical vehicle trajectory data includes: performing a cleaning operation on the historical vehicle trajectory data to obtain the historical vehicle trajectory data after removing abnormal data and supplementing missing data; performing data extraction on the cleaned historical vehicle trajectory data to obtain the BEV map and the coordinate data of the vehicle trajectory; The process of processing the preprocessed historical vehicle trajectory data includes: S1: Use the spatiotemporal pyramid network to process the preprocessed historical vehicle trajectory data to obtain the scene characteristics of the vehicle trajectory; S11: Extracting multi-scale spatial features of vehicle trajectories using 2D convolution in spatiotemporal pyramid network; S12: Extract multi-scale temporal features of vehicle trajectories using one-dimensional convolution in spatiotemporal pyramid networks; S13: horizontally connecting the multi-scale spatial features of the vehicle trajectory and the multi-scale temporal features of the vehicle trajectory to obtain scene features of different scales of the vehicle trajectory; S2: The preprocessed historical vehicle trajectory data is processed using a spatiotemporal transformer network to obtain the interactive features of the vehicle trajectory; the spatiotemporal transformer network includes a first encoder, a second encoder and a decoder; S21: using the first encoder to process the preprocessed historical vehicle trajectory data to obtain spatiotemporal coding features; the first encoder includes three connection layers, a temporal transformer network and a spatial transformer network; Two fully connected layers are used to process the preprocessed historical vehicle trajectory data to obtain the embedded representation of the historical vehicle trajectory; The embedded representation of the historical vehicle trajectory is input into the temporal transformer network and the spatial transformer network respectively to obtain the temporal features and spatial features; the output of the node in the spatial transformer network is expressed as: h i '=f out (At(i))+At(i) Among them, Att(i) represents the output of the self-attention mechanism of vehicle node i, m j→i represents the interaction between vehicle node j and vehicle node i, M represents the sum of the number of neighboring nodes of vehicle node i and vehicle node i, d k represents the dimension of the query matrix of the attention mechanism, v j represents the weight matrix of vehicle node j, h i represents the feature vector of the current vehicle node i, f out represents the output function, h i ' represents the feature vector of vehicle node i after updating; Input the temporal features and spatial features into the fully connected layer to obtain spatiotemporal coding features; S22: using a second encoder to process the spatiotemporal coding features to obtain new spatiotemporal coding features; the second encoder includes a temporal transformer network and a spatial transformer network; Input the spatiotemporal coding features into the spatial transformer network to obtain the spatiotemporal coding features containing spatial interaction information; The spatiotemporal coding features containing spatial interaction information are input into the temporal transformer network to obtain the spatiotemporal coding features that enhance temporal attention, namely the new spatiotemporal coding features. S23: using a decoder to decode the new spatiotemporal encoding features to obtain the interactive features of the vehicle trajectory; S3: The scene features of the vehicle trajectory and the interactive features of the vehicle trajectory are processed using the feature fusion network to obtain fusion features; the formula for processing the scene features of the vehicle trajectory and the interactive features of the vehicle trajectory using the feature fusion network is: Among them, F i represents the fusion feature, X i represents the scene features, h i " represents the interaction feature, ω1 represents the first weight factor, ω2 represents the second weight factor, and N represents the number of vehicles; S4: Analyze and process the fused features to obtain the vehicle historical trajectory prediction result.

Images