Vehicle multi-modal trajectory prediction method based on improved attention network

Abstract

The invention discloses a vehicle multi-modal trajectory prediction method based on an improved attention network, and belongs to the technical field of intelligent vehicle trajectory prediction, and the method comprises the steps: collecting historical trajectory data of a target vehicle and surrounding vehicles as an input sequence; secondly, constructing a vehicle multi-modal trajectory prediction model which comprises a motion feature extraction module, a space-time interaction module, a space-time fusion module and a trajectory output module; the motion feature extraction module uses a multi-scale convolution attention network and a gating circulation unit for processing, the space-time interaction module uses a dynamic graph attention network for extracting vehicle interaction information, and the space-time fusion module splices and fuses target vehicle motion features and space-time interaction features to obtain space-time fusion features; the track output module inputs the fusion features into a gating circulation unit, decodes the fusion features and then inputs the fusion features into a mixed density network, and multi-mode output of vehicle tracks is achieved; and finally, a proper loss function is selected for training, so that the prediction precision and the convergence speed of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent vehicle trajectory prediction, and in particular relates to a vehicle multimodal trajectory prediction method based on an improved attention network. Background Art

[0002] With the rapid development of autonomous driving technology, vehicle trajectory prediction, as a core link in environmental perception and decision-making planning, directly affects the safety and reliability of autonomous driving systems. In complex dynamic traffic scenarios, the future trajectory of the target vehicle is not only affected by its own motion laws, but is also closely related to the interactive behaviors of surrounding vehicles. Existing trajectory prediction methods are mainly divided into two categories: traditional methods based on physical models and data-driven deep learning methods. Traditional methods (such as Kalman filtering and Markov chains) rely on vehicle kinematic models to predict future positions by extrapolating historical trajectories. However, such methods have difficulty in constructing complex vehicle interaction relationships, especially in scenarios with dense vehicles and changing intentions, where prediction errors are significant. Data-driven methods (such as LSTM and CNN) improve prediction capabilities by extracting spatiotemporal features of trajectories, but there are still the following bottlenecks: (1) Feature confusion and interference: Traditional vehicle trajectory prediction methods usually mix the historical trajectories of the target vehicle and surrounding vehicles and input them into the same feature extraction module, ignoring the essential differences between the two in terms of motion patterns, temporal dependency strength, and contribution to the prediction results.

[0003] (2) Limitations of temporal feature extraction: Existing vehicle trajectory prediction methods usually use fixed-scale convolutional networks or recurrent neural networks to extract the temporal features of vehicle historical trajectories, which makes it difficult to capture the local detail changes and global behavior trends of the vehicle, resulting in insufficient perception of the trajectory evolution law in dynamic target scenarios.

[0004] (3) Static interaction modeling: Existing spatiotemporal interaction modules mostly use a fixed Euclidean distance threshold (e.g., <30m) to construct the adjacency matrix, making it impossible to dynamically perceive the changes in the degree of influence of surrounding vehicles in scenarios such as highways. At the same time, the traditional graph attention mechanism ignores the spatiotemporal differences in interaction intensity, causing the model to be easily disturbed by invalid or weakly interacting vehicles, increasing the computational burden and potentially reducing prediction accuracy. Summary of the Invention

[0005] In response to the above-mentioned problems existing in the prior art, the present invention proposes a vehicle multimodal trajectory prediction method based on an improved attention network, which has a reasonable design, solves the shortcomings of the prior art, and has good results.

[0006] A vehicle multimodal trajectory prediction method based on an improved attention network includes the following steps: Step 1: Collect historical trajectory data of the target vehicle and surrounding vehicles in the target scene as input sequence; Step 2: Construct a vehicle multimodal trajectory prediction model, including a motion feature extraction module, a spatiotemporal interaction module, a spatiotemporal fusion module, and a trajectory output module; The motion feature extraction module divides the historical states of the target vehicle and its surrounding vehicles into two parts, which are processed using a multi-scale convolutional attention network and a gated recurrent unit respectively. The spatiotemporal interaction module uses a learnable threshold to construct a dynamic graph mechanism and extracts vehicle interaction information through the dynamic graph attention network. The spatiotemporal fusion module combines the target vehicle motion features with the spatiotemporal interaction features to obtain spatiotemporal fusion features. The trajectory output module inputs the fused features into a gated recurrent unit, decodes them, and then inputs them into a mixture density network to achieve multimodal output of vehicle trajectories. Step 3: Select a loss function for training to improve the prediction accuracy and convergence speed of the model.

[0007] Furthermore, in step 1, the historical trajectory data includes the target vehicle and its surrounding vehicles in the time step The historical state within is expressed as: ;(1) in, Indicates selection The vehicle is the target vehicle. Represents the historical trajectory data of the target vehicle; Indicates the surrounding vehicle index, , is the number of surrounding vehicles; Represents historical trajectory data of surrounding vehicles; and Represent the target vehicle and surrounding vehicles respectively The vehicle status at the moment, including the vehicle and Six-dimensional features of position, velocity and acceleration in direction; The vehicle's historical trajectory data is input into a pre-built vehicle multimodal trajectory prediction model based on an improved attention network to perform motion feature extraction and dynamic interaction modeling operations, and extract the high-dimensional spatiotemporal fusion features of the target vehicle for multimodal trajectory prediction.

[0008] Furthermore, step 2 includes the following sub-steps: Step 21: Convert the historical trajectory sequence of the target vehicle and surrounding vehicles into a high-dimensional vector through an embedding layer; Step 22: Input the target vehicle's historical trajectory sequence into a multi-scale convolutional attention network to capture temporal features and process local and global trends in the target vehicle's time series. Input the surrounding vehicle's historical trajectory sequence into a gated recurrent unit (GRU) to process the surrounding vehicle's historical time series input features. Step 23: Build a dynamic spatiotemporal interaction graph, using the target vehicle and surrounding vehicles as nodes in the graph. Dynamically construct an adjacency matrix using a learnable threshold to determine the edge relationships between vehicles. Based on this, use the dynamic graph attention mechanism to extract spatiotemporal interaction features. Step 24: Align the target vehicle motion features and the spatiotemporal interaction features in the time dimension and then perform splicing and fusion to obtain spatiotemporal fusion features; Step 25: Input the fused features into the GRU, and after decoding, input them into the mixture density network to learn the potential trajectory distribution from the spatiotemporal features and realize the multimodal output of the vehicle trajectory.

[0009] Furthermore, in step 21, the embedding layer converts the original six-dimensional information of the scene vehicle into a high-dimensional vector through linear transformation and activation function, which is expressed as: ;(2) in, 、 Respectively represent the motion state embedding features of the target vehicle and surrounding vehicles at time step t; represents the weight matrix of the embedding layer; represents a linear layer; GELU represents a Gaussian error linear unit.

[0010] Furthermore, in step 22, the multi-scale convolutional attention network first encodes the target vehicle position vector Add to the embedding vector after linear transformation and activation function to get the embedding vector after position encoding ,in, Use sine and cosine functions of different frequencies to encode the temporal position of each time step; The multi-scale convolutional attention network utilizes a multi-head convolutional mapping mechanism and a multi-head attention mechanism to jointly capture the local behavior trend characteristics and global time dependencies of the target vehicle trajectory in different time intervals; The multi-head convolution mapping mechanism adopts an improved TCN network, which takes the historical trajectory feature sequence of the target vehicle after position encoding as input, and outputs the sequence through two layers of stacked dynamic expansion causal convolution. , weight normalization and GELU convolution activation are performed after each layer of dynamically expanded causal convolution. Layer normalization is performed after the GELU convolution activation of the first layer to enhance network stability and expressiveness, and residual connections are added to the GELU activation of the second layer to alleviate the gradient vanishing problem; The dynamic convolution expansion rate is used in each layer of dynamic expansion causal convolution to control the sampling step size. The expressions of the two layers of expansion causal convolution are: ;(8) ;(9) in, Indicates the current time step position; and Represent the first layer of convolution kernels respectively and the second layer of convolution kernel In the The weight of each position; and Represent the sizes of the first and second convolution kernels respectively; The dynamic expansion causal convolution extracts the real-time vehicle speed from the target vehicle's historical trajectory feature sequence. , calculate the dynamic convolution expansion rate of each layer through learnable parameters , specifically expressed as: ;(10) in, Indicates the Layer dynamic dilation causal convolution; and Represents independent learnable parameters of each layer; Represents the truncation function, which is used to Limit to a preset minimum value and maximum value within the scope; The convolution trend perception feature matrix Perform linear projection and generate a query matrix by mapping the dimensions from the convolution feature space to the attention space, capturing the temporal dependency features and trend features in different subspaces. , key matrix , value matrix The multi-head attention mechanism is specifically based on each group of multi-head convolutional mappings generated 、 and , calculate the attention weight of the target vehicle in each subspace, concatenate the outputs of all attention heads and linearly transform them to obtain the final trend perception time series features ; The resulting multi-head attention output and convolution output features After performing residual connection and layer normalization, we get , Enter the feedforward neural network FNN and connect it again with residual connection, and the final output is the target vehicle motion feature ; The historical trajectory sequence of surrounding vehicles is input into the gated recurrent unit GRU to extract the motion features of surrounding vehicles, and the motion features of each vehicle are obtained. The final hidden state at the moment , and finally output the motion characteristics of each surrounding vehicle .

[0011] Furthermore, the step 23 is specifically as follows: for the target vehicle At time step Speed , acceleration , local traffic density and motion time series feature vector , jointly form the scene feature vector , input to the multi-layer perceptron MLP to predict the dynamic neighbor selection threshold , the expression is: ; (18) in, 、 、 、 is a learnable parameter, Ensure that the output is positive; The predicted value is linearly adjusted by introducing the minimum safety threshold and scaling factor to obtain the set threshold , determine the target vehicle With surrounding vehicles The Euclidean distance between Less than the set threshold , based on which the time step is constructed The spatial adjacency matrix of , the expression is: ; (20) Build dynamic spatial interaction diagrams , each car in the scene is regarded as a set of nodes in the graph , each node corresponds to the motion characteristics of the target vehicle and surrounding vehicles in step 22, based on the time step Adjacency matrix obtained by learning threshold screening , will satisfy Node pairs As edges, formed at time step The edge set ; The dynamic graph attention network uses a multi-head attention mechanism, where each attention head uses attention weights to At time step Surrounding vehicles The weighted sum of the features of the nodes is performed to achieve the effective fusion of vehicle dynamic interaction information and the modeling of spatial dependency, and the output results of all heads are averaged in the feature dimension to obtain the node at the current time step. Updated feature representation , the specific calculation expression is as follows: ;(twenty two) in, is the number of attention heads; and Indicates the The attention weights and weight matrices of each head; is a nonlinear activation function GELU; the attention weight is measured under the constraints of the dynamic adjacency matrix. Moment target vehicles in the attention heads With surrounding vehicles The quantitative indicator of interaction strength is expressed as follows: ;(twenty three) Where, express Moment Target vehicle node in the attention head Neighboring vehicle nodes The attention weight of ; ∥ represents the vector concatenation operation; Indicates the The learnable attention vector of the attention head, T represents the transpose; LeakyReLU represents the nonlinear activation function; represents the spatial neighbor index, Indicates that the target vehicle A set of vehicles with spatial adjacency; By observing the time window The above updates are performed for each time step and stacked in time order to obtain the spatiotemporal interaction feature sequence of the target vehicle in the observation sequence .

[0012] Furthermore, in step 24, the target vehicle motion characteristics Interaction characteristics with time and space After the time dimension is aligned, splicing and fusion are performed to obtain the spatiotemporal fusion features , the expression is: ;(twenty four) The spliced ​​spatiotemporal fusion features are input into the The multi-layer perceptron MLP is used, and the GELU nonlinear activation function is used to enhance the representation ability, obtain the feature representation of the dimension required for decoding, and standardize the feature distribution through layer normalization to obtain the sequence representation for subsequent trajectory prediction decoding , the expression is: (25).

[0013] Furthermore, in step 25, the trajectory output module includes a GRU and a mixture density network; The GRU will fuse the feature sequence As input, the initial hidden state of the trajectory output module Initialized by the last moment state of the fused features; Progressively decode future time steps through GRU The hidden state of is expressed as: ; (27) The mixture density network will As input, it is used to predict the multimodal future trajectory distribution and output the multimodal Gaussian distribution parameters, including the mean of each mixture component , covariance matrix and blend weights , the expression is: ; (28) in, Indicates the number of preset Gaussian mixture prediction trajectory distributions; Indicates the The predicted mean position of the predicted trajectory; Indicates the The prediction covariance matrix of the predicted trajectory; Indicates the The mixed weight of the predicted trajectory defines the possibility of being sampled as the final trajectory, satisfying ; Based on the mean , covariance matrix and blend weights , for each future time step The trajectory position , output its trajectory distribution probability, the expression is: ;(29) in, represents the probability of trajectory distribution; represents a multivariate Gaussian distribution.

[0014] Furthermore, in step 3, the loss function The negative log-likelihood is used as the regression loss, which measures the distribution of the predicted trajectory. The difference between the actual trajectory and the actual trajectory is expressed as: ; (30) in, Indicates the length of the predicted future time step; represents the actual position of the target vehicle at the prediction time t.

[0015] Beneficial technical effects brought about by the present invention: 1. This paper establishes a vehicle multimodal trajectory prediction model based on an improved attention network, combining the collaborative architecture of dynamic graph attention mechanism and multi-scale convolutional attention, which can effectively solve the problems of inaccurate interactive vehicle recognition and long-term behavior prediction in different scenarios, and effectively improve the vehicle trajectory prediction accuracy through multimodal prediction.

[0016] 2. By separating the historical trajectories of the target vehicle from those of surrounding vehicles, and employing a multi-scale convolutional attention network and a gated recurrent unit for temporal modeling, this approach overcomes the feature confusion caused by mixed inputs in traditional methods. The multi-scale convolutional attention network focuses on the local motion details and global trends of the target vehicle, while the gated recurrent unit efficiently captures the dynamic changes of surrounding vehicles, significantly improving the targeted and independent nature of feature extraction.

[0017] 3. Design a multi-scale convolutional attention network. The bottom layer uses dynamically expanded causal convolution to extract multi-channel local features. The top layer integrates global dependencies through a multi-head attention mechanism and uses residual connections to stabilize deep training. This network can simultaneously capture local behavioral changes and global trend information of trajectories at different time scales, making the temporal feature extraction of trajectory prediction more accurate.

[0018] 4. Dynamic Graph Attention: Based on information such as vehicle speed, acceleration, and local traffic density, it dynamically constructs a spatial adjacency matrix using a learnable threshold. Combined with the dynamic graph attention mechanism, it models the interaction weights between vehicles, accurately capturing the dynamic interactions between vehicles. This approach overcomes the limitations of traditional static graph models, adaptively capturing the interactions between vehicles in complex scenes, and significantly improving the accuracy of spatial relationship modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is the overall structural framework diagram of the vehicle trajectory prediction method of the present invention.

[0020] Figure 2 This is a diagram of the multi-scale convolutional attention network structure of the vehicle trajectory prediction method of the present invention.

[0021] Figure 3Schematic diagram of the spatiotemporal interaction module structure of the vehicle trajectory prediction method of the present invention.

[0022] Figure 4 This is a training stability verification diagram of the vehicle trajectory prediction method of the present invention.

[0023] Figure 5 This is an analysis chart of the impact of the number of multimodal trajectories on the evaluation indicators of the present invention.

[0024] Figure 6 This is an analysis chart of the impact of the single-mode prediction duration on the evaluation index of the present invention.

[0025] Figure 7 This is an analysis chart of the impact of the multimodal prediction duration on the evaluation indicators of the present invention. DETAILED DESCRIPTION

[0026] The specific implementation of the present invention will be further described below with reference to specific embodiments: A vehicle multimodal trajectory prediction method based on improved attention network, such as Figure 1 As shown, the following steps are included: Step 1: Collect historical trajectory data of the target vehicle and surrounding vehicles in the target scene as input sequence; The vehicle trajectory prediction problem can be formulated as estimating the future trajectory information of the target vehicle based on the historical trajectory data of the target vehicle and its surrounding vehicles; The historical trajectory data includes the target vehicle and its surrounding vehicles at the time step The historical state within is expressed as: ;(1) in, Indicates selection The vehicle is the target vehicle. There is only one target vehicle in a target scene. Represents the historical trajectory data of the target vehicle; Indicates the surrounding vehicle index, , is the number of surrounding vehicles; Represents historical trajectory data of surrounding vehicles; and Represent the target vehicle and surrounding vehicles respectively The vehicle status at the moment, including the vehicle and Six-dimensional features of position, velocity and acceleration in direction; The vehicle's historical trajectory data is input into a pre-built vehicle multimodal trajectory prediction model based on an improved attention network to perform motion feature extraction and dynamic interaction modeling operations, and extract the high-dimensional spatiotemporal fusion features of the target vehicle for multimodal trajectory prediction.

[0027] Step 2: Construct a vehicle multimodal trajectory prediction model, including a motion feature extraction module, a spatiotemporal interaction module, a spatiotemporal fusion module, and a trajectory output module; The motion feature extraction module divides the historical states of the target vehicle and its surrounding vehicles into two parts, which are processed using a multi-scale convolutional attention network and a gated recurrent unit respectively. The spatiotemporal interaction module uses a learnable threshold to construct a dynamic graph mechanism and extracts vehicle interaction information through the dynamic graph attention network. The spatiotemporal fusion module combines the target vehicle motion features with the spatiotemporal interaction features to obtain spatiotemporal fusion features. The trajectory output module inputs the fused features into a gated recurrent unit, decodes them, and then inputs them into a mixture density network to achieve multimodal output of vehicle trajectories. Step 2 includes the following sub-steps: Step 21: Convert the historical trajectory sequence of the target vehicle and surrounding vehicles into a high-dimensional vector through an embedding layer; The embedding layer converts the original six-dimensional information of the scene vehicle into a high-dimensional vector through linear transformation and activation function, which is expressed as: ;(2) in, 、 Respectively represent the motion state embedding features of the target vehicle and surrounding vehicles at time step t; represents the weight matrix of the embedding layer; represents a linear layer; GELU represents a Gaussian error linear unit.

[0028] Step 22: Input the target vehicle's historical trajectory sequence into the multi-scale convolutional attention network encoding layer to capture temporal features and process local and global trends in the target vehicle's time series. Input the surrounding vehicle's historical trajectory sequence into the gated recurrent unit (GRU) to process the surrounding vehicle's historical time series input features. like Figure 2 As shown, the multi-scale convolutional attention network encoding layer first encodes the target vehicle position vector Add to the embedding vector after linear transformation and activation function to get the embedding vector after position encoding , the expression is: ;(3) Use sine and cosine functions of different frequencies to encode the time position for each time step, as expressed by: ;(4) ;(5) in, Represents the position encoding dimension set in the model, Represents each dimension The position encoding vector of The multi-scale convolutional attention network uses a multi-head convolutional mapping mechanism and a multi-head attention mechanism to jointly capture the local behavioral trend characteristics and global temporal dependencies of the target vehicle trajectory in different time intervals; Among them, the multi-head convolution mapping mechanism adopts an improved TCN network, takes the historical trajectory feature sequence of the target vehicle after position encoding as input, and replaces the traditional standard full connection mapping output sequence through two layers of stacked dynamic expansion causal convolution , WeightNorm normalization and GELU activation function are performed after each layer of dynamic expansion causal convolution. Layer normalization is performed after the GELU convolution activation of the first layer to enhance network stability and expression ability. Residual connection is added in the GELU activation of the second layer to alleviate the gradient disappearance problem. The specific expression is: The first layer of dynamic expansion causal convolution: ;(6) The second layer of dynamic expansion causal convolution: ;(7) in, represents the output of the first layer of dynamically expanded causal convolution, represents the trend-aware feature matrix output by the second layer of dynamically expanded causal convolution, and denote the dilation factors of the first and second layers respectively; and Represent the convolution kernels of the first and second layers respectively; The dynamic convolution expansion rate is used in each layer of dynamic expansion causal convolution to control the sampling step size. The expressions of the two layers of expansion causal convolution are: ;(8) ;(9) in, Indicates the current time step position; and Represent the first layer of convolution kernels respectively and the second layer of convolution kernel In the The weight of each position; and Represent the sizes of the first and second convolution kernels respectively; Dynamically dilated causal convolution extracts real-time vehicle speed from the target vehicle's historical trajectory feature sequence , calculated by learnable parameters , controls the sampling step of temporal convolution and realizes adaptive adjustment of the receptive field range. Specifically expressed as: ;(10) Among them, among them, Indicates the Layer dynamic dilation causal convolution; and Represents independent learnable parameters of each layer; Represents the truncation function, which is used to Limit to a preset minimum value and maximum value range, to prevent the receptive field from being too small or too large; The convolution trend perception feature matrix Perform linear projection and generate a query matrix by mapping the dimensions from the convolution feature space to the attention space, capturing the temporal dependency features and trend features in different subspaces. , key matrix , value matrix : ;(11) in, 、 and represents the learnable weight matrix; 、 、 Respectively The query matrix, key matrix and value matrix corresponding to the subspace; The multi-head attention mechanism learns features from different trajectory subspaces of different complexity and shares them at each time step, specifically based on the features generated by each group of multi-head convolutional maps. 、 and , calculate the attention weight of the target vehicle in each subspace, concatenate the outputs of all attention heads and linearly transform them to obtain the final trend perception time series features , to capture the global dependency in the time dimension, the expression is: ;(12) ;(13) in, It is Head self-attention output, is the number of attention heads; is the dimension of the value vector; Indicates a merge Fully connected layer of the head; Will and Obtained through residual connection and layer normalization , improve feature stability and expression ability, the expression is: ;(14) Through the feedforward neural network FNN, nonlinear mapping is introduced to enhance feature expression; FNN consists of two fully connected layers, connected by GELU activation function and Dropout in the middle, and the expression is: ;(15) in, is the output of the FNN layer; and are the weight matrices of the first and second fully connected layers of the feedforward network respectively; and are the bias vectors of the first and second fully connected layers of the feedforward network respectively; Perform residual connection and normalization before output, and finally output the target vehicle motion characteristics. , the expression is: ; (16) The historical trajectory sequence of surrounding vehicles is input into the gated recurrent unit GRU to extract the motion features of surrounding vehicles, and the motion features of each vehicle are obtained. The final hidden state at the moment , the expression is: ; (17) in, Indicates at time Surrounding vehicles The hidden feature vector of the surrounding vehicle is finally output. .

[0029] Step 23: Establish a dynamic spatiotemporal interaction graph. Take the target vehicle and surrounding vehicles as nodes in the graph. Dynamically construct an adjacency matrix through a learnable threshold to determine the edge relationship between vehicles. On this basis, use the dynamic graph attention mechanism to extract spatiotemporal interaction features, such as Figure 3 As shown; The dynamic spatiotemporal interaction graph introduces a dynamic learnable threshold prediction mechanism, which dynamically predicts the neighbor selection threshold based on the speed, acceleration, local traffic density and historical time series characteristics of the target vehicle at each time step. , construct a spatial position graph structure based on the vehicles within this threshold range , specifically: For target vehicle At time step Speed , acceleration , local traffic density and motion time series feature vector , jointly form the scene feature vector , input to the multi-layer perceptron MLP to predict the dynamic neighbor selection threshold , the expression is: ; (18) in, 、 、 、 is a learnable parameter, Ensure that the output is positive; The predicted value is linearly adjusted by introducing the minimum safety threshold and scaling factor to obtain the set threshold , to ensure that the model stably selects a reasonable range of neighbors for spatial interaction modeling under different traffic density and speed scenarios. The expression is: ; (19) in, is the minimum safety threshold, is the scaling factor; Determine target vehicle With surrounding vehicles The Euclidean distance between Less than the set threshold , based on which the time step is constructed The spatial adjacency matrix of , the expression is: ; (20) Build dynamic spatial interaction diagrams , each car in the scene is regarded as a set of nodes in the graph , each node corresponds to the motion characteristics of the target vehicle and surrounding vehicles in step 22, based on the time step Adjacency matrix obtained by learning threshold screening , will satisfy Node pairs As edges, formed at time step The edge set , the expression is: ;(twenty one) The dynamic graph attention network uses a multi-head attention mechanism, where each attention head uses attention weights to At time step Surrounding vehicles The weighted sum of the features of the nodes is performed to achieve the effective fusion of vehicle dynamic interaction information and the modeling of spatial dependency, and the output results of all heads are averaged in the feature dimension to obtain the node at the current time step. Updated feature representation , the specific calculation expression is as follows: ;(twenty two) in, is the number of attention heads; and Indicates the The attention weights and weight matrices of each head; is a nonlinear activation function GELU; the attention weight is measured under the constraints of the dynamic adjacency matrix. Moment target vehicles in the attention heads With surrounding vehicles The quantitative indicator of interaction strength is expressed as follows: ;(twenty three) Where, express Moment Target vehicle node in the attention head Neighboring vehicle nodes The attention weight of ; ∥ represents the vector concatenation operation; Indicates the The learnable attention vector of the attention head, T represents the transpose; LeakyReLU represents the nonlinear activation function; represents the spatial neighbor index, Indicates that the target vehicle A set of vehicles with spatial adjacency; By observing the time window The above updates are performed for each time step and stacked in time order to obtain the spatiotemporal interaction feature sequence of the target vehicle in the observation sequence .

[0030] Step 24: Align the target vehicle motion features and the spatiotemporal interaction features in the time dimension and then perform splicing and fusion to obtain spatiotemporal fusion features; The target vehicle motion characteristics Interaction characteristics with time and space After the time dimension is aligned, splicing and fusion are performed to obtain the spatiotemporal fusion features , the expression is: ;(twenty four) The spliced ​​spatiotemporal fusion features are input into the The multi-layer perceptron MLP is used, and the GELU nonlinear activation function is used to enhance the representation ability, obtain the feature representation of the dimension required for decoding, and standardize the feature distribution through layer normalization to obtain the sequence representation for subsequent trajectory prediction decoding , the expression is: (25).

[0031] Step 25: Input the fused features into the GRU, and after decoding, input them into the mixture density network to learn the potential trajectory distribution from the spatiotemporal features and realize the multimodal output of the vehicle trajectory.

[0032] The trajectory output module includes GRU and mixture density network MDN; GRU will fuse the feature sequence As input, the initial hidden state of the trajectory output module Initialized by the last moment state of the fusion feature, the expression is: ; (26) Progressively decode future time steps through GRU The hidden state of is expressed as: ; (27) Mixed density networks will As input, it is used to predict the multimodal future trajectory distribution and output the multimodal Gaussian distribution parameters, including the mean of each mixture component , covariance matrix and blend weights , the expression is: ; (28) in, Indicates the number of preset Gaussian mixture prediction trajectory distributions; Indicates the The predicted mean position of the predicted trajectory; Indicates the The prediction covariance matrix of the predicted trajectory; Indicates the The mixed weight of the predicted trajectory defines the possibility of being sampled as the final trajectory, satisfying ; Based on the mean , covariance matrix and blend weights , for each future time step The trajectory position , output its trajectory distribution probability, the expression is: ;(29) in, represents the probability of trajectory distribution; represents a multivariate Gaussian distribution.

[0033] Step 3: Improve the prediction accuracy and convergence speed of the model by selecting a suitable loss function for training.

[0034] To ensure accurate multimodal trajectory prediction, the loss function The negative log-likelihood is used as the regression loss, which measures the distribution of the predicted trajectory. The difference between the true trajectory and the real trajectory ensures accurate multimodal trajectory prediction, which is expressed as: ; (30) in, Indicates the length of the predicted future time step; Indicates the target vehicle at the predicted time The real location.

[0035] This paper trains a model using US-101 and I-80 vehicle trajectory data from the public NGSIM dataset. This dataset records vehicle position, velocity, acceleration, and other state information in real-world highway scenarios at a 10Hz frequency. Each trajectory is segmented into fixed time windows, each spanning 8 seconds. The first 3 seconds serve as the input sequence for model feature learning, while the last 5 seconds serve as the prediction target. The dataset is divided into training, validation, and test sets in a 6:2:2 ratio to improve model generalization.

[0036] The proposed spatiotemporal attention-based multimodal vehicle trajectory prediction model was trained and evaluated using the Pytorch deep learning framework. The training iterations were 100, the batch size was 64, and the Adam optimizer was used as the model optimizer with an initial learning rate of 0.001. Figure 4 The graph below shows the change in the loss function of the model on the training and validation sets. The figure shows that the loss function value of the training set continues to converge with the number of iterations, eventually stabilizing around 0.026. The overall trend of the loss function value of the validation set is consistent with that of the training set, also showing a continuous downward and convergent trend. The difference between the loss curves of the training and validation sets remains small, indicating that the model has good fitting ability and strong generalization performance.

[0037] This embodiment of the present invention considers both a unimodal prediction (k=1) model and a multimodal prediction (k=6) model. During the prediction process, 3 seconds of historical data are used to predict the trajectory for the next 5 seconds. Evaluation metrics widely used in the field of trajectory prediction were selected, including minimum average displacement error (minADE), minimum final displacement error (minFDE), and miss rate (MR), which are expressed as: minADE represents the average Euclidean distance between the predicted trajectory and the true trajectory at all time steps, and the model output is selected. The one with the smallest error among the predicted trajectories is selected, and the mean error of all time steps is calculated as shown in the following formula: ; (31) in, Indicates the number of prediction time steps; represents the number of trajectories for multimodal prediction; Indicates the The predicted trajectory at time The predicted position coordinates of time The real position coordinates of represents the Euclidean distance (L2 norm) between the predicted trajectory and the true trajectory.

[0038] minFDE means selecting K predicted trajectories at the final moment The one closest to the true trajectory, calculate its The displacement error at the moment is shown as follows: ; (32) in, Indicates the The predicted trajectory at the final prediction time location; The true trajectory is The location at the moment.

[0039] The miss rate indicates the proportion of prediction failures when the distance between the predicted trajectory and all true trajectories is greater than a certain threshold. In this invention, the threshold is set to 2.0m.

[0040] In order to analyze the impact of different numbers of multimodal trajectories and different prediction durations on trajectory evaluation indicators, a study and analysis was conducted on the test set. Figure 5 The results show the impact of different numbers of multimodal trajectories on various evaluation indicators when predicting the trajectory 5 seconds into the future. It can be seen that as the number of multimodal trajectories increases, each evaluation indicator shows better performance, indicating that the prediction accuracy of the vehicle trajectory prediction method proposed in this invention increases with the number of multimodal trajectories. Figure 6 and Figure 7The influence of different prediction durations on the evaluation indicators under single-mode and multi-modal conditions is shown. It can be seen that the vehicle trajectory prediction method proposed in this invention shows high prediction accuracy in the first 3 seconds for both single-mode and multi-modal predictions, and shows better results in multi-modal predictions. At the 4th and 5th seconds, the upward trend of its evaluation indicators slows down, indicating that this research model is suitable for long-term prediction and can ensure long-term prediction accuracy to a certain extent.

[0041] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the scope of protection of the present invention.

Claims

1. A vehicle multimodal trajectory prediction method based on an improved attention network, characterized in that: The following steps are involved: Step 1: Collect historical trajectory data of the target vehicle and surrounding vehicles in the target scene as input sequence; Step 2: Construct a vehicle multimodal trajectory prediction model, including a motion feature extraction module, a spatiotemporal interaction module, a spatiotemporal fusion module, and a trajectory output module; The motion feature extraction module divides the historical states of the target vehicle and its surrounding vehicles into two parts, which are processed using a multi-scale convolutional attention network and a gated recurrent unit respectively. The spatiotemporal interaction module uses a learnable threshold to construct a dynamic graph mechanism and extracts vehicle interaction information through the dynamic graph attention network. The spatiotemporal fusion module combines the target vehicle motion features with the spatiotemporal interaction features to obtain spatiotemporal fusion features. The trajectory output module inputs the fused features into the gated recurrent unit, which is then decoded and input into the mixture density network to achieve multimodal output of the vehicle trajectory. Step 3: Select a loss function for training to improve the prediction accuracy and convergence speed of the model.

2. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 1 is characterized in that: In step 1, the historical trajectory data includes the target vehicle and its surrounding vehicles in the time step The historical state within is expressed as: ；(1) in, Indicates selection The vehicle is the target vehicle. Represents the historical trajectory data of the target vehicle; Indicates the surrounding vehicle index, , is the number of surrounding vehicles; Represents historical trajectory data of surrounding vehicles; and Represent the target vehicle and surrounding vehicles respectively The vehicle status at the moment, including the vehicle and Six-dimensional features of position, velocity and acceleration in direction; The vehicle's historical trajectory data is input into a pre-built vehicle multimodal trajectory prediction model based on an improved attention network to perform motion feature extraction and dynamic interaction modeling operations, and extract the high-dimensional spatiotemporal fusion features of the target vehicle for multimodal trajectory prediction.

3. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 1 is characterized in that: The step 2 includes the following sub-steps: Step 21: Convert the historical trajectory sequence of the target vehicle and surrounding vehicles into a high-dimensional vector through an embedding layer; Step 22: Input the target vehicle's historical trajectory sequence into a multi-scale convolutional attention network to capture temporal features and process local and global trends in the target vehicle's time series. Input the surrounding vehicle's historical trajectory sequence into a gated recurrent unit (GRU) to process the surrounding vehicle's historical time series input features. Step 23: Build a dynamic spatiotemporal interaction graph, using the target vehicle and surrounding vehicles as nodes in the graph. Dynamically construct an adjacency matrix using a learnable threshold to determine the edge relationships between vehicles. Based on this, use the dynamic graph attention mechanism to extract spatiotemporal interaction features. Step 24: Align the target vehicle motion features and the spatiotemporal interaction features in the time dimension and then perform splicing and fusion to obtain spatiotemporal fusion features; Step 25: Input the fused features into the GRU, and after decoding, input them into the mixture density network to learn the potential trajectory distribution from the spatiotemporal features and realize the multimodal output of the vehicle trajectory.

4. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 3 is characterized in that: In step 21, the embedding layer converts the original six-dimensional information of the scene vehicle into a high-dimensional vector through linear transformation and activation function, which is expressed as: ； (2) in, 、 Respectively represent the motion state embedding features of the target vehicle and surrounding vehicles at time step t; represents the weight matrix of the embedding layer; represents a linear layer; GELU represents a Gaussian error linear unit.

5. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 3 is characterized in that: In step 22, the multi-scale convolutional attention network first encodes the target vehicle position vector Add to the embedding vector after linear transformation and activation function to get the embedding vector after position encoding ,in, Use sine and cosine functions of different frequencies to encode the temporal position of each time step; The multi-scale convolutional attention network utilizes a multi-head convolutional mapping mechanism and a multi-head attention mechanism to jointly capture the local behavior trend characteristics and global time dependencies of the target vehicle trajectory in different time intervals; The multi-head convolution mapping mechanism adopts an improved TCN network, which takes the historical trajectory feature sequence of the target vehicle after position encoding as input, and outputs the sequence through two layers of stacked dynamic expansion causal convolution. , weight normalization and GELU convolution activation are performed after each layer of dynamically expanded causal convolution. Layer normalization is performed after the GELU convolution activation of the first layer to enhance network stability and expressiveness, and residual connections are added to the GELU activation of the second layer to alleviate the gradient vanishing problem; The dynamic convolution expansion rate is used in each layer of dynamic expansion causal convolution to control the sampling step size. The expressions of the two layers of expansion causal convolution are: ；(8) ；(9) in, Indicates the current time step position; and Represent the first layer of convolution kernels respectively and the second layer of convolution kernel In the The weight of each position; and Represent the sizes of the first and second convolution kernels respectively; The dynamic expansion causal convolution extracts the real-time vehicle speed from the target vehicle's historical trajectory feature sequence. , calculate the dynamic convolution expansion rate of each layer through learnable parameters , specifically expressed as: ；(10) in, Indicates the Layer dynamic dilation causal convolution; and Represents independent learnable parameters of each layer; Represents the truncation function, which is used to Limit to a preset minimum value and maximum value within the scope; The convolution trend perception feature matrix Perform linear projection and generate a query matrix by mapping the dimensions from the convolution feature space to the attention space, capturing the temporal dependency features and trend features in different subspaces. , key matrix , value matrix The multi-head attention mechanism is specifically based on each group of multi-head convolutional mappings generated 、 and , calculate the attention weight of the target vehicle in each subspace, concatenate the outputs of all attention heads and linearly transform them to obtain the final trend perception time series features ; The resulting multi-head attention output and convolution output features After performing residual connection and layer normalization, we get , Enter the feedforward neural network FNN and connect it again with residual connection, and the final output is the target vehicle motion feature ; The historical trajectory sequence of surrounding vehicles is input into the gated recurrent unit GRU to extract the motion features of surrounding vehicles, and the motion features of each vehicle are obtained. The final hidden state at the moment , and finally output the motion characteristics of each surrounding vehicle .

6. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 3 is characterized in that: The step 23 is specifically as follows: At time step Speed , acceleration , local traffic density and motion time series feature vector , jointly form the scene feature vector , input to the multi-layer perceptron MLP to predict the dynamic neighbor selection threshold , the expression is: ；(18) in, 、 、 、 is a learnable parameter, Ensure that the output is positive; The predicted value is linearly adjusted by introducing the minimum safety threshold and scaling factor to obtain the set threshold , determine the target vehicle With surrounding vehicles The Euclidean distance between Less than the set threshold , based on which the time step is constructed The spatial adjacency matrix of , the expression is: ；(20) Build dynamic spatial interaction diagrams , each car in the scene is regarded as a set of nodes in the graph , each node corresponds to the motion characteristics of the target vehicle and surrounding vehicles in step 22, based on the time step Adjacency matrix obtained by learning threshold screening , will satisfy Node pairs As edges, formed at time step The edge set ; The dynamic graph attention network uses a multi-head attention mechanism, where each attention head uses attention weights to At time step Surrounding vehicles The weighted sum of the features of the nodes is performed to achieve the effective fusion of vehicle dynamic interaction information and the modeling of spatial dependency, and the output results of all heads are averaged in the feature dimension to obtain the node at the current time step. Updated feature representation , the specific calculation expression is as follows: ；(22) in, is the number of attention heads; and Indicates the The attention weights and weight matrices of each head; is a nonlinear activation function GELU; the attention weight is measured under the constraints of the dynamic adjacency matrix. Moment target vehicles in the attention heads With surrounding vehicles The quantitative indicator of interaction strength is expressed as follows: ；(23) Where, express Moment Target vehicle node in the attention head Neighboring vehicle nodes The attention weight of ; ∥ represents the vector concatenation operation; Indicates the The learnable attention vector of the attention head, T represents the transpose; LeakyReLU represents the nonlinear activation function; represents the spatial neighbor index, Indicates that the target vehicle A set of vehicles with spatial adjacency; By observing the time window The above updates are performed for each time step and stacked in time order to obtain the spatiotemporal interaction feature sequence of the target vehicle in the observation sequence .

7. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 3 is characterized in that: In step 24, the target vehicle motion characteristics Interaction characteristics with time and space After the time dimension is aligned, splicing and fusion are performed to obtain the spatiotemporal fusion features , the expression is: ； (24) The spliced ​​spatiotemporal fusion features are input into the The multi-layer perceptron MLP is used, and the GELU nonlinear activation function is used to enhance the representation ability, obtain the feature representation of the dimension required for decoding, and standardize the feature distribution through layer normalization to obtain the sequence representation for subsequent trajectory prediction decoding , the expression is: (25)。 8. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 3 is characterized in that: In step 25, the trajectory output module includes GRU and mixture density network; The GRU will fuse the feature sequence As input, the initial hidden state of the trajectory output module Initialized by the last moment state of the fused features; Progressively decode future time steps through GRU The hidden state of is expressed as: ；(27) The mixture density network will As input, it is used to predict the multimodal future trajectory distribution and output the multimodal Gaussian distribution parameters, including the mean of each mixture component , covariance matrix and blend weights , the expression is: ； (28) in, Indicates the number of preset Gaussian mixture prediction trajectory distributions; Indicates the The predicted mean position of the predicted trajectory; Indicates the The prediction covariance matrix of the predicted trajectory; Indicates the The mixed weight of the predicted trajectory defines the possibility of being sampled as the final trajectory, satisfying ; Based on the mean , covariance matrix and blend weights , for each future time step The trajectory position , output its trajectory distribution probability, the expression is: ；(29) in, represents the probability of trajectory distribution; represents a multivariate Gaussian distribution.

9. The vehicle multimodal trajectory prediction method based on improved attention network according to claim 8 is characterized in that: In step 3, the loss function The negative log-likelihood is used as the regression loss, which measures the distribution of the predicted trajectory. The difference between the actual trajectory and the actual trajectory is expressed as: ；(30) in, Indicates the length of the predicted future time step; represents the actual position of the target vehicle at the prediction time t.

Images