A vehicle multi-modal trajectory prediction method based on an improved attention network

Abstract

The application discloses a kind of vehicle multimodal trajectory prediction methods based on improved attention network, belong to intelligent vehicle trajectory prediction technical field, steps are as follows: collect the historical trajectory data of target vehicle and surrounding vehicle as input sequence;Secondly, a vehicle multimodal trajectory prediction model is built, including motion feature extraction module, space-time interaction module and space-time fusion module and trajectory output module;Motion feature extraction module is handled using a multiscale convolution attention network and gated recurrent unit, space-time interaction module uses a dynamic graph attention network to extract vehicle interaction information, space-time fusion module is spliced and fused to obtain space-time fusion feature with the motion feature of target vehicle and space-time interaction feature, trajectory output module is input into gated recurrent unit with fusion feature, after decoding, it is input into hybrid density network, realize the multimodal output of vehicle trajectory;Finally, through selecting appropriate loss function, improve the prediction accuracy and convergence speed of model.

Description

Technical Field

[0001] This invention belongs to the field of intelligent vehicle trajectory prediction technology, specifically relating to a method for predicting multimodal vehicle trajectories based on an improved attention network. Background Technology

[0002] With the rapid development of autonomous driving technology, vehicle trajectory prediction, as a core component of environmental perception and decision-making, directly impacts the safety and reliability of autonomous driving systems. In complex and dynamic traffic scenarios, the future trajectory of a target vehicle is not only influenced by its own motion patterns but also closely related to the interactive behavior of surrounding vehicles. Existing trajectory prediction methods are mainly divided into two categories: traditional methods based on physical models and data-driven deep learning methods. Traditional methods (such as Kalman filtering and Markov chains) rely on vehicle kinematic models to predict future positions by extrapolating historical trajectories. However, such methods struggle to construct complex vehicle interaction relationships, especially in scenarios with dense traffic and variable intentions, where prediction errors are significant. Data-driven methods (such as LSTM and CNN) improve prediction capabilities by extracting spatiotemporal features of the trajectory, but still suffer from the following bottlenecks:

[0003] (1) Feature confusion and interference: Traditional vehicle trajectory prediction methods usually mix the historical trajectories of the target vehicle and the surrounding vehicles and input them into the same feature extraction module, ignoring the essential differences between the two in motion patterns, temporal dependence strength and contribution to the prediction results.

[0004] (2) Limitations of temporal feature extraction: Existing vehicle trajectory prediction methods usually use fixed-scale convolutional networks or recurrent neural networks to extract the temporal features of vehicle historical trajectories. It is difficult to capture the local details of vehicle changes and global behavioral trends, resulting in insufficient perception of trajectory evolution patterns in dynamic target scenarios.

[0005] (3) Staticization of interaction modeling: Existing spatiotemporal interaction modules mostly use fixed Euclidean distance thresholds (such as <30m) to construct adjacency matrices, which makes it impossible to dynamically perceive changes in the influence of surrounding vehicles in high-speed scenarios. At the same time, traditional graph attention mechanisms ignore the spatiotemporal differences in interaction intensity, making the model susceptible to interference from vehicles with ineffective or weak interactions, increasing computational burden and potentially reducing prediction accuracy. Summary of the Invention

[0006] To address the aforementioned problems in existing technologies, this invention proposes a vehicle multimodal trajectory prediction method based on an improved attention network. The method is rationally designed, overcomes the shortcomings of existing technologies, and achieves good results.

[0007] A method for predicting vehicle multimodal trajectories based on an improved attention network includes the following steps:

[0008] Step 1: Collect historical trajectory data of the target vehicle and surrounding vehicles in the target scene as the input sequence;

[0009] Step 2: Construct a multimodal trajectory prediction model for the vehicle, including a motion feature extraction module, a spatiotemporal interaction module, a spatiotemporal fusion module, and a trajectory output module;

[0010] The motion feature extraction module divides the historical state of the target vehicle and its surrounding vehicles into two parts, which are processed by a multi-scale convolutional attention network and a gated recurrent unit, respectively. The spatiotemporal interaction module uses a learnable threshold to construct a dynamic graph mechanism and extracts vehicle interaction information through a dynamic graph attention network. The spatiotemporal fusion module concatenates and fuses the motion features of the target vehicle with the spatiotemporal interaction features to obtain spatiotemporal fusion features. The trajectory output module inputs the fusion features into the gated recurrent unit, decodes them, and then inputs them into a hybrid density network to achieve multimodal output of the vehicle trajectory.

[0011] Step 3: Select a loss function for training to improve the model's prediction accuracy and convergence speed.

[0012] Furthermore, in step 1, the historical trajectory data includes the target vehicle and surrounding vehicles at a time step. The historical state within is expressed as:

[0013] (1)

[0014] in, Indicates the selection of the first The vehicle is designated as the target vehicle. This represents the target vehicle's historical trajectory data; Indicates the index of surrounding vehicles. , The number of vehicles in the surrounding area; This indicates historical trajectory data of surrounding vehicles; and Representing the target vehicle and surrounding vehicles respectively. The vehicle status at any given time, including the vehicle's position in... and Six-dimensional features including position, velocity, and acceleration in the direction;

[0015] Historical vehicle trajectory data is input into a pre-built multimodal trajectory prediction model based on an improved attention network. Motion feature extraction and dynamic interaction modeling operations are performed to extract high-dimensional spatiotemporal fusion features of the target vehicle for multimodal trajectory prediction.

[0016] Furthermore, step 2 includes the following sub-steps:

[0017] Step 21: Convert the sequence of historical trajectories of the target vehicle and surrounding vehicles into high-dimensional vectors through an embedding layer;

[0018] Step 22: Input the target vehicle's historical trajectory sequence into a multi-scale convolutional attention network to capture temporal features and process the local and global trends in the target vehicle's time series. Input the surrounding vehicles' historical trajectory sequences into a gated recurrent unit (GRU) to process the historical time series input features of the surrounding vehicles.

[0019] Step 23: Establish a dynamic spatiotemporal interaction graph, taking the target vehicle and surrounding vehicles as nodes in the graph, dynamically constructing an adjacency matrix through a learnable threshold to determine the edge relationships between vehicles, and then using a dynamic graph attention mechanism to extract spatiotemporal interaction features.

[0020] Step 24: Align the target vehicle motion features and spatiotemporal interaction features in the time dimension and then stitch and fuse them to obtain spatiotemporal fused features;

[0021] Step 25: Input the fused features into the GRU, decode them, and then input them into the hybrid density network to learn the potential trajectory distribution from the spatiotemporal features, thereby achieving multimodal output of the vehicle trajectory.

[0022] Further, in step 21, the embedding layer transforms the original six-dimensional information of the scene vehicle into a high-dimensional vector through linear transformation and activation function, as expressed by:

[0023] (2)

[0024] in, , These represent the motion state embedding features of the target vehicle and surrounding vehicles at time step t, respectively. This represents the weight matrix of the embedding layer; represents a linear layer; GELU represents a Gaussian error linear unit.

[0025] Furthermore, in step 22, the multi-scale convolutional attention network first encodes the target vehicle position vector. The position-encoded embedding vector is obtained by adding it to the embedding vector after linear transformation and activation function. ,in, Use sine and cosine functions of different frequencies to encode the time position for each time step;

[0026] The multi-scale convolutional attention network utilizes a multi-head convolutional mapping mechanism and a multi-head attention mechanism to jointly capture the local behavioral trend features and global temporal dependencies of the target vehicle trajectory in different time intervals;

[0027] The multi-head convolutional mapping mechanism employs an improved TCN network, taking the historical trajectory feature sequence of the target vehicle after location encoding as input, and outputting the sequence through two stacked layers of dynamically dilated causal convolutions. After each layer of dynamically dilated causal convolution, weight normalization and GELU convolution activation are performed. After the GELU convolution activation in the first layer, layer normalization is used to enhance the stability and expressive power of the network. Residual connections are added to the GELU activation in the second layer to alleviate the gradient vanishing problem.

[0028] In each layer of dynamically dilated causal convolution, the dynamic dilation rate controls the sampling stride. The expressions for the two layers of dilated causal convolution are as follows:

[0029] (8)

[0030] (9)

[0031] in, Indicates the current time step position; and These represent the first layer convolution kernels respectively. Second layer convolution kernel In the Weights at each position; and These represent the sizes of the first and second convolutional kernels, respectively.

[0032] The dynamically dilated causal convolution extracts the real-time vehicle speed from the historical trajectory feature sequence of the target vehicle. The dynamic convolutional dilation rate of each layer is calculated using learnable parameters. Specifically, it is expressed as:

[0033] (10)

[0034] in, Indicates the first Layered dynamic dilated causal convolution; and This represents the independently learnable parameters of each layer; This represents a truncation function used to truncate... Limit to a preset minimum value and maximum value Within the range;

[0035] The convolutional trend-aware feature matrix Linear projection is performed to capture temporal dependency and trend features in different subspaces by mapping the dimensions from the convolutional feature space to the attention space, thereby generating a query matrix. Key matrix Value matrix The multi-head attention mechanism is specifically generated based on each group of multi-head convolutional mappings. , and The attention weight of the target vehicle in each subspace is calculated, and the outputs of all attention heads are concatenated and linearly transformed to obtain the final trend-aware temporal features. ;

[0036] Multi-head attention output With convolution output features After performing residual connections and layer normalization, we obtain , The data is fed into a feedforward neural network (FNN) and then reconnected using residuals. The final output is the motion features of the target vehicle. ;

[0037] The historical trajectory sequences of surrounding vehicles are input and the motion features of the surrounding vehicles are extracted through a gated recurrent unit (GRU) to obtain the motion features of each vehicle. The final hidden state of time Finally, the motion characteristics of each surrounding vehicle are output. .

[0038] Furthermore, step 23 specifically involves: for the target vehicle... At time step speed acceleration Local traffic density and motion time sequence feature vector Together, they form a scene feature vector. The input is fed into the multilayer perceptron (MLP) to predict the dynamic neighbor selection threshold. The expression is:

[0039] (18)

[0040] in, , , , For learnable parameters, Ensure the output is positive;

[0041] The predicted value is linearly adjusted by introducing a minimum safety threshold and a scaling factor to obtain the set threshold. Determine the target vehicle With surrounding vehicles Euclidean distance between Less than the set threshold Based on this, a time step is constructed. Spatial adjacency matrix The expression is:

[0042] (20)

[0043] Constructing dynamic spatial interaction graphs Each car in the scene is treated as a set of nodes in a graph. Each node corresponds to the motion characteristics of the target vehicle and surrounding vehicles in step 22, based on the time step. Adjacency matrix obtained by learnable threshold filtering , will satisfy node pairs As an edge, it forms in time step edge set ;

[0044] Dynamic graph attention networks use a multi-head attention mechanism, where each attention head focuses on the target vehicle through attention weights. At time step surrounding vehicles The features are weighted and summed to achieve effective fusion of vehicle dynamic interaction information and modeling of spatial dependencies. The outputs of all heads are averaged along the feature dimension to obtain the node's position at the current time step. Updated feature representation The specific calculation expression is as follows:

[0045] ;(twenty two)

[0046] in, It's about the number of heads; and Indicates the first Attention weights and weight matrices for each size; It is a non-linear activation function GELU; the attention weights are measured under the constraint of a dynamic adjacency matrix. Time of the first The target vehicle was under everyone's attention. With surrounding vehicles The quantitative indicator of interaction strength is expressed as follows:

[0047] ;(twenty three)

[0048] In the formula, express Time of the first Attention focus on the target vehicle node For neighboring vehicle nodes Attention weights; || denotes vector concatenation operation; Indicates the first The learnable attention vectors of attention heads, where T represents the transpose; LeakyReLU represents the non-linear activation function; Represents the spatial neighbor index. Indicates the target vehicle A set of vehicles that are spatially adjacent;

[0049] By observation time window The above updates are performed at each time step, and the results are stacked in chronological order to obtain the spatiotemporal interaction feature sequence of the target vehicle within the observation sequence. .

[0050] Furthermore, in step 24, the motion characteristics of the target vehicle are... Spatiotemporal interaction features After aligning along the time dimension, the data is stitched and fused to obtain spatiotemporal fusion features. The expression is:

[0051] ;(twenty four)

[0052] The spatiotemporal fusion features are then input into a parameterized input. A multilayer perceptron (MLP) is constructed, and the GELU nonlinear activation function is employed to enhance representation capability, thereby obtaining the feature representation of the dimension required for decoding. Furthermore, the feature distribution is standardized through layer normalization to obtain the sequence representation used for subsequent trajectory prediction decoding. The expression is:

[0053] (25).

[0054] Furthermore, in step 25, the trajectory output module includes a GRU and a hybrid density network;

[0055] The GRU will fuse feature sequences. The initial hidden state of the trajectory output module as input. Initialized from the last moment state of the fused features;

[0056] Decoding future time steps step by step using GRU The hidden state is expressed as:

[0057] (27)

[0058] The hybrid density network will As input, it is used to predict the future trajectory distribution of the multimodal system, and the output is a multimodal Gaussian distribution parameter, including the mean of each mixture component. Covariance matrix and mixed weights The expression is:

[0059] (28)

[0060] in, This indicates the number of preset Gaussian mixture prediction trajectory distributions; Indicates the first The predicted mean position of the predicted trajectory; Indicates the first The prediction covariance matrix of the predicted trajectory; Indicates the first The mixed weights of the predicted trajectories define the probability of being sampled as the final trajectory, satisfying... ;

[0061] Based on mean Covariance matrix and mixed weights For every step of the future trajectory position Output its trajectory distribution probability, expressed as:

[0062] (29)

[0063] in, Indicates the probability of trajectory distribution; This represents a multivariate Gaussian distribution.

[0064] Furthermore, in step 3, the loss function Using negative log-likelihood as the regression loss, this loss function measures the distribution of the predicted trajectory. The difference between the actual trajectory and the true trajectory is expressed as:

[0065] (30)

[0066] in, Indicates the predicted length of the future time step; This represents the actual position of the target vehicle at the predicted time t.

[0067] The beneficial technical effects of this invention are as follows:

[0068] 1. This invention establishes a vehicle multimodal trajectory prediction model based on an improved attention network. By combining a dynamic graph attention mechanism with a multi-scale convolutional attention architecture, it can effectively solve the problems of inaccurate interactive vehicle recognition and long-term behavior prediction in different scenarios. At the same time, it can effectively improve the accuracy of vehicle trajectory prediction through multimodal prediction.

[0069] 2. By separating the historical trajectories of the target vehicle from those of surrounding vehicles, and employing multi-scale convolutional attention networks and gated recurrent units for temporal modeling respectively, the feature confusion problem caused by mixed inputs in traditional methods is resolved. The multi-scale convolutional attention network focuses on the local motion details and global trends of the target vehicle, while the gated recurrent unit efficiently captures the dynamic changes of surrounding vehicles, significantly improving the targeting and independence of feature extraction.

[0070] 3. A multi-scale convolutional attention network is designed. The bottom layer uses dynamic dilated causal convolution to extract multi-channel local features, and the top layer uses a multi-head attention mechanism to fuse global dependencies. Residual connections are used to stabilize deep training. This network can capture local behavioral changes and global trend information of trajectories at different time scales, making the extraction of temporal features for trajectory prediction more accurate.

[0071] 4. Dynamic graph attention, based on information such as vehicle speed, acceleration, and local traffic density, dynamically constructs a spatial adjacency matrix through learnable thresholds and combines it with a dynamic graph attention mechanism to model the interaction weights between vehicles, enabling accurate capture of dynamic interaction relationships between vehicles. This method overcomes the limitations of traditional static graph models, adaptively captures the interaction effects between vehicles in complex scenes, and significantly improves the accuracy of spatial relationship modeling. Attached Figure Description

[0072] Figure 1 This is a structural framework diagram of the vehicle trajectory prediction method of the present invention.

[0073] Figure 2 This is a diagram of the multi-scale convolutional attention network structure for the vehicle trajectory prediction method of this invention.

[0074] Figure 3 This is a schematic diagram of the spatiotemporal interaction module structure of the vehicle trajectory prediction method of the present invention.

[0075] Figure 4 This is a training stability verification diagram for the vehicle trajectory prediction method of the present invention.

[0076] Figure 5 This is a graph showing the impact of the number of multimodal trajectories on the evaluation index in this invention.

[0077] Figure 6 This is a graph showing the impact of the single-modal prediction duration on the evaluation index in this invention.

[0078] Figure 7 This is a graph showing the impact of the multimodal prediction duration on the evaluation index in this invention. Detailed Implementation

[0079] The specific embodiments of the present invention will be further described below with reference to specific examples:

[0080] A method for predicting vehicle multimodal trajectories based on an improved attention network, such as... Figure 1 As shown, it includes the following steps:

[0081] Step 1: Collect historical trajectory data of the target vehicle and surrounding vehicles in the target scene as the input sequence;

[0082] The vehicle trajectory prediction problem can be described as estimating the future trajectory information of a target vehicle using historical trajectory data of the target vehicle and its surrounding vehicles.

[0083] Historical trajectory data includes the target vehicle and surrounding vehicles at time steps. The historical state within is expressed as:

[0084] (1)

[0085] in, Indicates the selection of the first Each vehicle is designated as the target vehicle, and there is only one target vehicle in a given target scenario. This represents the target vehicle's historical trajectory data; Indicates the index of surrounding vehicles. , The number of vehicles in the surrounding area; This indicates historical trajectory data of surrounding vehicles; and Representing the target vehicle and surrounding vehicles respectively. The vehicle status at any given time, including the vehicle's position in... and Six-dimensional features including position, velocity, and acceleration in the direction;

[0086] Historical vehicle trajectory data is input into a pre-built multimodal trajectory prediction model based on an improved attention network. Motion feature extraction and dynamic interaction modeling operations are performed to extract high-dimensional spatiotemporal fusion features of the target vehicle for multimodal trajectory prediction.

[0087] Step 2: Construct a multimodal trajectory prediction model for the vehicle, including a motion feature extraction module, a spatiotemporal interaction module, a spatiotemporal fusion module, and a trajectory output module;

[0088] The motion feature extraction module divides the historical state of the target vehicle and its surrounding vehicles into two parts, which are processed by a multi-scale convolutional attention network and a gated recurrent unit, respectively. The spatiotemporal interaction module uses a learnable threshold to construct a dynamic graph mechanism and extracts vehicle interaction information through a dynamic graph attention network. The spatiotemporal fusion module concatenates and fuses the motion features of the target vehicle with the spatiotemporal interaction features to obtain spatiotemporal fusion features. The trajectory output module inputs the fusion features into the gated recurrent unit, decodes them, and then inputs them into a hybrid density network to achieve multimodal output of the vehicle trajectory.

[0089] Step 2 includes the following sub-steps:

[0090] Step 21: Convert the sequence of historical trajectories of the target vehicle and surrounding vehicles into high-dimensional vectors through an embedding layer;

[0091] The embedding layer transforms the original six-dimensional information of the vehicle in the scene into a high-dimensional vector through linear transformation and activation functions, as expressed in the following expression:

[0092] (2)

[0093] in, , These represent the motion state embedding features of the target vehicle and surrounding vehicles at time step t, respectively. This represents the weight matrix of the embedding layer; represents a linear layer; GELU represents a Gaussian error linear unit.

[0094] Step 22: Input the target vehicle's historical trajectory sequence into the encoding layer of a multi-scale convolutional attention network to capture temporal features and process the local and global trends in the target vehicle's time series. Input the surrounding vehicles' historical trajectory sequences into a gated recurrent unit (GRU) to process the historical time series input features of the surrounding vehicles.

[0095] like Figure 2 As shown, the encoding layer of the multi-scale convolutional attention network first encodes the target vehicle's position vector. The position-encoded embedding vector is obtained by adding it to the embedding vector after linear transformation and activation function. The expression is:

[0096] (3)

[0097] Time position encoding is performed for each time step using sine and cosine functions of different frequencies, as expressed in the following expression:

[0098] (4)

[0099] (5)

[0100] in, This indicates the location encoding dimension set in the model. Represent each dimension Location encoding vector;

[0101] Multi-scale convolutional attention networks utilize multi-head convolutional mapping and multi-head attention mechanisms to jointly capture the local behavioral trend features and global temporal dependencies of target vehicle trajectories in different time intervals;

[0102] The multi-head convolutional mapping mechanism employs an improved TCN network, taking the historical trajectory feature sequence of the target vehicle after location encoding as input, and replacing the traditional standard fully connected mapping output sequence with two stacked layers of dynamically dilated causal convolutions. After each layer of dynamically dilated causal convolution, WeightNorm normalization and GELU activation are applied. Layer normalization is used after the GELU activation in the first layer to enhance network stability and expressive power. Residual connections are added to the GELU activation in the second layer to alleviate the gradient vanishing problem. Specifically:

[0103] First layer of dynamically dilated causal convolution:

[0104] (6)

[0105] Second layer of dynamically dilated causal convolution:

[0106] (7)

[0107] in, This represents the output of the first layer of dynamically dilated causal convolution. This represents the trend-aware feature matrix output by the second-layer dynamically dilated causal convolution. and These represent the expansion factors of the first and second layers, respectively. and These represent the convolution kernels of the first and second layers, respectively.

[0108] In each layer of dynamically dilated causal convolution, the dynamic dilation rate controls the sampling stride. The expressions for the two layers of dilated causal convolution are as follows:

[0109] (8)

[0110] (9)

[0111] in, Indicates the current time step position; and These represent the first layer convolution kernels respectively. Second layer convolution kernel In the Weights at each position; and These represent the sizes of the first and second convolutional kernels, respectively.

[0112] Dynamically dilated causal convolution extracts real-time vehicle speed from the historical trajectory feature sequence of the target vehicle. Calculated through learnable parameters This controls the sampling stride of the temporal convolution, enabling adaptive adjustment of the receptive field. Specifically, it is expressed as:

[0113] (10)

[0114] Among them, among them, Indicates the first Layered dynamic dilated causal convolution; and This represents the independently learnable parameters of each layer; This represents a truncation function used to truncate... Limit to a preset minimum value and maximum value Within the range, prevent the receptive field from being too small or too large;

[0115] The convolutional trend-aware feature matrix Linear projection is performed to capture temporal dependency and trend features in different subspaces by mapping the dimensions from the convolutional feature space to the attention space, thereby generating a query matrix. Key matrix Value matrix :

[0116] (11)

[0117] in, , and Represents the learnable weight matrix; , , The first Each subspace corresponds to a query matrix, a key matrix, and a value matrix;

[0118] The multi-head attention mechanism learns features from different trajectory subspaces of varying complexity and shares them at each time step, specifically based on the multi-head convolutional mappings generated for each group. , and The attention weight of the target vehicle in each subspace is calculated, and the outputs of all attention heads are concatenated and linearly transformed to obtain the final trend-aware temporal features. To capture global dependencies over time, the expression is:

[0119] (12)

[0120] (13)

[0121] in, It is the first Head self-attention output, For the number of attention heads; The dimension of the value vector; Indicates merger A fully connected layer of considerable size;

[0122] Will and We obtain the results through residual connection and layer normalization. To improve feature stability and expressive power, the expression is:

[0123] (14)

[0124] A feedforward neural network (FNN) is used to introduce nonlinear mapping to enhance feature representation. The FNN consists of two fully connected layers connected by the GELU activation function and Dropout, as shown in the expression:

[0125] (15)

[0126] in, For the output of the FNN layer; and These are the weight matrices for the first and second fully connected layers of the feedforward network, respectively. and These are the bias vectors for the first and second fully connected layers of the feedforward network, respectively.

[0127] Residual connections and normalization are performed before output, ultimately outputting the target vehicle motion features. The expression is:

[0128] (16)

[0129] The historical trajectory sequences of surrounding vehicles are input and the motion features of the surrounding vehicles are extracted through a gated recurrent unit (GRU) to obtain the motion features of each vehicle. The final hidden state of time The expression is:

[0130] (17)

[0131] in, Indicates at time surrounding vehicles The hidden feature vectors are used to output the motion features of each surrounding vehicle. .

[0132] Step 23: Establish a dynamic spatiotemporal interaction graph, treating the target vehicle and surrounding vehicles as nodes in the graph. Dynamically construct an adjacency matrix using learnable thresholds to determine the edge relationships between vehicles. Based on this, extract spatiotemporal interaction features using a dynamic graph attention mechanism, such as... Figure 3 As shown;

[0133] The dynamic spatiotemporal interaction graph introduces a dynamic learnable threshold prediction mechanism, which dynamically predicts the neighbor selection threshold based on the target vehicle's velocity, acceleration, local traffic density, and historical time series characteristics at each time step. Based on this threshold range, a spatial location map structure is constructed. Specifically:

[0134] For target vehicle At time step speed acceleration Local traffic density and motion time sequence feature vector Together, they form a scene feature vector. The input is fed into the multilayer perceptron (MLP) to predict the dynamic neighbor selection threshold. The expression is:

[0135] (18)

[0136] in, , , , For learnable parameters, Ensure the output is positive;

[0137] The predicted value is linearly adjusted by introducing a minimum safety threshold and a scaling factor to obtain the set threshold. To ensure the model stably selects a reasonable range of neighbors for spatial interaction modeling under different traffic density and speed scenarios, the expression is:

[0138] (19)

[0139] in, As the minimum safety threshold, This is the scaling factor;

[0140] Identify the target vehicle With surrounding vehicles Euclidean distance between Less than the set threshold Based on this, a time step is constructed. Spatial adjacency matrix The expression is:

[0141] (20)

[0142] Constructing dynamic spatial interaction graphs Each car in the scene is treated as a set of nodes in a graph. Each node corresponds to the motion features of the target vehicle and surrounding vehicles in step 22, based on the time step. Adjacency matrix obtained by learnable threshold filtering , will satisfy node pairs As an edge, it forms in time step edge set The expression is:

[0143] ;(twenty one)

[0144] Dynamic graph attention networks use a multi-head attention mechanism, where each attention head focuses on the target vehicle through attention weights. At time step surrounding vehicles The features are weighted and summed to achieve effective fusion of vehicle dynamic interaction information and modeling of spatial dependencies. The outputs of all heads are averaged along the feature dimension to obtain the node's position at the current time step. Updated feature representation The specific calculation expression is as follows:

[0145] ;(twenty two)

[0146] in, It's about the number of heads; and Indicates the first Attention weights and weight matrices for each size; It is a non-linear activation function GELU; the attention weights are measured under the constraint of a dynamic adjacency matrix. Time of the first The target vehicle was under everyone's attention. With surrounding vehicles The quantitative indicator of interaction strength is expressed as follows:

[0147] ;(twenty three)

[0148] In the formula, express Time of the first Attention focus on the target vehicle node For neighboring vehicle nodes Attention weights; || denotes vector concatenation operation; Indicates the first The learnable attention vectors of attention heads, where T represents the transpose; LeakyReLU represents the non-linear activation function; Represents the spatial neighbor index. Indicates the target vehicle A set of vehicles that are spatially adjacent;

[0149] By observation time window The above updates are performed at each time step, and the results are stacked in chronological order to obtain the spatiotemporal interaction feature sequence of the target vehicle within the observation sequence. .

[0150] Step 24: Align the target vehicle motion features and spatiotemporal interaction features in the time dimension and then stitch and fuse them to obtain spatiotemporal fused features;

[0151] target vehicle motion characteristics Spatiotemporal interaction features After aligning along the time dimension, the data is stitched and fused to obtain spatiotemporal fusion features. The expression is:

[0152] ;(twenty four)

[0153] The spatiotemporal fusion features are then input into a parameterized input. A multilayer perceptron (MLP) is constructed, and the GELU nonlinear activation function is employed to enhance representation capability, thereby obtaining the feature representation of the dimension required for decoding. Furthermore, the feature distribution is standardized through layer normalization to obtain the sequence representation used for subsequent trajectory prediction decoding. The expression is:

[0154] (25).

[0155] Step 25: Input the fused features into the GRU, decode them, and then input them into the hybrid density network to learn the potential trajectory distribution from the spatiotemporal features, thereby achieving multimodal output of the vehicle trajectory.

[0156] The trajectory output module includes GRU and hybrid density network MDN;

[0157] GRU will fuse feature sequences The initial hidden state of the trajectory output module as input. Initialized from the last-moment state of the fused features, the expression is:

[0158] (26)

[0159] Decoding future time steps step by step using GRU The hidden state is expressed as:

[0160] (27)

[0161] Hybrid density networks will As input, it is used to predict the future trajectory distribution of the multimodal system, and the output is a multimodal Gaussian distribution parameter, including the mean of each mixture component. Covariance matrix and mixed weights The expression is:

[0162] (28)

[0163] in, This indicates the number of preset Gaussian mixture prediction trajectory distributions; Indicates the first The predicted mean position of the predicted trajectory; Indicates the first The prediction covariance matrix of the predicted trajectory; Indicates the first The mixed weights of the predicted trajectories define the probability of being sampled as the final trajectory, satisfying... ;

[0164] Based on mean Covariance matrix and mixed weights For every step of the future trajectory position Output its trajectory distribution probability, expressed as:

[0165] (29)

[0166] in, Indicates the probability of trajectory distribution; This represents a multivariate Gaussian distribution.

[0167] Step 3: Improve the model's prediction accuracy and convergence speed by selecting an appropriate loss function for training.

[0168] To ensure accurate multimodal trajectory prediction, the loss function... Using negative log-likelihood as the regression loss, this loss function measures the distribution of the predicted trajectory. The difference between the actual trajectory and the true trajectory ensures accurate multimodal trajectory prediction, expressed as:

[0169] (30)

[0170] in, Indicates the predicted length of the future time step; Indicates the target vehicle at the predicted time. The true location.

[0171] This invention uses US-101 and I-80 vehicle trajectory data from the publicly available NGSIM dataset to train the model. The dataset records data at a frequency of 10Hz, capturing vehicle position, speed, acceleration, and other state information in real highway scenarios. Each trajectory is divided into fixed time windows, with each segment spanning 8 seconds. The first 3 seconds are used as the input sequence for model feature learning, and the last 5 seconds are used as the prediction target. The dataset is divided into training, validation, and test sets in a 6:2:2 ratio to improve the model's generalization ability.

[0172] The proposed spatiotemporal attention-based vehicle multimodal trajectory prediction model was trained and evaluated using the PyTorch deep learning framework. The training iterations were 100, the batch size was 64, the Adam optimizer was used, and the initial learning rate was set to 0.001. Figure 4 The graph shows the changes in the loss function of the model on the training and validation sets. As can be seen, the loss function value on the training set converges continuously with the increase in iterations, eventually stabilizing around 0.026. The overall trend of the loss function value on the validation set is consistent with that of the training set, also showing a continuous decreasing and converging trend. The loss curves of the training and validation sets maintain a small difference, indicating that the model has good fitting ability and strong generalization performance.

[0173] This invention considers both a single-modal prediction (k=1) model and a multi-modal prediction (k=6) model. During the prediction process, historical 3-second data is used to predict the trajectory for the next 5 seconds. Widely used evaluation metrics in trajectory prediction are selected, including minimum average displacement error (minADE), minimum final displacement error (minFDE), and miss rate (MR), which are expressed as follows:

[0174] minADE represents the average Euclidean distance between the predicted trajectory and the true trajectory over all time steps, selecting the model output. The predicted trajectory with the smallest error is selected, and the mean error over all time steps is calculated as shown in the following formula:

[0175] (31)

[0176] in, Indicates the prediction time steps; This represents the number of trajectories predicted by the multimodal method. Indicates the first The predicted trajectory at time The predicted location coordinates; time The actual location coordinates; This represents the Euclidean distance (L2 norm) between the predicted trajectory and the actual trajectory.

[0177] minFDE represents selecting K predicted trajectories at the final time. The one that most closely resembles the actual trajectory is calculated in The displacement error at time t is given by the following formula:

[0178] (32)

[0179] in, Indicates the first The predicted trajectory at the final prediction time Location; Indicates the actual trajectory in The position at any given moment.

[0180] The miss rate represents the proportion of prediction failures when the distance between the predicted trajectory and all real trajectories is greater than a certain threshold. In this invention, the threshold is set to 2.0m.

[0181] To analyze the impact of different numbers of multimodal trajectories and different prediction durations on trajectory evaluation metrics, a study was conducted on a test set. Figure 5 The results show the impact of different numbers of multimodal trajectories on various evaluation indicators when predicting the trajectory in the next 5 seconds. It can be seen that as the number of multimodal trajectories increases, each evaluation indicator shows better performance, which indicates that the vehicle trajectory prediction method proposed in this invention has higher prediction accuracy as the number of multimodal trajectories increases. Figure 6 and Figure 7 The study shows the impact of different prediction durations on the evaluation index under single-modal and multi-modal conditions. It can be seen that the vehicle trajectory prediction method proposed in this invention exhibits high prediction accuracy in both single-modal and multi-modal predictions in the first 3 seconds, and shows better performance in multi-modal prediction. In the 4th and 5th seconds, the upward trend of its evaluation index slows down, indicating that the model in this study is suitable for long-term prediction and can ensure long-term prediction accuracy to a certain extent.

[0182] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for predicting vehicle multimodal trajectories based on an improved attention network, characterized in that, Includes the following steps: Step 1: Collect historical trajectory data of the target vehicle and surrounding vehicles in the target scene as the input sequence; Step 2: Construct a multimodal trajectory prediction model for the vehicle, including a motion feature extraction module, a spatiotemporal interaction module, a spatiotemporal fusion module, and a trajectory output module; The motion feature extraction module divides the historical state of the target vehicle and its surrounding vehicles into two parts, which are processed by a multi-scale convolutional attention network and a gated recurrent unit, respectively. The spatiotemporal interaction module uses a learnable threshold to construct a dynamic graph mechanism and extracts vehicle interaction information through a dynamic graph attention network. The spatiotemporal fusion module concatenates and fuses the motion features of the target vehicle with the spatiotemporal interaction features to obtain spatiotemporal fusion features. The trajectory output module inputs the fusion features into the gated recurrent unit, decodes them, and then inputs them into a hybrid density network to achieve multimodal output of the vehicle trajectory. Step 3: Select a loss function for training to improve the model's prediction accuracy and convergence speed; Step 2 includes the following sub-steps: Step 21: Convert the sequence of historical trajectories of the target vehicle and surrounding vehicles into high-dimensional vectors through an embedding layer; Step 22: Input the target vehicle's historical trajectory sequence into a multi-scale convolutional attention network to capture temporal features and process the local and global trends in the target vehicle's time series. Input the surrounding vehicles' historical trajectory sequences into a gated recurrent unit (GRU) to process the historical time series input features of the surrounding vehicles. Step 23: Establish a dynamic spatiotemporal interaction graph, taking the target vehicle and surrounding vehicles as nodes in the graph, dynamically constructing an adjacency matrix through a learnable threshold to determine the edge relationships between vehicles, and then using a dynamic graph attention mechanism to extract spatiotemporal interaction features. Step 24: Align the target vehicle motion features and spatiotemporal interaction features in the time dimension and then stitch and fuse them to obtain spatiotemporal fused features; Step 25: Input the fused features into the GRU, decode them, and then input them into the hybrid density network to learn the potential trajectory distribution from the spatiotemporal features and achieve multimodal output of the vehicle trajectory; In step 22, the multi-scale convolutional attention network first encodes the target vehicle's position vector. The position-encoded embedding vector is obtained by adding it to the embedding vector after linear transformation and activation function. ,in, Use sine and cosine functions of different frequencies to encode the time position for each time step; The multi-scale convolutional attention network utilizes a multi-head convolutional mapping mechanism and a multi-head attention mechanism to jointly capture the local behavioral trend features and global temporal dependencies of the target vehicle trajectory in different time intervals; The multi-head convolutional mapping mechanism employs an improved TCN network, taking the historical trajectory feature sequence of the target vehicle after location encoding as input, and outputting the sequence through two stacked layers of dynamically dilated causal convolutions. After each layer of dynamically dilated causal convolution, weight normalization and GELU convolution activation are performed. After the GELU convolution activation in the first layer, layer normalization is used to enhance the stability and expressive power of the network. Residual connections are added to the GELU activation in the second layer to alleviate the gradient vanishing problem. In each layer of dynamically dilated causal convolution, the dynamic dilation rate controls the sampling stride. The expressions for the two layers of dilated causal convolution are as follows: ；(8) ；(9) in, Indicates the current time step position; and These represent the first layer convolution kernels respectively. Second layer convolution kernel In the Weights at each position; and These represent the sizes of the first and second convolutional kernels, respectively. The dynamically dilated causal convolution extracts the real-time vehicle speed from the historical trajectory feature sequence of the target vehicle. The dynamic convolutional dilation rate of each layer is calculated using learnable parameters. Specifically, it is expressed as: ；(10) in, Indicates the first Layered dynamic dilated causal convolution; and This represents the independently learnable parameters of each layer; This represents a truncation function used to truncate... Limit to a preset minimum value and maximum value Within the range; The convolutional trend-aware feature matrix Linear projection is performed to capture temporal dependency and trend features in different subspaces by mapping the dimensions from the convolutional feature space to the attention space, thereby generating a query matrix. Key matrix Value matrix The multi-head attention mechanism is specifically generated based on each group of multi-head convolutional mappings. , and The attention weight of the target vehicle in each subspace is calculated, and the outputs of all attention heads are concatenated and linearly transformed to obtain the final trend-aware temporal features. ; Multi-head attention output With convolution output features After performing residual connections and layer normalization, we obtain , The data is fed into a feedforward neural network (FNN) and then reconnected using residuals. The final output is the motion features of the target vehicle. ; The historical trajectory sequences of surrounding vehicles are input and the motion features of the surrounding vehicles are extracted through a gated recurrent unit (GRU) to obtain the motion features of each vehicle. The final hidden state of time Finally, the motion characteristics of each surrounding vehicle are output. ; Step 23 specifically involves: for vehicles based on the target vehicle At time step speed acceleration Local traffic density and motion time sequence feature vector Together, they form a scene feature vector. The input is fed into the multilayer perceptron (MLP) to predict the dynamic neighbor selection threshold. The expression is: ；(18) in, , , , For learnable parameters, Ensure the output is positive; The predicted value is linearly adjusted by introducing a minimum safety threshold and a scaling factor to obtain the set threshold. Determine the target vehicle With surrounding vehicles Euclidean distance between Less than the set threshold Based on this, a time step is constructed. Spatial adjacency matrix The expression is: ；(20) Constructing dynamic spatial interaction graphs Each car in the scene is treated as a set of nodes in a graph. Each node corresponds to the motion characteristics of the target vehicle and surrounding vehicles in step 22, based on the time step. Adjacency matrix obtained by learnable threshold filtering , will satisfy node pairs As an edge, it forms in time step edge set ; Dynamic graph attention networks use a multi-head attention mechanism, where each attention head focuses on the target vehicle through attention weights. At time step Surrounding vehicles The features are weighted and summed to achieve effective fusion of vehicle dynamic interaction information and modeling of spatial dependencies. The outputs of all heads are averaged along the feature dimension to obtain the node's position at the current time step. Updated feature representation The specific calculation expression is as follows: ；(22) in, It's about the number of heads; and Indicates the first Attention weights and weight matrices for each size; It is a non-linear activation function GELU; the attention weights are measured under the constraint of a dynamic adjacency matrix. Time of the first The target vehicle was under everyone's attention. With surrounding vehicles The quantitative indicator of interaction strength is expressed as follows: ；(23) In the formula, express Time of the first Attention focus on the target vehicle node For neighboring vehicle nodes Attention weights; || denotes vector concatenation operation; Indicates the first The learnable attention vectors of attention heads, where T represents the transpose; LeakyReLU represents the non-linear activation function; Represents the spatial neighbor index. Indicates the target vehicle A set of vehicles that are spatially adjacent; By observation time window The above updates are performed at each time step, and the results are stacked in chronological order to obtain the spatiotemporal interaction feature sequence of the target vehicle within the observation sequence. .

2. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 1, characterized in that, In step 1, the historical trajectory data includes the target vehicle and surrounding vehicles at a time step. The historical state within is expressed as: ； (1) in, Indicates selecting the first The vehicle is the target vehicle. This represents the target vehicle's historical trajectory data; Indicates the index of surrounding vehicles. , The number of vehicles in the surrounding area; This indicates historical trajectory data of surrounding vehicles; and These represent the target vehicle and surrounding vehicles, respectively. The vehicle status at any given time, including the vehicle's position in... and Six-dimensional features including position, velocity, and acceleration in the direction; Historical vehicle trajectory data is input into a pre-built multimodal trajectory prediction model based on an improved attention network. Motion feature extraction and dynamic interaction modeling operations are performed to extract high-dimensional spatiotemporal fusion features of the target vehicle for multimodal trajectory prediction.

3. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 2, characterized in that, In step 21, the embedding layer transforms the original six-dimensional information of the scene vehicle into a high-dimensional vector through linear transformation and activation function, as expressed by: ； (2) in, , These represent the motion state embedding features of the target vehicle and surrounding vehicles at time step t, respectively. This represents the weight matrix of the embedding layer; represents a linear layer; GELU represents a Gaussian error linear unit.

4. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 3, characterized in that, In step 24, the motion characteristics of the target vehicle are... Spatiotemporal interaction features After aligning along the time dimension, the data is stitched and fused to obtain spatiotemporal fusion features. The expression is: ； (24) The spatiotemporal fusion features are then input into a parameterized input. A multilayer perceptron (MLP) is constructed, and the GELU nonlinear activation function is employed to enhance representation capability, thereby obtaining the feature representation of the dimension required for decoding. Furthermore, the feature distribution is standardized through layer normalization to obtain the sequence representation used for subsequent trajectory prediction decoding. The expression is: (25)。 5. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 4, characterized in that, In step 25, the trajectory output module includes a GRU and a hybrid density network; The GRU will fuse feature sequences. The initial hidden state of the trajectory output module as input. Initialized from the last moment state of the fused features; Decoding future time steps step by step using GRU The hidden state is expressed as: ；(27) The hybrid density network will As input, it is used to predict the future trajectory distribution of the multimodal system, and the output is a multimodal Gaussian distribution parameter, including the mean of each mixture component. Covariance matrix and mixed weights The expression is: ； (28) in, This indicates the number of preset Gaussian mixture prediction trajectory distributions; Indicates the first The predicted mean position of the predicted trajectory; Indicates the first The prediction covariance matrix of the predicted trajectory; Indicates the first The mixed weights of the predicted trajectories define the probability of being sampled as the final trajectory, satisfying... ; Based on mean Covariance matrix and mixed weights For every step of the future trajectory position Output its trajectory distribution probability, expressed as: ；(29) in, Indicates the probability of trajectory distribution; This represents a multivariate Gaussian distribution.

6. The vehicle multimodal trajectory prediction method based on an improved attention network according to claim 5, characterized in that, In step 3, the loss function Using negative log-likelihood as the regression loss, this loss function measures the distribution of the predicted trajectory. The difference between the actual trajectory and the true trajectory is expressed as: ；(30) in, Indicates the predicted length of the future time step; This represents the actual position of the target vehicle at the predicted time t.

Images