Vehicle multi-modal trajectory prediction method

Abstract

A vehicle multi-mode trajectory prediction method relates to the technical field of automatic driving, is based on a joint space-time social interaction attention mechanism, and comprises the following steps: obtaining historical trajectories and motion state information of a target vehicle and surrounding vehicles; preprocessing the historical track and motion state information of the vehicle; inputting the processed trajectory data into an encoder to extract a hidden state; the space interaction module extracts space interaction features of the target vehicle; inputting the obtained space interaction characteristics into a time interaction module to output time interaction characteristics of the target vehicle; inputting the two into a joint space-time interaction module, and outputting to obtain joint space-time interaction characteristics of the target vehicle; carrying out intention fusion on the joint space-time interaction characteristics and transverse and longitudinal intention maneuvering; and the decoder decodes and outputs the multi-modal prediction trajectory of the target vehicle. According to the method, the interaction relationship under the mutual influence of time and space interaction characteristics is fully considered, the interpretability of the method is improved, and the precision of vehicle trajectory prediction can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle automatic driving, and specifically relates to a vehicle multimodal trajectory prediction method. Background Art

[0002] In recent years, autonomous driving technology has become a research hotspot in the global automotive industry. Due to the development of artificial intelligence, autonomous driving technology has gradually appeared in real life. With the further development of autonomous driving technology, autonomous driving cars may subvert people's traditional travel methods.

[0003] However, due to the complex traffic environment, autonomous driving technology faces huge challenges. In order to drive safely in a real driving environment, autonomous vehicles need to predict the action intentions and trajectories of surrounding vehicles or pedestrians. By predicting future trajectories, autonomous vehicles can conduct risk assessments for future moments in advance, thereby determining the risk of collision during driving and making more reasonable decision-making plans. Predicting the future trajectories of vehicles or pedestrians around the car requires the trajectory prediction module in autonomous driving technology. However, the future trajectory of a vehicle is often difficult to predict. The vehicle is not only affected by its historical trajectory, but also by the influence of surrounding vehicles. Therefore, the technology of vehicle trajectory prediction faces the challenge of analyzing the spatiotemporal interaction relationship of multiple vehicles in a complex traffic environment.

[0004] In the early days, trajectory prediction models based on physical models were usually used to solve the trajectory prediction problem. However, such models have poor generalization ability and can only perform short-term predictions. Then, trajectory prediction models based on machine learning were proposed. Such models still have the disadvantages of poor generalization ability and cannot process high-dimensional data. In recent years, with the development of deep learning, trajectory prediction models based on deep learning have become the mainstream in the field of trajectory prediction. However, some models only analyze the social interaction relationship between vehicles at the temporal or spatial level separately, and do not combine the two to consider the interaction relationship between vehicles, resulting in low prediction accuracy. In addition, the trajectory prediction results of vehicles are often not unimodal. The uncertainty of vehicle driving intentions leads to the prediction results not being unique, but there are multiple different prediction trajectories according to different driving intentions. Therefore, multimodal trajectory prediction is required to comprehensively analyze the trajectories that may be generated under various intentions. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a vehicle multimodal trajectory prediction method based on a joint spatiotemporal social interaction attention mechanism to solve the problem that the current prediction method does not achieve multimodality and has low prediction accuracy.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A vehicle multimodal trajectory prediction method, based on a joint spatiotemporal social interaction attention mechanism, comprises the following steps:

[0007] S1: Obtain the historical trajectory and motion status information of all vehicles within the set range through sensors;

[0008] S2: preprocessing the acquired vehicle historical trajectory and motion status information;

[0009] S3: The encoder composed of n+1 LSTM networks encodes the processed vehicle history trajectory and its motion state information, extracts the temporal relationship between the target vehicle and the surrounding vehicles’ history trajectory, and outputs the hidden state of the target vehicle and the surrounding vehicles.

[0010] S4: After obtaining the hidden state of the historical trajectory of the target vehicle and the surrounding vehicles, the hidden state of the target vehicle and the surrounding vehicles is Input spatial interaction layer, use multi-head attention mechanism to output the interaction features between target vehicle and surrounding vehicles in space

[0011] S5: Transform the spatial interaction feature vector Input the time interaction layer and output the time interaction features of the target vehicle

[0012] S6: Transform the spatial eigenvector and the time feature vector The input joint space-time interaction layer can fully consider the interaction relationship between vehicles at the spatial and temporal levels, and output the joint space-time interaction feature V of the target vehicle. spa-tem ;

[0013] S7: Considering that the trajectory of the vehicle is uncertain, the spatiotemporal joint interaction feature vector V spa-tem It is integrated with different intention maneuvers, and through the self-attention mechanism, it adaptively determines the weight of different intention maneuvers on the trajectory and outputs features with intention information.

[0014] S8: The vector with the intent feature Input the LSTM network for decoding, and output the final multimodal trajectory;

[0015] S9: Train the trajectory prediction model through the mean square error loss function and minimize the negative log-likelihood loss function, and update the model parameters.

[0016] The step S1 is specifically as follows:

[0017] Define the scope of the set scene: ±50m longitudinally of the vehicle, two adjacent lanes horizontally, and define the historical trajectory and motion information:

[0018]

[0019] Where, X i is the i-th vehicle at time step t his The status information below includes the vehicle's position coordinates and the vehicle's acceleration.

[0020] The step S2 is specifically as follows:

[0021] The median filter method is used to correct the speed mutation error, and the Kalman filter is used to eliminate the possible noise in the trajectory data.

[0022] The step S3 is specifically as follows:

[0023] S3.1: The historical trajectories and motion state information of the target vehicle and surrounding vehicles are transformed and embedded using a multi-layer perceptron (MLP). The formula is as follows:

[0024]

[0025] Where Ψ is a multi-layer perceptron, is the motion state information of vehicle i at time t, w emd is the weight parameter of the fully connected layer;

[0026] S3.2: Embedding Representation The hidden state of the vehicle is obtained by encoding it into the LSTM network as shown below:

[0027]

[0028] Where LSTM is the LSTM encoder, is the hidden state of vehicle i at time t, w lstm -enconder is the weight parameter of the LSTM encoder.

[0029] The step S4 is specifically as follows:

[0030] S4.1: The hidden state of the target vehicle The hidden state sequence of the surrounding vehicles is mapped to query, key and value using three different convolutions. The specific formula is as follows:

[0031]

[0032] In the formula, and is the learnable weight matrix of the convolutional transformation, is the hidden state of the target vehicle, is the hidden status of surrounding vehicles;

[0033] S4.2: Using the dot product attention mechanism in multi-head attention to q t , k t and v t The specific formula is as follows:

[0034]

[0035] Where softmax is the softmax function, <> represents the dot product, and d k is a hyperparameter;

[0036] S4.3: N independent heads are set up to more fully extract the spatial interaction relationship between the target vehicle and surrounding vehicles. These N heads are connected together. The specific formula is as follows:

[0037]

[0038] S4.4: Performing layer normalization and residual connection processing on h helps to improve the training effect and prevent gradient explosion. The specific formula is as follows:

[0039]

[0040] in, is the spatial feature of the target vehicle, Addandnorm is the layer normalization and residual connection operation, GLU is the activation function, W GLU It is the weight parameter in the GLU function.

[0041] The step S5 is specifically as follows:

[0042] S5.1: Three different convolutions are used to map to queries, keys, and values. The specific formulas are as follows:

[0043]

[0044] Among them, W tem_ α,W tem_ β and W tem_ γ is the learnable weight matrix of the convolution transformation;

[0045] S5.2: Use the dot product scaling attention mechanism in multi-head self-attention to process Q, K and V. The specific formula is as follows:

[0046]

[0047] Head=BV t

[0048] S5.3: We set up N independent heads to more fully extract the impact of the target vehicle and surrounding vehicles at different times on the current moment. Connect these N heads together. The specific formula is as follows:

[0049] H=Concat(Head1,Head2,…,Head N )

[0050] S5.4: Performing layer normalization and residual connection processing on H helps to improve the training effect and prevent gradient explosion. The specific formula is as follows:

[0051]

[0052] In the formula, is the temporal interaction feature of the target vehicle.

[0053] The step S6 is specifically as follows:

[0054] S6.1: V spa and V tem Send it to the joint space-time interaction layer, and V spa and V tem Three different convolutions are used to map to queries, keys, and values. The specific formulas are as follows:

[0055]

[0056] Among them, W spatem_α ,W spatem_β and W spatem_γ is the learnable weight matrix of the convolution transformation;

[0057] S6.2: Using the dot product attention mechanism in multi-head attention and The specific formula is as follows:

[0058]

[0059] S6.3: We set up N independent heads to more fully extract the interaction between the target vehicle and surrounding vehicles at the spatiotemporal fusion level. Connect these N heads, and the specific formula is as follows:

[0060]

[0061] S6.4: Performing layer normalization and residual connection processing on H helps to improve the training effect and prevent gradient explosion. The specific formula is as follows:

[0062]

[0063] Where Vspa- t em It is the joint interaction feature of space and time.

[0064] The step S7 is specifically as follows:

[0065] S7.1: The intention types are divided into lateral intentions and longitudinal intentions. The lateral intentions include three types of maneuvers: change lane left (LCL), change lane right (LCR), and keep straight (KL). The longitudinal intentions include three types of maneuvers: acceleration (ACC), deceleration (DEC), and constant speed (CS):

[0066] m la ={LCL,LCR,KL}

[0067] m lo ={ACC,DEC,CS}

[0068] Among them, m la is the set of lateral intended maneuver types, m lo is the set of longitudinal intention maneuver types;

[0069] S7.2: The six maneuver types are presented in one-hot encoding. and

[0070] S7.3: The context vector V with the spatiotemporal joint interaction features spa—tem Combined with W, the predicted trajectory under various possible intention maneuver types can be realized. The specific formula is as follows:

[0071]

[0072] In the formula, is the context vector with intent fusion features, t pred It’s a prediction moment;

[0073] S7.4: Use the Softmax activation function to output the horizontal intention probability and the vertical intention probability. The specific formula is as follows:

[0074]

[0075] Among them, P(M lo ) is the probability of lateral intention, P(M la ) is the longitudinal intention probability, W la and W lo is the weight parameter;

[0076] S7.5 uses a fully connected layer to P(M la ),P(M lo ) are connected, the specific formula is as follows:

[0077]

[0078] right Processing with self-attention mechanism to obtain vector with intention information The self-attention mechanism can realize the function of adaptive weighting according to the horizontal and vertical intention probabilities through attention weights, and can better integrate intention information.

[0079] 9. The vehicle multi-modal trajectory prediction method according to claim 1 is characterized in that: the step S8 is specifically as follows:

[0080] S8.1: The LSTM encoder outputs a probability distribution:

[0081]

[0082] At time t = {t his +1,t his +2,…,t his +t f}, the distribution of the predicted position can be expressed as with parameter θ t Gaussian bivariate distribution, is the bivariate Gaussian distribution parameter at each moment in the prediction time,

[0083] S8.2: Input LSTM network for decoding, the specific formula is as follows:

[0084]

[0085] Where W lstm-decoder is the weight parameter of LSTM in the decoder;

[0086] S8.3: Transformed into bivariate Gaussian distribution parameters, the specific formula is as follows:

[0087]

[0088] Where W Guass Yes The weight parameter converted to bivariate Gaussian distribution parameter.

[0089] 10. The step S9 is specifically as follows:

[0090] S9.1: This model is trained in two stages. The first stage of training uses the mean square error loss function, or MSE, which is defined as follows:

[0091]

[0092] The second stage uses the minimization of the negative log-likelihood loss function, or NLL, to train the model. NLL is defined as follows:

[0093]

[0094] S9.2: Training details This model is trained on the NVIDIA GeForce RTX 3060TI GPU based on the Pytorch deep learning framework. The optimizer selected is the Adam optimizer, and the learning rate is selected as 0.0005s.

[0095] The beneficial effects of the present invention are as follows: the prediction method of the present invention is based on the joint spatiotemporal interaction layer of the attention mechanism, which can jointly analyze the interaction relationship of the target vehicle with the spatial interaction features, improve the trajectory prediction accuracy and interpretability, and introduce the self-attention mechanism in the fusion intention maneuver, which can adaptively weight the influence of different intention maneuvers on the predicted trajectory, thereby improving the rationality of multimodality. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 This is a flow chart of vehicle multimodal trajectory prediction according to the present invention.

[0097] Figure 2 This is a model structure diagram for vehicle multimodal trajectory prediction according to the present invention.

[0098] Figure 3 Schematic diagram of the principle of the joint spatiotemporal interaction module for multimodal vehicle trajectory prediction of the present invention.

[0099] Figure 4 This is the intention fusion graph for the vehicle multimodal trajectory prediction of the present invention.

[0100] Figure 5 This is the RMSE comparison experimental results table of different models in 0~5s.

[0101] Figure 6 This is the experimental result table of ablation experiment. DETAILED DESCRIPTION

[0102] The technical solution of the present invention will be described clearly and completely below in conjunction with the drawings of this specification. It should be noted that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0103] See also Figure 1-Figure 6 ,A vehicle multimodal trajectory prediction method based on the joint spatiotemporal social interaction attention mechanism includes the following steps:

[0104] S1: Obtain the historical trajectory and motion status information of all vehicles within the set range through sensors;

[0105] The step S1 is specifically as follows:

[0106] Define the scope of the set scene: ±50m longitudinally of the vehicle, two adjacent lanes horizontally, and define the historical trajectory and motion information:

[0107]

[0108] Where, X i is the i-th vehicle at time step t his The status information below includes the vehicle's position coordinates and the vehicle's acceleration.

[0109] S2: preprocessing the acquired vehicle historical trajectory and motion status information;

[0110] The step S2 is specifically as follows:

[0111] The median filter method is used to correct the speed mutation error, and the Kalman filter is used to eliminate the possible noise in the trajectory data.

[0112] S3: The encoder composed of n+1 LSTM networks encodes the processed vehicle history trajectory and its motion state information, extracts the temporal relationship between the target vehicle and the surrounding vehicles’ history trajectory, and outputs the hidden state of the target vehicle and the surrounding vehicles.

[0113] The step S3 is specifically as follows:

[0114] S3.1: The historical trajectories and motion state information of the target vehicle and surrounding vehicles are transformed and embedded using a multi-layer perceptron (MLP). The formula is as follows:

[0115]

[0116] Where Ψ is a multi-layer perceptron, is the motion state information of vehicle i at time t, w emd is the weight parameter of the fully connected layer;

[0117] S3.2: Embedding Representation The hidden state of the vehicle is obtained by encoding it into the LSTM network as shown below:

[0118]

[0119] Where LSTM is the LSTM encoder, is the hidden state of vehicle i at time t, w lstm -enconder is the weight parameter of the LSTM encoder.

[0120] S4: After obtaining the hidden state of the historical trajectory of the target vehicle and the surrounding vehicles, the hidden state of the target vehicle and the surrounding vehicles is Input spatial interaction layer, use multi-head attention mechanism to output the interaction features between target vehicle and surrounding vehicles in space

[0121] The step S4 is specifically as follows:

[0122] S4.1: The hidden state of the target vehicle The hidden state sequence of the surrounding vehicles is mapped to query, key and value using three different convolutions. The specific formula is as follows:

[0123]

[0124] In the formula, and is the learnable weight matrix of the convolutional transformation, is the hidden state of the target vehicle, is the hidden status of surrounding vehicles;

[0125] S4.2: Using the dot product attention mechanism in multi-head attention to q t , k t and v t The specific formula is as follows:

[0126]

[0127] Where softmax is the softmax function, <> represents the dot product, and d k is a hyperparameter;

[0128] S4.3: N independent heads are set up to more fully extract the spatial interaction relationship between the target vehicle and surrounding vehicles. These N heads are connected together. The specific formula is as follows:

[0129]

[0130] S4.4: Performing layer normalization and residual connection processing on h helps to improve the training effect and prevent gradient explosion. The specific formula is as follows:

[0131]

[0132] in, is the spatial feature of the target vehicle, Addandnorm is the layer normalization and residual connection operation, GLU is the activation function, W GLU It is the weight parameter in the GLU function.

[0133] S5: Transform the spatial interaction feature vector Input the time interaction layer and output the time interaction features of the target vehicle

[0134] The step S5 is specifically as follows:

[0135] S5.1: Three different convolutions are used to map to queries, keys, and values. The specific formulas are as follows:

[0136]

[0137] Among them, W tem_α ,W tem_β and W tem_γ is the learnable weight matrix of the convolution transformation;

[0138] S5.2: Use the dot product scaling attention mechanism in multi-head self-attention to process Q, K and V. The specific formula is as follows:

[0139]

[0140] Head=BV t

[0141] S5.3: We set up N independent heads to more fully extract the impact of the target vehicle and surrounding vehicles at different times on the current moment. Connect these N heads together. The specific formula is as follows:

[0142] H=Concat(Head1,Head2,…,Head N )

[0143] S5.4: Performing layer normalization and residual connection processing on H helps to improve the training effect and prevent gradient explosion. The specific formula is as follows:

[0144]

[0145] In the formula, is the temporal interaction feature of the target vehicle.

[0146] S6: Transform the spatial eigenvector and the time feature vector The input joint space-time interaction layer can fully consider the interaction relationship between vehicles at the spatial and temporal levels, and output the joint space-time interaction feature V of the target vehicle. spa-tem;

[0147] The step S6 is specifically as follows:

[0148] S6.1: V spa and V tem Send it to the joint space-time interaction layer, and V spa and V tem Three different convolutions are used to map to queries, keys, and values. The specific formulas are as follows:

[0149]

[0150] Among them, W spatem_α ,W spatem_β and W spatem_γ is the learnable weight matrix of the convolution transformation;

[0151] S6.2: Using the dot product attention mechanism in multi-head attention and The specific formula is as follows:

[0152]

[0153] S6.3: We set up N independent heads to more fully extract the interaction between the target vehicle and surrounding vehicles at the spatiotemporal fusion level. Connect these N heads, and the specific formula is as follows:

[0154]

[0155] S6.4: Performing layer normalization and residual connection processing on H helps to improve the training effect and prevent gradient explosion. The specific formula is as follows:

[0156]

[0157] Where V spa- t em It is the joint interaction feature of space and time.

[0158] S7: Considering that the trajectory of the vehicle is uncertain, the spatiotemporal joint interaction feature vector V spa-tem It is integrated with different intention maneuvers, and through the self-attention mechanism, it adaptively determines the weight of different intention maneuvers on the trajectory and outputs features with intention information.

[0159] The step S7 is specifically as follows:

[0160] S7.1: The intention types are divided into lateral intentions and longitudinal intentions. The lateral intentions include three types of maneuvers: change lane left (LCL), change lane right (LCR), and keep straight (KL). The longitudinal intentions include three types of maneuvers: acceleration (ACC), deceleration (DEC), and constant speed (CS):

[0161] m la ={LCL,LCR,KL}

[0162] m lo ={ACC,DEC,CS}

[0163] Among them, m la is the set of lateral intended maneuver types, m lo is the set of longitudinal intention maneuver types

[0164] S7.2: The six maneuver types are presented in one-hot encoding. and

[0165] S7.3: The context vector V with the spatiotemporal joint interaction features spa—tem Combined with W, the predicted trajectory under various possible intention maneuver types can be realized. The specific formula is as follows:

[0166]

[0167] In the formula, is the context vector with intent fusion features, t pred It’s a prediction moment;

[0168] S7.4: Use the Softmax activation function to output the horizontal intention probability and the vertical intention probability. The specific formula is as follows:

[0169]

[0170] Among them, P(M lo ) is the probability of lateral intention, P(M la ) is the longitudinal intention probability, W la and W lo is the weight parameter;

[0171] S7.5 uses a fully connected layer to P(M la ),P(M lo ) are connected, the specific formula is as follows:

[0172]

[0173] right Processing with self-attention mechanism to obtain vector with intention information The self-attention mechanism can realize the function of adaptive weighting according to the horizontal and vertical intention probabilities through attention weights, and can better integrate intention information.

[0174] S8: The vector with the intent feature Input the LSTM network for decoding, and output the final multimodal trajectory;

[0175] The step S8 is specifically as follows:

[0176] S8.1: The LSTM encoder outputs a probability distribution:

[0177]

[0178] At time t = {t his +1,t his +2,…,t his +t f}, the distribution of the predicted position can be expressed as with parameter θ t Gaussian bivariate distribution, is the bivariate Gaussian distribution parameter at each moment in the prediction time,

[0179] S8.2: Input LSTM network for decoding, the specific formula is as follows:

[0180]

[0181] Where W lstm-decoder is the weight parameter of LSTM in the decoder;

[0182] S8.3: Transformed into bivariate Gaussian distribution parameters, the specific formula is as follows:

[0183]

[0184] Where W Guass Yes The weight parameter converted to bivariate Gaussian distribution parameter.

[0185] S9: Train the trajectory prediction model through the mean square error loss function and minimize the negative log-likelihood loss function, and update the model parameters;

[0186] The step S9 is specifically as follows:

[0187] S9.1: This model is trained in two stages. The first stage of training uses the mean square error loss function, or MSE, which is defined as follows:

[0188]

[0189] The second stage uses the minimization of the negative log-likelihood loss function, or NLL, to train the model. NLL is defined as follows:

[0190]

[0191] S9.2: Training details This model is trained on the NVIDIA GeForce RTX 3060TI GPU based on the Pytorch deep learning framework. The optimizer used is the Adam optimizer, and the learning rate is selected as 0.0005s;

[0192] In order to verify the prediction accuracy of the method of the present invention, the open source data set NGSIM released by the German Ministry of Highways is used to verify the method of the present invention, and the prediction effect is compared with other existing trajectory prediction models;

[0193] The present invention uses the root mean square error (RMSE) to quantitatively analyze the model proposed in the present invention and other models. The smaller the RMSE, the closer the predicted trajectory of the model proposed in the present invention is to the actual trajectory, and the higher the prediction accuracy. The RMSE is defined as follows:

[0194]

[0195] In the formula, and is the true trajectory at time t pred The horizontal and vertical coordinates of and is the predicted trajectory at time t pred The horizontal and vertical coordinates of .

[0196] The spatiotemporal interactive multimodal trajectory prediction method based on LSTM-attention mechanism proposed in this paper is compared with baseline models such as S-LSTM, CS-LSTM, MHA-LSTM, MFP and MMnTP. The RMSE is used to evaluate the prediction accuracy of each model. The model comparison experimental results are shown in Figure 2. Figure 5 As shown, the results show that the proposed model outperforms other baseline models;

[0197] In order to verify the role of the two key components proposed in this invention, a series of ablation experiments were conducted. Similarly, RMSE was used to evaluate whether the ablated components were effective. The experimental results of the ablation experiments are shown in the figure. Figure 6 As shown in the figure, ASAT is a spatiotemporal cross-interaction module, S-ATT is a multi-head self-attention deployed in the decoder, ⅹ indicates that the component is ablated, and √ indicates the opposite. The experimental results show that the two key components proposed in this invention are effective in improving the prediction accuracy.

[0198] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0199] The parts not described in detail in this invention are prior art.

Claims

1. A vehicle multimodal trajectory prediction method based on a joint spatiotemporal social interaction attention mechanism, Its characteristics are: The following steps are involved: S1: Obtain the historical trajectory and motion status information of all vehicles within the set range through sensors; S2: preprocessing the acquired vehicle historical trajectory and motion status information; S3: The encoder composed of n+1 LSTM networks encodes the processed vehicle history trajectory and its motion state information, extracts the temporal relationship between the target vehicle and the surrounding vehicles’ history trajectory, and outputs the hidden state of the target vehicle and the surrounding vehicles. S4: After obtaining the hidden state of the historical trajectory of the target vehicle and the surrounding vehicles, the hidden state of the target vehicle and the surrounding vehicles is Input spatial interaction layer, use multi-head attention mechanism to output the interaction features between target vehicle and surrounding vehicles in space S5: Transform the spatial interaction feature vector Input the time interaction layer and output the time interaction features of the target vehicle S6: Transform the spatial eigenvector and the time feature vector The input joint space-time interaction layer can fully consider the interaction relationship between vehicles at the spatial and temporal levels, and output the joint space-time interaction feature V of the target vehicle. spa-tem ; S7: Considering that the trajectory of the vehicle is uncertain, the spatiotemporal joint interaction feature vector V spa-tem It is integrated with different intention maneuvers, and through the self-attention mechanism, it adaptively determines the weight of different intention maneuvers on the trajectory and outputs features with intention information. S8: The vector with the intent feature Input the LSTM network for decoding, and output the final multimodal trajectory; S9: Train the trajectory prediction model through the mean square error loss function and minimize the negative log-likelihood loss function, and update the model parameters.

2. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S1 is specifically as follows: Define the scope of the set scene: ±50m longitudinally of the vehicle, two adjacent lanes horizontally, and define the historical trajectory and motion information: Where, X i is the i-th vehicle at time step t his The status information below includes the vehicle's position coordinates and the vehicle's acceleration.

3. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S2 is specifically as follows: The median filter method is used to correct the speed mutation error, and the Kalman filter is used to eliminate the possible noise in the trajectory data.

4. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S3 is specifically as follows: S3.1: The historical trajectories and motion state information of the target vehicle and surrounding vehicles are transformed and embedded using a multi-layer perceptron (MLP). The formula is as follows: Where Ψ is a multi-layer perceptron, is the motion state information of vehicle i at time t, w emd is the weight parameter of the fully connected layer; S3.2: Embedding Representation The hidden state of the vehicle is obtained by encoding it into the LSTM network as shown below: Where LSTM is the LSTM encoder, is the hidden state of vehicle i at time t, w lstm-enconder is the weight parameter of the LSTM encoder.

5. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S4 is specifically as follows: S4.1: The hidden state of the target vehicle The hidden state sequence of the surrounding vehicles is mapped to query, key and value using three different convolutions. The specific formula is as follows: In the formula, and is the learnable weight matrix of the convolutional transformation, is the hidden state of the target vehicle, is the hidden status of surrounding vehicles; S4.2: Using the dot product attention mechanism in multi-head attention to q t , k t and v t The specific formula is as follows: Where softmax is the softmax function, <> represents the dot product, and d k is a hyperparameter; S4.3: N independent heads are set up to more fully extract the spatial interaction relationship between the target vehicle and surrounding vehicles. These N heads are connected together. The specific formula is as follows: S4.4: Performing layer normalization and residual connection processing on h helps to improve the training effect and prevent gradient explosion. The specific formula is as follows: in, is the spatial feature of the target vehicle, Addandnorm is the layer normalization and residual connection operation, GLU is the activation function, W GLU It is the weight parameter in the GLU function.

6. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S5 is specifically as follows: S5.1: Three different convolutions are used to map to queries, keys, and values. The specific formulas are as follows: Among them, W tem_α ,W tem_β and W tem_γ is the learnable weight matrix of the convolution transformation; S5.2: Use the dot product scaling attention mechanism in multi-head self-attention to process Q, K and V. The specific formula is as follows: Head=BV t S5.3: We set up N independent heads to more fully extract the impact of the target vehicle and surrounding vehicles at different times on the current moment. Connect these N heads together. The specific formula is as follows: H=Concat(Head 1 ,Head 2 ,…,Head N ) S5.4: Performing layer normalization and residual connection processing on H helps to improve the training effect and prevent gradient explosion. The specific formula is as follows: In the formula, is the temporal interaction feature of the target vehicle.

7. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S6 is specifically as follows: S6.1: V spa and V tem Send it to the joint space-time interaction layer, and V spa and V tem Three different convolutions are used to map to queries, keys, and values. The specific formulas are as follows: Among them, W spatem_α ,W spatem_β and W spatem_γ is the learnable weight matrix of the convolution transformation; S6.2: Using the dot product attention mechanism in multi-head attention and The specific formula is as follows: S6.3: We set up N independent heads to more fully extract the interaction between the target vehicle and surrounding vehicles at the spatiotemporal fusion level. Connect these N heads, and the specific formula is as follows: S6.4: Performing layer normalization and residual connection processing on H helps to improve the training effect and prevent gradient explosion. The specific formula is as follows: Where V spa-tem It is the joint interaction feature of space and time.

8. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S7 is specifically as follows: S7.1: The intention types are divided into lateral intentions and longitudinal intentions. The lateral intentions include three types of maneuvers: lane change left (LCL), lane change right (LCR), and stay straight (KL). The longitudinal intentions include three types of maneuvers: acceleration (ACC), deceleration (DEC), and constant speed (CS): m la ={LCL,LCR,KL} m lo ={ACC,DEC,CS} Among them, m la is the set of lateral intended maneuver types, m lo is the set of longitudinal intention maneuver types; S7.2: The six maneuver types are presented in one-hot encoding. and S7.3: The context vector V with the spatiotemporal joint interaction features spa—tem Combined with W, the predicted trajectory under various possible intention maneuver types can be realized. The specific formula is as follows: In the formula, is the context vector with intent fusion features, t pred It’s a prediction moment; S7.4: Use the Softmax activation function to output the horizontal intention probability and the vertical intention probability. The specific formula is as follows: Among them, P(M lo ) is the probability of lateral intention, P(M la ) is the longitudinal intention probability, W la and W lo is the weight parameter; S7.5 uses a fully connected layer to P(M la ),P(M lo ) are connected, the specific formula is as follows: right Processing with self-attention mechanism to obtain vector with intention information The self-attention mechanism can realize the function of adaptive weighting according to the horizontal and vertical intention probabilities through attention weights, and can better integrate intention information.

9. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S8 is specifically as follows: S8.1: The LSTM encoder outputs a probability distribution: At time t = {t his +1,t his +2,…,t his +t f }, the distribution of the predicted position can be expressed as with parameter θ t Gaussian bivariate distribution, is the bivariate Gaussian distribution parameter at each moment in the prediction time, S8.2: Input LSTM network for decoding, the specific formula is as follows: Where W lstm-decoder is the weight parameter of LSTM in the decoder; S8.3: Transformed into bivariate Gaussian distribution parameters, the specific formula is as follows: Where W Guass Yes The weight parameter converted to bivariate Gaussian distribution parameter.

10. A vehicle multimodal trajectory prediction method according to claim 1, Its characteristics are: The step S9 is specifically as follows: S9.1: This model is trained in two stages. The first stage of training uses the mean square error loss function, or MSE, which is defined as follows: The second stage uses the minimization of the negative log-likelihood loss function, or NLL, to train the model. NLL is defined as follows: S9.2: Training details This model is trained on the NVIDIA GeForce RTX 3060TI GPU based on the Pytorch deep learning framework. The optimizer selected is the Adam optimizer, and the learning rate is selected as 0.0005s.

Images