A vehicle trajectory prediction system and method based on LSTM
Through the combination of LSTM, graph convolution network and graph attention mechanism, the instability problem of vehicle trajectory prediction model in complex traffic scenarios is solved, and vehicle trajectory prediction with higher accuracy and reliability is achieved.
Patent Information
- Application Number
- CN202510379333.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-28
- Publication Date
- 2025-08-26
- Estimated Expiration
- 2045-03-28
AI Technical Summary
Existing vehicle trajectory prediction models are difficult to effectively utilize the intrinsic correlation between successive frames in complex traffic scenarios, resulting in prediction instability and time inconsistency, and fail to fully utilize surrounding vehicle information for accurate modeling.
The LSTM, graph convolution network and graph attention mechanism are used to screen important vehicle features through data preprocessing, and feature expression is enhanced by combining graph convolution and graph attention mechanisms, and dynamic relationships and temporal attention between vehicles are used for trajectory prediction.
It improves the accuracy and reliability of vehicle trajectory prediction, especially in complex traffic scenarios, which can better capture the time and space dependence and the mutual influence between vehicles, and improves the authenticity and accuracy of prediction.
Smart Images
Figure CN119888413B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to a vehicle trajectory prediction system and method based on LSTM, graph convolutional network, and graph attention mechanism. Background Art
[0002] With the rapid development of autonomous driving technology in recent years, vehicle trajectory prediction, as a core foundational technology, is crucial for ensuring reliable and trustworthy autonomous driving. Improving the accuracy and reliability of trajectory prediction models is a key challenge in ensuring autonomous driving safety. Many current methods typically rely on historical frames of fixed duration for prediction and fail to effectively exploit the inherent correlations between consecutive frames in the prediction of adjacent time steps, leading to potential instability and temporal inconsistency. Furthermore, modeling becomes particularly difficult in complex traffic scenarios, as the surrounding environment, including vehicles and pedestrians, significantly influences vehicle trajectory. Therefore, utilizing information about surrounding vehicles to model and predict a vehicle's future trajectory has become a thorny issue currently being researched and challenged by scholars. Summary of the Invention
[0003] In view of the above problems, the purpose of the present invention is to provide a vehicle trajectory prediction system and method based on LSTM, which utilizes LSTM and group convolutional neural networks and multiple attention mechanisms to extract the characteristics of the predicted vehicle and the impact of the surrounding environment on the vehicle, and ultimately predict the future trajectory of the vehicle.
[0004] The present invention provides a vehicle trajectory prediction system based on LSTM, graph convolutional network and graph attention mechanism, comprising the following steps:
[0005] Step S1: Data preprocessing: obtaining the original vehicle trajectory dataset, filtering out the feature text data required by the model, assigning trajectory categories, filtering edge cases, dividing the dataset into training, validation, and test sets, and generating image data for them;
[0006] Among them, the feature text data needs to filter out the surrounding vehicles with a distance greater than the set distance (40) in the original vehicle trajectory data set (vehicles that have little impact on the predicted target vehicle), and retain the surrounding vehicles with a distance less than the set distance in the vehicle trajectory data set; and perform a standardization operation on the vehicle trajectory data set to select the feature text data required by the model, including the data set ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position;
[0007] Step S2: Training the trajectory prediction model. The processed time series data is encoded and extracted using an LSTM (long short-term memory network) encoder. The features are then fed into the graph convolution and graph attention mechanism modules to enhance the feature representation and perform feature fusion. The decoder then generates the future vehicle trajectory.
[0008] Step S3: comparative experiment, verifying the validity of the amplification data through comparative experiment.
[0009] As a preferred embodiment of the present invention, step S1 further includes the following steps:
[0010] Step S11: preprocess the original vehicle trajectory dataset and normalize the data using MATLAB software to ensure that the input meets the model requirements and generate image data;
[0011] Step S12: Data loading and processing, loading the vehicle trajectory dataset, assigning a unique identifier to each data file, extracting the required columns, and performing preliminary processing and cleaning on the data;
[0012] Step S13: Data structure construction, using a Map container to store and index vehicle trajectory data according to vehicle ID and time frame for subsequent data access and processing;
[0013] Step S14: Field parsing and feature extraction, by traversing the data of each data set, calculating the longitudinal position and lateral position maneuver type, determining the position of the vehicle in the grid, and storing the feature information in the trajectory data;
[0014] Step S15: Dataset segmentation: segmenting the processed vehicle trajectory dataset into a training set, a validation set, and a test set according to a predefined ratio;
[0015] Step S16: Edge case filtering: edge case detection and filtering are performed on the trajectory data in the training set, validation set, and test set, retaining the trajectory data within the valid time range and excluding data points outside the valid time range;
[0016] Step S17: Dataset saving: save the processed, segmented, and filtered training, validation, and test data sets as .mat files.
[0017] Step S18: Training. Each training cycle is divided into two phases. The pre-training phase uses the MSE loss function to train the model, with the goal of accelerating the model's initial learning of future trajectories. The formal training phase switches to the NLL loss function training and adds a classification loss for control behaviors. The goal is to optimize the accuracy of trajectory prediction and the classification accuracy of control behaviors. Control behaviors include four maneuver categories: left turn, right turn, straight ahead, and stationary.
[0018] As a preference of the present invention, step S2 further includes the following steps:
[0019] Step S21: Input the image data filtered in step S1 into the LSTM encoder for encoding and extracting vehicle trajectory features. Different weights are assigned according to the interaction relationship between vehicle nodes to construct a graph structure. The extracted features are placed in the graph convolution and graph attention mechanism modules to process and filter out surrounding vehicles whose impact on the predicted vehicle's future trajectory is greater than the set threshold (little impact, the set threshold is 0.3), and retain the features of vehicles whose impact is less than the set threshold (large impact). At the same time, the image data is processed by VIT (Vision Transformer) to extract features, and then the features extracted from the feature text data are fused using the multi-head attention layer (MultiHeadSelfAttention) to obtain the trajectory encoding. The softmax function is used to assign probabilities to the possible future trajectories of the vehicle, and the decoder is used to generate multiple future trajectories. The output result is each possible trajectory and the probability of the trajectory.
[0020] As a preference of the present invention, step S3 further includes the following steps:
[0021] Step S31: Use the RMSE indicator to evaluate the model obtained in step S2 and the models obtained by the MultiScaleSEBlocke method and the CS-LSTM method to verify the effectiveness of the vehicle trajectory prediction method based on LSTM, graph convolutional network and graph attention mechanism.
[0022] Another object of the present invention is to provide a vehicle trajectory prediction system based on LSTM, graph convolution network and graph attention mechanism, comprising: a data preprocessing module, an LSTM and graph convolution graph attention module and a comparative experiment module;
[0023] The data preprocessing module is used to process the original vehicle trajectory dataset, filter the original vehicle trajectory dataset to find the features required by the model, assign trajectory categories, filter edge cases, divide the dataset into training sets, validation sets, and test sets, and generate image data for them. The features required by the model include dataset ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position.
[0024] The LSTM and graph convolution and graph attention module includes: an LSTM encoder, a decoder, and a graph convolution and graph attention mechanism module. The LSTM encoder is used to encode the processed time series data and extract features, and pass them into the graph convolution and graph attention mechanism module. The graph convolution and graph attention mechanism module is used to enhance feature expression and perform feature fusion. The decoder is used to generate the trajectory of future vehicles.
[0025] The comparative experiment module is used to use the same data set under the same equipment and configuration to conduct comparative experiments to verify the improvement effect of using LSTM and graph convolution graph attention module and not using LSTM and graph convolution graph attention module on the prediction results.
[0026] The beneficial effects of the present invention are as follows:
[0027] 1. This invention improves the LSTM architecture by encoding the dynamic relationships between a vehicle's historical trajectory and neighboring vehicles, enhancing its ability to capture spatiotemporal dependencies in complex traffic scenarios. Furthermore, a temporal attention mechanism is incorporated to weight historical trajectories, filtering out information from time steps that have a significant impact on trajectory prediction and avoiding redundant information. A multi-head self-attention layer is also added between each module to further filter out features that are important for trajectory prediction.
[0028] 2. The present invention has achieved better results in vehicle trajectory prediction by comparing the vehicle trajectory prediction results with the same data set and the same equipment, with a smaller RMSE index value. The results are even better in long-term prediction experiments.
[0029] 3. During training, the LSTM model uses a loss based on trajectory prediction errors (e.g., negative log-likelihood loss (NLL)). The graph attention mechanism optimizes the model's predictive capabilities by assigning weights to each edge and learning its influence. By jointly training these two components, the model retains the motion patterns of individual vehicles while leveraging the interactions between them, thereby improving the realism and accuracy of predicted trajectories.
[0030] 4. This invention uses MSE (mean squared error) loss for pre-training and switches to NLL (negative log-likelihood) loss during formal training. Cross-entropy loss (BCELoss) is also used for the classification of control behaviors. The training process records training and validation losses, and uses the Adam optimizer for parameter updates.
[0031] 5. The present invention uses graph attention and graph convolutional networks to learn and predict the adjacency relationship between the target vehicle and surrounding vehicles, so that the model can better learn the impact of surrounding vehicles on the target vehicle trajectory during prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] By referring to the following description in conjunction with the accompanying drawings, and with a more complete understanding of the present invention, other objects and results of the present invention will become more clear and easy to understand. In the accompanying drawings:
[0033] Figure 1 Schematic diagram of the improved LSTM graph convolutional network and graph attention mechanism structure in the present invention;
[0034] Figure 2 This is a schematic diagram of a sample of the original vehicle trajectory text data in the present invention;
[0035] Figure 3 This is a schematic diagram of the RMSE experimental results of the comparative test in the present invention;
[0036] Figure 4 Schematic diagram of the RMSE experimental results of the ablation test in the present invention; DETAILED DESCRIPTION
[0037] Example 1
[0038] See Figure 1-4 This embodiment proposes a vehicle trajectory prediction system based on LSTM, graph convolutional network and graph attention mechanism, including the following steps:
[0039] Step S1: Data preprocessing: obtaining the original vehicle trajectory dataset, filtering out the feature text data required by the model, assigning trajectory categories, filtering edge cases, dividing the dataset into training, validation, and test sets, and generating image data for them;
[0040] Among them, the feature text data needs to filter out the surrounding vehicles with a distance greater than the set distance (40) in the original vehicle trajectory data set (vehicles that have little impact on the predicted target vehicle), and retain the surrounding vehicles with a distance less than the set distance in the vehicle trajectory data set; and perform a standardization operation on the vehicle trajectory data set to select the feature text data required by the model, including the data set ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position;
[0041] Step S11: preprocess the original vehicle trajectory dataset and normalize the data using MATLAB software to ensure that the input meets the model requirements and generate image data;
[0042] Step S12: Data loading and processing, loading the vehicle trajectory dataset, assigning a unique identifier to each data file, extracting the required columns, and performing preliminary processing and cleaning on the data;
[0043] Step S13: Data structure construction, using a Map container to store and index vehicle trajectory data according to vehicle ID and time frame for subsequent data access and processing;
[0044] Step S14: Field parsing and feature extraction, by traversing the data of each data set, calculating the longitudinal position and lateral position maneuver type, determining the position of the vehicle in the grid, and storing the feature information in the trajectory data;
[0045] Step S15: Dataset segmentation: segmenting the processed vehicle trajectory dataset into a training set, a validation set, and a test set according to a predefined ratio;
[0046] Step S16: Edge case filtering: edge case detection and filtering are performed on the trajectory data in the training set, validation set, and test set, retaining the trajectory data within the valid time range and excluding data points outside the valid time range;
[0047] Step S17: Dataset saving: save the processed, segmented, and filtered training, validation, and test data sets as .mat files.
[0048] Step S18: Training. Each training cycle is divided into two phases. The pre-training phase uses the MSE loss function to train the model, with the goal of accelerating the model's initial learning of future trajectories. The formal training phase switches to the NLL loss function training and adds a classification loss for control behavior. The goal is to optimize the accuracy of trajectory prediction and the classification accuracy of control behavior.
[0049] Step S2: Training the trajectory prediction model. The processed time series data is encoded and extracted using the LSTM encoder. The features are then passed to the graph convolution and graph attention mechanism modules to enhance feature expression and perform feature fusion. The future vehicle trajectory is then generated using the decoder.
[0050] Step S21: Input the image data filtered in step S1 into the LSTM encoder for encoding and extracting vehicle trajectory features. Different weights are assigned according to the interaction relationship between vehicle nodes to construct a graph structure. The extracted features are placed in the graph convolution and graph attention mechanism modules to filter out surrounding vehicles that have little impact on the prediction of the vehicle's future trajectory, and retain the features of vehicles with greater impact. At the same time, the image data is processed by VIT (Vision Transformer) to extract features, and then the features extracted from the feature text data are fused using the multi-head attention layer (MultiHeadSelfAttention) to obtain the trajectory encoding. The softmax function is used to assign probabilities to the vehicle's possible future trajectories, and the decoder is used to generate multiple future trajectories. The output result is each possible trajectory and the probability of the trajectory.
[0051] Step S3: comparative experiment, verifying the validity of the amplification data through comparative experiment.
[0052] Step S31: Use the RMSE indicator to evaluate the model obtained in step S2 and the models obtained by the MultiScaleSEBlocke method and the CS-LSTM method to verify the effectiveness of the vehicle trajectory prediction method based on LSTM, graph convolutional network and graph attention mechanism.
[0053] Example 2
[0054] This embodiment provides a vehicle trajectory prediction system based on LSTM, graph convolution network and graph attention mechanism, including: a data preprocessing module, an LSTM and graph convolution graph attention module and a comparative experiment module;
[0055] The data preprocessing module is used to process the original vehicle trajectory dataset, filter the original vehicle trajectory dataset to find the features required by the model, assign trajectory categories, filter edge cases, divide the dataset into training sets, validation sets, and test sets, and generate image data for them. The features required by the model include dataset ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position.
[0056] The LSTM and graph convolution and graph attention module includes: an LSTM encoder, a decoder, and a graph convolution and graph attention mechanism module. The LSTM encoder is used to encode the processed time series data and extract features, and pass them into the graph convolution and graph attention mechanism module. The graph convolution and graph attention mechanism module is used to enhance feature expression and perform feature fusion. The decoder is used to generate the trajectory of future vehicles.
[0057] The comparative experiment module is used to use the same data set under the same equipment and configuration to conduct comparative experiments to verify the improvement effect of using LSTM and graph convolution graph attention module and not using LSTM and graph convolution graph attention module on the prediction results.
[0058] Example 3
[0059] See Figure 1This embodiment introduces an encoder and decoder architecture using an LSTM (long short-term memory network) graph convolutional network and a graph attention mechanism module for multimodal vehicle prediction. The LSTM encoder and decoder are used to encode vehicle trajectory information and decode it to generate predicted future vehicle trajectories. The raw vehicle trajectory dataset, after data preprocessing, is encoded using the LSTM encoder and a social mask is used to clarify the vehicle's position within the grid. The historical trajectories of neighboring vehicles and the target vehicle are encoded and passed into the graph convolution and graph attention mechanism module to generate an adjacency matrix to represent adjacency relationships in the traffic scene. Features of surrounding vehicles that have a significant impact on the target vehicle are selected and then fused with features from the vehicle's historical trajectory and image data extracted using VIT to enhance feature representation. A multi-head attention mechanism is used to select important information, filter redundant information, and improve model computational efficiency. The trajectory encoding is then passed through a softmax function to generate trajectory category probabilities. The LSTM decoder generates future trajectories for each of the four trajectory categories and combines these with the probabilities generated by the softmax function to ultimately generate a multimodal trajectory for the target vehicle.
[0060] See Figure 2 ,First, the original vehicle trajectory dataset is preprocessed and the data is ,normalized using Matlab software to ensure that the input meets ,the model requirements and generates image data;
[0061] Data loading and processing: load the vehicle trajectory dataset, assign a unique identifier to each data file, extract the required columns, and perform preliminary processing and cleaning of the data;
[0062] Data structure construction: Using Map containers to store and index vehicle trajectory data by vehicle ID and time frame for subsequent data access and processing;
[0063] Field parsing and feature extraction, by traversing the data of each dataset, calculating the longitudinal position and lateral position maneuver type, determining the position of the vehicle in the grid, and storing the feature information in the trajectory data;
[0064] Dataset segmentation: split the processed vehicle trajectory dataset into training set, validation set, and test set according to a predefined ratio;
[0065] Edge case filtering: edge case detection and filtering of trajectory data in the training set, validation set, and test set, retaining trajectory data within the valid time range and excluding data points outside the valid time range;
[0066] Save the dataset and save the processed, segmented and filtered training set, validation set and test set data as .mat files;
[0067] In the training phase, each training cycle is divided into two stages. The pre-training phase uses the MSE loss function to train the model, with the goal of accelerating the model's initial learning of future trajectories. The formal training phase switches to the NLL loss function training and adds the classification loss of the control behavior, with the goal of optimizing the accuracy of trajectory prediction and the classification accuracy of the control behavior.
[0068] A trajectory prediction model is designed, using an LSTM to learn the historical trajectories of individual vehicles and extract their motion features. Through a multi-layer LSTM network, the model effectively captures the vehicle's motion patterns and time series information. The graph attention mechanism further enhances the modeling of vehicle interactions based on the LSTM. By constructing a vehicle graph, the graph attention mechanism assigns different attention weights to each vehicle's neighbors, helping the model focus on the neighboring vehicles most relevant to the target vehicle's trajectory. This allows the model to effectively leverage the spatial relationships between vehicles to improve trajectory prediction accuracy. Features extracted by the graph attention mechanism are combined with features extracted using the Vision Transformer (VIT). These features are then fused and filtered with features extracted from text data using a multi-head self-attention layer. An LSTM decoder is used to generate vehicle trajectories.
[0069] During training, the LSTM model's loss is based on trajectory prediction errors (e.g., negative log-likelihood loss (NLL)). The graph attention mechanism optimizes the model's prediction capabilities by assigning weights to each edge and learning its influence. By jointly training these two components, the model preserves the motion patterns of individual vehicles while leveraging the interactions between them, thereby improving the realism and accuracy of predicted trajectories.
[0070] See Figure 3 and Figure 4 , the RMSE indicator is used to evaluate the prediction effects of multiple methods and the model in Example 1 to verify the prediction effect of the model. Through comparative experiments, the improvement effect of using LSTM and graph convolution graph attention modules on the prediction results is verified. Specifically, the method GA+MSA proposed in Example 1 and the CS-LSTM method and MultiScaleSEBlocke method in other methods are experimented with using the same data set under the same equipment and configuration. The MultiScaleSEBlocke method in this embodiment is a CNN-LSTM prediction method based on the residual attention mechanism. The CS-LSTM method is a type of CNN-LSTM method, which consists of a CNN convolutional neural network and a social pooling mechanism plus an LSTM encoder and an LSTM decoder. It introduces SEBLOCK (residual attention mechanism module) on the basis of CS-LSTM. By Figure 3 and Figure 4 The comparative experiments and ablation experiments in the paper show that the vehicle trajectory prediction method proposed in Example 1 performs better in the comparative experiments and has a more obvious advantage in long-term prediction.
[0071] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.
Claims
1. A vehicle trajectory prediction method based on LSTM, characterized in that: The following steps are involved: Step S1: Data preprocessing: obtaining the original vehicle trajectory dataset, filtering out the feature text data required by the model, assigning trajectory categories, filtering edge cases, dividing the dataset into training, validation, and test sets, and generating image data for them; The feature text data needs to filter out surrounding vehicles in the original vehicle trajectory dataset that are greater than the set distance, and retain surrounding vehicles in the vehicle trajectory dataset that are less than the set distance. The vehicle trajectory dataset is then standardized to select the feature text data required by the model, including dataset ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position. Step S2: Training the trajectory prediction model. The processed time series data is encoded and extracted using the LSTM encoder. The features are then passed to the graph convolution and graph attention mechanism modules to enhance feature expression and perform feature fusion. The future vehicle trajectory is then generated using the decoder. Step S21: Input the feature text data filtered in step S1 into the LSTM encoder for encoding and extracting vehicle trajectory features. Different weights are assigned according to the interaction relationship between vehicle nodes to construct a graph structure. The extracted features are placed in the graph convolution and graph attention mechanism modules to filter out surrounding vehicles whose predicted future trajectory is greater than a set threshold, and retain the features of vehicles whose trajectory is less than the set threshold. At the same time, the image data is processed by VIT to extract features, and then the features extracted from the feature text data are fused using a multi-head attention layer to obtain a trajectory encoding. The softmax function is used to assign probabilities to the possible future trajectories of the vehicle. Multiple future trajectories are generated by the decoder, and the output result is each possible trajectory and the probability of the trajectory. Step S3: comparative experiment, verifying the validity of the amplification data through comparative experiment.
2. The vehicle trajectory prediction method based on LSTM according to claim 1, further comprising the following steps in step S1: Step S11: preprocess the original vehicle trajectory dataset and normalize the data using MATLAB software to ensure that the input meets the model requirements and generate image data; Step S12: Data loading and processing, loading the vehicle trajectory dataset, assigning a unique identifier to each data file, extracting the required columns, and performing preliminary processing and cleaning on the data; Step S13: Data structure construction, using a Map container to store and index vehicle trajectory data according to vehicle ID and time frame for subsequent data access and processing; Step S14: Field parsing and feature extraction, by traversing the data of each data set, calculating the longitudinal position and lateral position maneuver type, determining the position of the vehicle in the grid, and storing the feature information in the trajectory data; Step S15: Dataset segmentation: segmenting the processed vehicle trajectory dataset into a training set, a validation set, and a test set according to a predefined ratio; Step S16: Edge case filtering: edge case detection and filtering are performed on the trajectory data in the training set, validation set, and test set, retaining the trajectory data within the valid time range and excluding data points outside the valid time range; Step S17: Dataset saving: save the processed, segmented, and filtered training, validation, and test data sets as .mat files. Step S18: Training part. Each training cycle is divided into two stages. The pre-training stage uses the MSE loss function to train the model, with the goal of accelerating the model's initial learning of future trajectories. During the formal training phase, the NLL loss function is used to train the model, while adding a classification loss for control behaviors. The goal is to optimize the accuracy of trajectory prediction and the classification accuracy of control behaviors. Control behaviors include four maneuver categories: left turn, right turn, straight ahead, and stationary.
3. The vehicle trajectory prediction method based on LSTM according to claim 1, further comprising the following steps in step S3: Step S31: Use the RMSE indicator to evaluate the model obtained in step S2 and the models obtained by the MultiScaleSEBlocke method and the CS-LSTM method to verify the effectiveness of the vehicle trajectory prediction method based on LSTM, graph convolutional network and graph attention mechanism.
4. A vehicle trajectory prediction system based on LSTM, characterized in that: include: Data preprocessing module, LSTM and graph convolution and attention module, and comparative experiment module; The data preprocessing module is used to process the original vehicle trajectory dataset, filter the original vehicle trajectory dataset to obtain the feature text data required by the model, assign trajectory categories, filter edge cases, divide the dataset into training sets, validation sets, and test sets, and generate image data for them. The feature text data required by the model includes dataset ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position. The LSTM and graph convolution graph attention module includes: an LSTM encoder, a decoder, and a graph convolution and graph attention mechanism module. The LSTM encoder is used to encode the processed time series data to extract features and pass it into the graph convolution and graph attention mechanism module. The graph convolution and graph attention mechanism module is used to enhance feature expression and perform feature fusion. The decoder is used to generate the trajectory of the future vehicle; specifically, the filtered feature text data is input into the LSTM encoder for encoding and extracting vehicle trajectory features, different weights are assigned according to the interaction relationship between vehicle nodes to construct a graph structure, the extracted features are put into the graph convolution and graph attention mechanism module for processing to filter out surrounding vehicles whose predicted future trajectory is greater than a set threshold, and retain the features of vehicles whose future trajectory is less than the set threshold. At the same time, the image data is processed by VIT to extract features, and then the features extracted from the feature text data are fused using a multi-head attention layer to obtain a trajectory encoding, assigning probabilities to possible future trajectories of the vehicle through a softmax function, generating multiple future trajectories through a decoder, and outputting each possible trajectory and the probability of the trajectory. The comparative experiment module is used to use the same data set under the same equipment and configuration to conduct comparative experiments to verify the improvement effect of using LSTM and graph convolution graph attention module and not using LSTM and graph convolution graph attention module on the prediction results.