Vehicle track prediction system and method based on LSTM (Long Short Term Memory)

Abstract

The invention relates to the field of artificial intelligence, in particular to a vehicle trajectory prediction system and method based on LSTM, and the method comprises the following steps: S1, data preprocessing: obtaining an original vehicle trajectory data set, screening out feature text data required by a model, allocating trajectory categories, filtering edge conditions, and dividing a training set, a verification set and a test set, generating image data for the image data; s2, training a trajectory prediction method model, encoding the processed time series data through an LSTM encoder to extract features, transmitting the features to a graph convolution and graph attention mechanism module to enhance feature expression, performing feature fusion, and generating a trajectory of a future vehicle through a decoder; and S3, performing a contrast experiment, and verifying the validity of the amplified data through the contrast experiment. The method has the advantages that the adjacency relation of the vehicles in the complex traffic scene is better captured by introducing graph convolution and a graph attention mechanism, and therefore the precision of vehicle track prediction in the complex traffic scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a vehicle trajectory prediction system and method based on LSTM, graph convolutional network and graph attention mechanism. Background Art

[0002] With the rapid development of autonomous driving technology in recent years, vehicle trajectory prediction, as the core basic technology of autonomous driving technology, is necessary to make autonomous driving reliable and trustworthy. How to improve the accuracy and reliability of trajectory prediction models is a key challenge to ensure the safety of autonomous driving. Many current methods usually rely on historical frames of fixed length for prediction, and fail to effectively utilize the inherent correlation between consecutive frames in the prediction of adjacent time steps, resulting in potential instability and time inconsistency. In complex traffic scenarios, modeling becomes very difficult, and the influence of the surrounding environment, including vehicles and pedestrians, on the vehicle trajectory is very important. How to use the information of surrounding vehicles to model and predict the future trajectory of the vehicle has become a thorny problem that scholars are now studying and challenging. Summary of the invention

[0003] In view of the above problems, the purpose of the present invention is to provide a vehicle trajectory prediction system and method based on LSTM, which utilizes LSTM and group convolutional neural networks and multiple attention mechanisms to extract the characteristics of the predicted vehicle and the impact of the surrounding environment on the vehicle, and finally predict the future trajectory of the vehicle.

[0004] The present invention provides a vehicle trajectory prediction system based on LSTM, graph convolution network and graph attention mechanism, comprising the following steps: Step S1: Data preprocessing, obtaining the original vehicle trajectory data set, filtering out the feature text data required by the model, assigning trajectory categories, filtering edge cases, dividing the training set, validation set and test set, and generating image data for them; Among them, the feature text data needs to filter out the surrounding vehicles whose distance in the original vehicle trajectory data set is greater than the set distance (40) (vehicles that have little impact on the predicted target vehicle), and retain the surrounding vehicles whose distance in the vehicle trajectory data set is less than the set distance; and standardize the vehicle trajectory data set to select the feature text data required by the model, including data set ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position; Step S2: Training of trajectory prediction method model: encoding the processed time series data through LSTM (long short-term memory network) encoder to extract features, passing them into graph convolution and graph attention mechanism modules to enhance feature expression, perform feature fusion, and then generate the future vehicle trajectory through the decoder; Step S3: comparative experiment, verifying the validity of the amplification data through comparative experiment.

[0005] As a preferred embodiment of the present invention, step S1 also includes the following steps: Step S11: preprocess the original vehicle trajectory data set and normalize the data using Matlab software to ensure that the input meets the model requirements and generate image data; Step S12: Data loading and processing, loading the vehicle trajectory dataset, assigning a unique identifier to each data file, extracting the required columns, and performing preliminary processing and cleaning of the data; Step S13: Data structure construction, by using a Map container to store and index vehicle trajectory data according to vehicle ID and time frame for subsequent data access and processing; Step S14: Field parsing and feature extraction, by traversing the data of each data set, calculating the longitudinal position and lateral position maneuver type, determining the position of the vehicle in the grid, and storing the feature information in the trajectory data; Step S15: data set segmentation, segmenting the processed vehicle trajectory data set into a training set, a validation set, and a test set according to a predefined ratio; Step S16: edge case filtering, edge case detection and filtering of trajectory data in the training set, validation set and test set, retaining trajectory data within the valid time range, and excluding data points outside the valid time range; Step S17: Data set saving, saving the processed, segmented and filtered training set, validation set and test set data as .mat files; Step S18: In the training part, each training cycle is divided into two stages. In the pre-training stage, the MSE loss function is used to train the model, with the goal of accelerating the model's initial learning of future trajectories. In the formal training stage, the NLL loss function is switched to training, and the classification loss of the control behavior is added at the same time. The goal is to optimize the accuracy of trajectory prediction and the classification accuracy of the control behavior. The control behavior includes four maneuver categories: left turn, right turn, straight driving and stationary.

[0006] As a preferred embodiment of the present invention, step S2 also includes the following steps: Step S21: Input the image data filtered in step S1 into the LSTM encoder for encoding and extracting vehicle trajectory features. Different weights are assigned according to the interaction relationship between vehicle nodes to build a graph structure. The extracted features are placed in the graph convolution and graph attention mechanism modules to process and filter out surrounding vehicles whose predicted future trajectory is greater than the set threshold (little impact, the threshold is set to 0.3), and retain the features of vehicles whose impact is less than the set threshold (large impact). At the same time, the image data is processed by VIT (Vision Transformer) to extract features, and then the features extracted from the feature text data are fused using a multi-head attention layer (MultiHeadSelfAttention) to obtain the trajectory encoding. The softmax function is used to assign probabilities to the possible future trajectories of the vehicle, and multiple future trajectories are generated through the decoder. The output result is each possible trajectory and the probability of the trajectory.

[0007] As a preferred embodiment of the present invention, step S3 also includes the following steps: Step S31: Use the RMSE indicator to evaluate the model obtained in step S2 and the models obtained by the MultiScaleSEBlocke method and the CS-LSTM method to verify the effectiveness of the vehicle trajectory prediction method based on LSTM, graph convolutional network and graph attention mechanism.

[0008] Another object of the present invention is to provide a vehicle trajectory prediction system based on LSTM, graph convolution network and graph attention mechanism, comprising: a data preprocessing module, an LSTM and graph convolution graph attention module and a comparative experiment module; The data preprocessing module is used to process the original vehicle trajectory data set, filter out the features required by the model from the original vehicle trajectory data set, assign trajectory categories, filter edge cases, divide the training set, validation set and test set, and generate image data for them, wherein the features required by the model include data set ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position; The LSTM and graph convolution graph attention module includes: an LSTM encoder, a decoder, and a graph convolution and graph attention mechanism module. The LSTM encoder is used to encode the processed time series data to extract features and pass them into the graph convolution and graph attention mechanism module. The graph convolution and graph attention mechanism module is used to enhance feature expression and perform feature fusion. The decoder is used to generate the trajectory of future vehicles.

[0009] The comparative experiment module is used to use the same data set under the same equipment and configuration to conduct comparative experiments to verify the improvement effect of using LSTM and graph convolution graph attention module and not using LSTM and graph convolution graph attention module on the prediction results.

[0010] The beneficial effects of the present invention are as follows: 1. This invention improves LSTM and enhances the ability to capture spatiotemporal dependencies in complex traffic scenarios by encoding the dynamic relationship between the vehicle's historical trajectory and neighboring vehicles. In addition, the temporal attention mechanism is combined to perform weighted processing on the historical trajectory, filter out the information of the time step that has a greater impact on trajectory prediction, and avoid redundant information. A multi-head attention layer (MultiHeadSelfAttention) is also added between each module to further filter out features that are important for trajectory prediction.

[0011] 2. The present invention uses comparative tests to predict vehicle trajectories under the same data set and the same equipment, and the RMSE index value is small, and the effect is better. The effect is better in the long-term prediction experiment.

[0012] 3. During the training process, the loss of the LSTM model is based on the prediction error of the trajectory (such as negative log-likelihood loss NLL), while the graph attention mechanism optimizes the prediction ability of the model by assigning weights to each edge and learning the influence of the edge. By jointly training these two parts, the model can fully utilize the mutual influence between vehicles while retaining the movement rules of a single vehicle, thereby improving the authenticity and accuracy of the predicted trajectory.

[0013] 4. The present invention uses MSE (mean square error) loss for pre-training, and switches to NLL (negative log likelihood) loss in the formal training stage, while using cross entropy loss (BCELoss) for the classification results of the control behavior. The training process records the training and validation losses, and uses the Adam optimizer for parameter update.

[0014] 5. The present invention uses graph attention and graph convolutional networks to learn and predict the adjacency relationship between the target vehicle and surrounding vehicles, so that the model can better learn the impact of surrounding vehicles on the trajectory of the target vehicle during prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings: Figure 1 It is a schematic diagram of the structure of the improved LSTM graph convolutional network and graph attention mechanism in the present invention; Figure 2 A schematic diagram of a portion of the sample original vehicle trajectory text data in the present invention; Figure 3 This is a schematic diagram of the RMSE experimental results of the comparative test in the present invention; Figure 4 This is a schematic diagram of the RMSE experimental results of the ablation test in the present invention; DETAILED DESCRIPTION

[0016] Example 1

[0017] See also Figure 1-4 , this embodiment proposes a vehicle trajectory prediction system based on LSTM, graph convolution network and graph attention mechanism, including the following steps: Step S1: Data preprocessing, obtaining the original vehicle trajectory data set, filtering out the feature text data required by the model, assigning trajectory categories, filtering edge cases, dividing the training set, validation set and test set, and generating image data for them; Among them, the feature text data needs to filter out the surrounding vehicles whose distance in the original vehicle trajectory data set is greater than the set distance (40) (vehicles that have little impact on the predicted target vehicle), and retain the surrounding vehicles whose distance in the vehicle trajectory data set is less than the set distance; and standardize the vehicle trajectory data set to select the feature text data required by the model, including data set ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position; Step S11: preprocess the original vehicle trajectory data set and normalize the data using Matlab software to ensure that the input meets the model requirements and generate image data; Step S12: Data loading and processing, loading the vehicle trajectory dataset, assigning a unique identifier to each data file, extracting the required columns, and performing preliminary processing and cleaning of the data; Step S13: Data structure construction, by using a Map container to store and index vehicle trajectory data according to vehicle ID and time frame for subsequent data access and processing; Step S14: Field parsing and feature extraction, by traversing the data of each data set, calculating the longitudinal position and lateral position maneuver type, determining the position of the vehicle in the grid, and storing the feature information in the trajectory data; Step S15: data set segmentation, segmenting the processed vehicle trajectory data set into a training set, a validation set, and a test set according to a predefined ratio; Step S16: edge case filtering, edge case detection and filtering of trajectory data in the training set, validation set and test set, retaining trajectory data within the valid time range, and excluding data points outside the valid time range; Step S17: Data set saving, saving the processed, segmented and filtered training set, validation set and test set data as .mat files; Step S18: In the training part, each training cycle is divided into two stages. In the pre-training stage, the MSE loss function is used to train the model, with the goal of accelerating the model's initial learning of future trajectories. In the formal training stage, the NLL loss function is switched to training, and the classification loss of the control behavior is added at the same time, with the goal of optimizing the accuracy of trajectory prediction and the classification accuracy of the control behavior.

[0018] Step S2: Training of the trajectory prediction method model. The processed time series data is encoded and extracted through the LSTM encoder, and then passed to the graph convolution and graph attention mechanism modules to enhance the feature expression, perform feature fusion, and then generate the future vehicle trajectory through the decoder; Step S21: Input the image data filtered in step S1 into the LSTM encoder for encoding and extracting vehicle trajectory features. Different weights are assigned according to the interaction relationship between vehicle nodes to build a graph structure. The extracted features are placed in the graph convolution and graph attention mechanism modules to filter out surrounding vehicles that have little impact on the prediction of the vehicle's future trajectory, and retain the features of vehicles with greater impact. At the same time, the image data is processed by VIT (Vision Transformer) to extract features, and then the features extracted from the feature text data are fused using a multi-head attention layer (MultiHeadSelfAttention) to obtain the trajectory encoding. The softmax function is used to assign probabilities to the vehicle's possible future trajectories, and multiple future trajectories are generated through the decoder. The output result is each possible trajectory and the probability of the trajectory.

[0019] Step S3: comparative experiment, verifying the validity of the amplification data through comparative experiment.

[0020] Step S31: Use the RMSE indicator to evaluate the model obtained in step S2 and the models obtained by the MultiScaleSEBlocke method and the CS-LSTM method to verify the effectiveness of the vehicle trajectory prediction method based on LSTM, graph convolutional network and graph attention mechanism.

[0021] Example 2

[0022] This embodiment provides a vehicle trajectory prediction system based on LSTM, graph convolution network and graph attention mechanism, including: a data preprocessing module, an LSTM and graph convolution graph attention module and a comparative experiment module; The data preprocessing module is used to process the original vehicle trajectory data set, filter out the features required by the model from the original vehicle trajectory data set, assign trajectory categories, filter edge cases, divide the training set, validation set and test set, and generate image data for them, wherein the features required by the model include data set ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position; The LSTM and graph convolution graph attention module includes: an LSTM encoder, a decoder, and a graph convolution and graph attention mechanism module. The LSTM encoder is used to encode the processed time series data to extract features and pass them into the graph convolution and graph attention mechanism module. The graph convolution and graph attention mechanism module is used to enhance feature expression and perform feature fusion. The decoder is used to generate the trajectory of future vehicles.

[0023] The comparative experiment module is used to use the same data set under the same equipment and configuration to conduct comparative experiments to verify the improvement effect of using LSTM and graph convolution graph attention module and not using LSTM and graph convolution graph attention module on the prediction results.

[0024] Example 3

[0025] See also Figure 1 This embodiment introduces an LSTM (long short-term memory network) graph convolution network and a graph attention mechanism module to predict the encoder and decoder structure of a multimodal prediction vehicle. The LSTM encoder and LSTM decoder are used to encode vehicle trajectory information and decode to generate predicted future vehicle trajectories. The original vehicle trajectory dataset after data preprocessing is encoded using the LSTM encoder and the social mask is used to clarify the position of the vehicle in the grid. The neighbor vehicle historical trajectory and the target vehicle historical trajectory are encoded and passed into the graph convolution and graph attention mechanism module to generate an adjacency matrix to represent the adjacency relationship in the traffic scene. The surrounding vehicle features that have a greater impact on the target vehicle are screened out, and then fused with the features in the historical trajectory of the vehicle and the image data extracted using VIT to enhance the feature expression. The multi-head attention mechanism is used to screen important information, filter redundant information, and improve the model calculation efficiency. The trajectory encoding generates the probability of the trajectory category through the softmax function. The future trajectories are generated for the four trajectory categories through the LSTM decoder and combined with the probabilities generated by the softmax function to finally generate the multimodal vehicle trajectory of the target vehicle.

[0026] See also Figure 2 ,First, the original vehicle trajectory data set is preprocessed and the data is normalized by ,MATLAB software to ensure that the input meets the ,model requirements and generate image data; Data loading and processing, load the vehicle trajectory dataset, assign a unique identifier to each data file, extract the required columns, and perform preliminary processing and cleaning of the data; Data structure construction, by using Map container to store and index vehicle trajectory data according to vehicle ID and time frame for subsequent data access and processing; Field parsing and feature extraction, by traversing the data of each data set, calculating the longitudinal position and lateral position maneuver type, determining the position of the vehicle in the grid, and storing the feature information in the trajectory data; Dataset segmentation: the processed vehicle trajectory dataset is segmented into training set, validation set and test set according to a predefined ratio; Edge case filtering: edge case detection and filtering of trajectory data in the training set, validation set, and test set, retaining trajectory data within the valid time range and excluding data points outside the valid time range; Dataset saving: save the processed, segmented and filtered training set, validation set and test set data as .mat files; In the training part, each training cycle is divided into two stages. In the pre-training stage, the MSE loss function is used to train the model, with the goal of accelerating the model's initial learning of future trajectories. In the formal training stage, the NLL loss function is switched to training, and the classification loss of the control behavior is added at the same time, with the goal of optimizing the accuracy of trajectory prediction and the classification accuracy of the control behavior. Among them, a trajectory prediction model is designed, and LSTM is used to learn the historical trajectory of a single vehicle and extract the vehicle's motion features. Through a multi-layer LSTM network, the model can effectively capture the vehicle's motion patterns and time series information. The graph attention mechanism further strengthens the interaction modeling between vehicles on the basis of LSTM. By constructing a graph structure of vehicles, the graph attention mechanism assigns different attention weights to each vehicle's neighboring vehicles, helping the model focus on the neighboring vehicles that are most relevant to the target vehicle's trajectory. This enables the model to effectively utilize the spatial relationship between vehicles to improve the accuracy of trajectory prediction. The features extracted by the graph attention mechanism are combined with VIT (Vision Transformer) to extract features, and then the features extracted from the text book data are combined and filtered using a multi-head attention layer (MultiHeadSelfAttention). The vehicle trajectory is generated using an LSTM decoder.

[0027] During the training process, the loss of the LSTM model is based on the prediction error of the trajectory (such as negative log-likelihood loss NLL), while the graph attention mechanism optimizes the model's prediction ability by assigning weights to each edge and learning the influence of the edge. By jointly training these two parts, the model can fully utilize the mutual influence between vehicles while retaining the movement rules of a single vehicle, thereby improving the authenticity and accuracy of the predicted trajectory.

[0028] See also Figure 3 and Figure 4, the RMSE indicator is used to evaluate the prediction effects of multiple methods and the model in Example 1 to verify the prediction effect of the model. Through comparative experiments, the improvement effect of using LSTM and graph convolution graph attention modules on the prediction results is verified. Specifically, the method GA+MSA proposed in Example 1 and the CS-LSTM method and MultiScaleSEBlocke method in other methods are experimented with using the same data set under the same equipment and configuration. The MultiScaleSEBlocke method in this embodiment is a CNN-LSTM prediction method based on the residual attention mechanism. The CS-LSTM method is a type of CNN-LSTM method, which consists of a CNN convolutional neural network and a social pooling mechanism plus an LSTM encoder and an LSTM decoder. It introduces SEBLOCK (residual attention mechanism module) on the basis of CS-LSTM. By Figure 3 and Figure 4 The comparative experiments and ablation experiments in the examples show that the vehicle trajectory prediction method proposed in Example 1 performs better in the comparative experiments, and its advantages in long-term prediction are more obvious.

[0029] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A vehicle trajectory prediction method based on LSTM, characterized in that: The following steps are involved: Step S1: Data preprocessing, obtaining the original vehicle trajectory data set, filtering out the feature text data required by the model, assigning trajectory categories, filtering edge cases, dividing the training set, validation set and test set, and generating image data for them; Among them, the feature text data needs to filter out the surrounding vehicles whose distance in the original vehicle trajectory data set is greater than the set distance, and retain the surrounding vehicles whose distance in the vehicle trajectory data set is less than the set distance; and standardize the vehicle trajectory data set to select the feature text data required by the model, including data set ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position; Step S2: Training of the trajectory prediction method model. The processed time series data is encoded and extracted through the LSTM encoder, and then passed to the graph convolution and graph attention mechanism modules to enhance the feature expression, perform feature fusion, and then generate the future vehicle trajectory through the decoder; Step S3: comparative experiment, verifying the validity of the amplification data through comparative experiment.

2. According to the LSTM-based vehicle trajectory prediction method of claim 1, step S1 further comprises the following steps: Step S11: preprocess the original vehicle trajectory data set and normalize the data using Matlab software to ensure that the input meets the model requirements and generate image data; Step S12: Data loading and processing, loading the vehicle trajectory dataset, assigning a unique identifier to each data file, extracting the required columns, and performing preliminary processing and cleaning of the data; Step S13: Data structure construction, by using a Map container to store and index vehicle trajectory data according to vehicle ID and time frame for subsequent data access and processing; Step S14: Field parsing and feature extraction, by traversing the data of each data set, calculating the longitudinal position and lateral position maneuver type, determining the position of the vehicle in the grid, and storing the feature information in the trajectory data; Step S15: data set segmentation, segmenting the processed vehicle trajectory data set into a training set, a validation set, and a test set according to a predefined ratio; Step S16: edge case filtering, edge case detection and filtering of trajectory data in the training set, validation set and test set, retaining trajectory data within the valid time range, and excluding data points outside the valid time range; Step S17: Data set saving, saving the processed, segmented and filtered training set, validation set and test set data as .mat files; Step S18: Training part, each training cycle is divided into two stages. The pre-training stage uses the MSE loss function to train the model, the goal is to accelerate the model's initial learning of future trajectories; In the formal training phase, the NLL loss function is used to train the model, and the classification loss of the control behavior is added. The goal is to optimize the accuracy of trajectory prediction and the classification accuracy of the control behavior. The control behavior includes four maneuver categories: left turn, right turn, straight driving, and stationary.

3. The vehicle trajectory prediction method based on LSTM according to claim 1 further comprises the following steps in step S2: Step S21: Input the image data filtered in step S1 into the LSTM encoder for encoding and extracting vehicle trajectory features. Different weights are assigned according to the interaction relationship between vehicle nodes to build a graph structure. The extracted features are placed in the graph convolution and graph attention mechanism modules to filter out surrounding vehicles whose predicted future trajectories are greater than the set threshold, and retain the features of vehicles whose trajectories are less than the set threshold. At the same time, the image data is processed by VIT to extract features, and then the features extracted from the feature text data are fused using a multi-head attention layer to obtain the trajectory encoding. The softmax function is used to assign probabilities to the possible future trajectories of the vehicle, and multiple future trajectories are generated through the decoder. The output result is each possible trajectory and the probability of the trajectory.

4. The vehicle trajectory prediction method based on LSTM according to claim 1, further comprising the following steps in step S3: Step S31: Use the RMSE indicator to evaluate the model obtained in step S2 and the models obtained by the MultiScaleSEBlocke method and the CS-LSTM method to verify the effectiveness of the vehicle trajectory prediction method based on LSTM, graph convolutional network and graph attention mechanism.

5. A vehicle trajectory prediction system based on LSTM, characterized in that: include: Data preprocessing module, LSTM and graph convolutional attention module, and comparative experiment module; The data preprocessing module is used to process the original vehicle trajectory data set, filter out the features required by the model from the original vehicle trajectory data set, assign trajectory categories, filter edge cases, divide the training set, validation set and test set, and generate image data for them, wherein the features required by the model include data set ID, vehicle ID, timestamp, lane number, longitudinal position, and lateral position; The LSTM and graph convolution graph attention module includes: an LSTM encoder, a decoder, and a graph convolution and graph attention mechanism module. The LSTM encoder is used to encode the processed time series data to extract features and pass them into the graph convolution and graph attention mechanism module. The graph convolution and graph attention mechanism module is used to enhance feature expression and perform feature fusion. The decoder is used to generate the trajectory of the future vehicle. The comparative experiment module is used to use the same data set under the same equipment and configuration to conduct comparative experiments to verify the improvement effect of using LSTM and graph convolution graph attention module and not using LSTM and graph convolution graph attention module on the prediction results.

Images