A vehicle trajectory prediction method based on three-stage attention mechanism

By introducing a three-stage attention mechanism into the trajectory prediction algorithm of autonomous driving vehicles and combining with the LSTM network, the problem of ignoring dynamic data in the existing technology is solved, and the performance and accuracy of vehicle trajectory prediction are improved. It is suitable for various road conditions and environments. .

CN114495036BActive Publication Date: 2024-12-03ZHEJIANG LEAPMOTOR TECH CO LTD

Patent Information

Application Number
CN202111651987.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-27
Publication Date
2024-12-03
Estimated Expiration
2041-12-27

AI Technical Summary

Technical Problem

In the prior art, the trajectory prediction algorithm of the autonomous driving vehicle only relies on the historical trajectory information of the target vehicle and its surrounding vehicles, ignores other dynamic data, and cannot fully express the vehicle's historical state, resulting in insufficient prediction performance.

Method used

Using a vehicle trajectory prediction method based on a three-stage attention mechanism, the most suitable trajectory prediction information is extracted through the attention mechanism at the feature level, timing level and spatial level, including the feature attention module, timing level and spatial attention. The module, combined with the LSTM network, builds a vehicle trajectory prediction model and uses a negative log-likelihood loss function for training.

Benefits of technology

It improves the performance and accuracy of the vehicle trajectory prediction algorithm, is suitable for a variety of road conditions and environment scenarios, and enhances the interpretability and real-time prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114495036B_ABST
    Figure CN114495036B_ABST
Patent Text Reader

Abstract

The present invention discloses a vehicle trajectory prediction method based on a three-stage attention mechanism, comprising: step S1: collecting data information and performing data preprocessing; step S2: dividing the data set according to the preprocessed data set; step S3: building a vehicle trajectory prediction model; step S4: training the vehicle trajectory prediction model, using negative log likelihood as a loss function; step S5: saving the trained data model, inputting the data in the test set into the trained network model, and predicting the trajectory coordinates of the vehicle; by introducing the attention mechanism into the model, the most suitable information for trajectory prediction is adaptively selected, and the performance of the algorithm prediction is improved; the weights of various information obtained by the attention mechanism can be used to know the influence of the information on the prediction performance of the algorithm, and the interpretability of the algorithm is improved; the influence of the input information on the performance of the trajectory prediction algorithm is reflected according to the attention mechanism, and the method is applicable to various road environment scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a vehicle trajectory prediction method based on a three-stage attention mechanism. Background Art

[0002] For autonomous vehicles to drive safely and efficiently in complex traffic composed of human drivers, they need to have the ability to make active decisions, such as deciding when to change lanes, overtake, or slow down to allow other vehicles to merge. This requires autonomous vehicles to know the future trajectories of surrounding vehicles so that they can plan their driving status in advance to avoid danger. The vehicle trajectory prediction algorithm gives autonomous vehicles the ability to infer the future movement of surrounding vehicles. Specifically, the problem that the vehicle trajectory prediction algorithm needs to solve is to infer the future trajectory of the target vehicle given the historical state of the target vehicle (including trajectory, speed, acceleration, heading angle, lane, etc.).

[0003] The present invention relates to a long-short term memory network (LSTM), which is a type of neural network in deep learning, and is used to model the historical state of a target vehicle, and then a prediction algorithm predicts the future trajectory based on the modeled data.

[0004] The present invention relates to the technology of attention mechanism. Similar to the problem of human attention allocation, by introducing attention mechanism at different stages of the algorithm, the algorithm can select information that makes the prediction result more accurate during the prediction process. Summary of the invention

[0005] The present invention aims to overcome the problem in the prior art that only the historical trajectory information of the target vehicle and its surrounding vehicles is used while other dynamic data is ignored, and the historical state of the vehicle cannot be fully expressed. A vehicle trajectory prediction method based on a three-stage attention mechanism is provided. By adopting the attention mechanism at the feature level, time series level, and space level, the information most suitable for trajectory prediction is extracted, thereby improving the prediction performance of the prediction algorithm.

[0006] A vehicle trajectory prediction method based on a three-stage attention mechanism, comprising:

[0007] Step S1: Collect data information and perform data preprocessing;

[0008] Step S2: dividing the data set according to the preprocessed data set;

[0009] Step S3: construct a vehicle trajectory prediction model;

[0010] Step S4: training the vehicle trajectory prediction model, using negative log-likelihood as the loss function;

[0011] Step S5: Save the trained data model, input the data in the test set into the trained network model, and predict the trajectory coordinates of the vehicle.

[0012] The information collection data targets include: vehicle geographic coordinates, speed, acceleration, lane number, heading angle and other information, and the information is preprocessed; then the preprocessed data is merged and classified, and the vehicle trajectory prediction model is constructed through LSTM, feature attention module, temporal attention module, and spatial attention module; after the prediction model is constructed, the negative log-likelihood (NLL) is used as the loss function to complete the training of the prediction model and save it, and the data in the test set is input into the trained network model to predict the possible trajectory coordinates of the vehicle in the future.

[0013] The algorithm uses three attention mechanisms: feature-level attention, temporal-level attention, and spatial-level attention to extract the most suitable information for the trajectory prediction algorithm at different stages.

[0014] Preferably, step S1 comprises:

[0015] Step S1A: Select a single vehicle at a certain time as the target vehicle, and obtain the historical information of the target vehicle and its eight surrounding vehicles at time T and the future information at time M;

[0016] Step S1B: preprocess the information at each moment, use min-max to perform standardization, and use ZCA to perform standardization transformation;

[0017] Step S1C: Sample the vehicle trajectories of the dataset at intervals of 4 seconds.

[0018] The historical information of the target vehicle and the target vehicles in the eight directions around it at time T is captured, and the future information at time M is predicted. The real-time data information and future prediction information are summarized and stored in the database. The min-max standardization and Z-score standardization methods are used to scale the data according to a certain ratio so that the data falls into a specific interval, making data collection and prediction more accurate. Sampling analysis is performed at intervals of 4 seconds to enhance the real-time nature of data collection and prediction. By comparing the real-time data analysis with the prediction results, the range of subsequent vehicle motion predictions is corrected to improve the accuracy of vehicle trajectory prediction.

[0019] Preferably, the data set division in step S2 includes:

[0020] Training set, used to train the model;

[0021] Validation set, used to verify the model training results;

[0022] The test set is used to test the trained model.

[0023] The data division ratio of the training set, validation set, and test set is 7:1:2. The ratio of the training set, validation set, and test set is controlled at 7:1:2, which makes full use of the data to train the data model, which is beneficial to the subsequent vehicle trajectory prediction algorithm based on the three-stage attention mechanism of feature level attention, time level attention, and space level attention, and improves the data resource rate. By introducing the attention mechanism into the model, the most suitable information for trajectory prediction is adaptively selected to improve the performance of the algorithm prediction.

[0024] As a preference, step S3 includes a feature attention module, a temporal attention module and a spatial attention module. The prediction model algorithm process takes the target vehicle in the sample as an example, and the historical information at T moments is recorded as X=X t-T+1 ,…,X t , where X i =x1,…,x n is the feature preprocessed by S1.2 at the i-th moment, n represents the number of features, X t It is the feature of the vehicle at the current moment. It reflects the impact of input information on the performance of the trajectory prediction algorithm according to the attention mechanism. It is applicable to a variety of road environment scenarios, such as complex paths such as highways, elevated roads, and urban roads.

[0025] Therefore, the present invention has the following beneficial effects:

[0026] 1. By introducing the attention mechanism into the model, the most suitable information for trajectory prediction is adaptively selected to improve the performance of algorithm prediction;

[0027] 2. The weights of various information obtained through the attention mechanism can tell the impact of information on the prediction performance of the algorithm, thus improving the interpretability of the algorithm;

[0028] 3. The attention mechanism reflects the impact of input information on the performance of the trajectory prediction algorithm, which is applicable to a variety of road conditions and environment scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the operation process of the present invention;

[0030] Figure 2 It is the overall flow chart of the algorithm of the present invention;

[0031] Figure 3 is a flow chart of the feature attention module of the present invention;

[0032] Figure 4is a flow chart of the temporal attention module of the present invention;

[0033] Figure 5 It is a flow chart of the spatial attention module of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be described in further detail below based on the accompanying drawings and specific implementation methods.

[0035] The present invention aims to overcome the problem in the prior art that only the historical trajectory information of the target vehicle and its surrounding vehicles is used while other dynamic data is ignored, and the historical state of the vehicle cannot be fully expressed. A vehicle trajectory prediction method based on a three-stage attention mechanism is provided. By adopting the attention mechanism at the feature level, time series level, and space level, the information most suitable for trajectory prediction is extracted, thereby improving the prediction performance of the prediction algorithm.

[0036] like Figure 1 As shown, a vehicle trajectory prediction method based on a three-stage attention mechanism includes:

[0037] Step S1: Collect data information and perform data preprocessing;

[0038] Step S2: dividing the data set according to the preprocessed data set;

[0039] Step S3: construct a vehicle trajectory prediction model;

[0040] Step S4: training the vehicle trajectory prediction model, using negative log-likelihood as the loss function;

[0041] Step S5: Save the trained data model, input the data in the test set into the trained network model, and predict the trajectory coordinates of the vehicle.

[0042] The information collection data targets include: vehicle geographic coordinates, speed, acceleration, lane number, heading angle and other information, and the information is preprocessed; then the preprocessed data is merged and classified, and the vehicle trajectory prediction model is constructed through LSTM, feature attention module, temporal attention module, and spatial attention module; after the prediction model is constructed, the negative log-likelihood (NLL) is used as the loss function to complete the training of the prediction model and save it, and the data in the test set is input into the trained network model to predict the possible trajectory coordinates of the vehicle in the future.

[0043] The algorithm uses three attention mechanisms: feature-level attention, temporal-level attention, and spatial-level attention to extract the most suitable information for the trajectory prediction algorithm at different stages.

[0044] Step S1 includes:

[0045] Step S1A: Select a single vehicle at a certain time as the target vehicle, and obtain the historical information of the target vehicle and its eight surrounding vehicles at time T and the future information at time M;

[0046] Step S1B: preprocess the information at each moment, use min-max to perform standardization, and use ZCA to perform standardization transformation;

[0047] Step S1C: Sample the vehicle trajectories of the dataset at intervals of 4 seconds.

[0048] The historical information of the target vehicle and the target vehicles in the eight directions around it at time T is captured, and the future information at time M is predicted. The real-time data information and future prediction information are summarized and stored in the database. The min-max standardization and Z-score standardization methods are used to scale the data according to a certain ratio so that the data falls into a specific interval, making data collection and prediction more accurate. Sampling analysis is performed at intervals of 4 seconds to enhance the real-time nature of data collection and prediction. By comparing the real-time data analysis with the prediction results, the range of subsequent vehicle motion predictions is corrected to improve the accuracy of vehicle trajectory prediction.

[0049] The data set division in step S2 includes:

[0050] Training set, used to train the model;

[0051] Validation set, used to verify the model training results;

[0052] The test set is used to test the trained model.

[0053] The data division ratio of the training set, validation set, and test set is 7:1:2. The ratio of the training set, validation set, and test set is controlled at 7:1:2, which makes full use of the data to train the data model, which is beneficial to the subsequent vehicle trajectory prediction algorithm based on the three-stage attention mechanism of feature level attention, time level attention, and space level attention, and improves the data resource rate. By introducing the attention mechanism into the model, the most suitable information for trajectory prediction is adaptively selected to improve the performance of the algorithm prediction.

[0054] Step S3 includes a feature attention module, a temporal attention module and a spatial attention module. The prediction model algorithm process takes the target vehicle in the sample as an example, and the historical information at T moments is recorded as X=X t-T+1 ,…,X t , where X i =x1,…,x n is the feature preprocessed by S1.2 at the i-th moment, n represents the number of features, X tIt is the feature of the vehicle at the current moment. It reflects the impact of input information on the performance of the trajectory prediction algorithm according to the attention mechanism. It is applicable to a variety of road environment scenarios, such as complex paths such as highways, elevated roads, and urban roads.

[0055] Among them, Figure 2 As shown, the overall algorithm flow includes the following steps:

[0056] (a): The feature X of each vehicle at each moment i They are sent to the feature attention module for processing and obtained

[0057] X′=X′ t-T+1 ,…,X′ t .

[0058] (b) Send X′ to LSTM to obtain the hidden state H = h at each moment t-T+1 ,…,h t .

[0059] (c): Send H to the temporal attention module and get H′1=H′ t-T+1 ,…,H′ t .

[0060] (d) The surrounding vehicles are subjected to the above three operations respectively to obtain their respective hidden layer states H′ 2~9 .

[0061] (f): Separate H′1 from H′ 2~9 Spliced ​​together and sent to the spatial attention module to obtain O 1~8 outputs.

[0062] (g): Send H′1 to the MLP layer to obtain H″1.

[0063] (h): O 1~8 After addition, it is concatenated with H″1 and sent to the LSTM decoder to obtain the hidden layer output Y=Y t+1 ,…,Y t+M .

[0064] (i): Map Y through the MLP layer to obtain the possible future trajectory of the target vehicle.

[0065] like Figure 3 As shown in the figure, the input data of the feature attention module is the preprocessed data at a certain moment, which is a 1×T matrix, such as the data of the target vehicle at the i-th moment, X i ={x1,…,x n}. After two fully connected layers and sigmoid, n weights are generated, which are i Multiply them and get X′i ;

[0066] like Figure 4 As shown, the input data of the temporal attention module is X′=X′ t-T+1 ,…,X′ t The two-dimensional matrix H composed of the hidden layer states at each moment obtained after LSTM, the number of rows v represents the dimension of the hidden layer at each moment, and the number of columns T represents the length of the historical data, such as the hidden layer state H of the target vehicle = h t-T+1 ,…,h t ,in That is, the dimension of the hidden layer at a certain moment is v. First, global pooling is performed on H row by row to obtain a 1×T matrix, and then two fully connected layers and sigmoid are used to generate T weights. Finally, H and T are multiplied to obtain H′1;

[0067] like Figure 5 As shown, after the data acquisition of the feature attention module and the temporal attention module, the operation of the spatial attention module includes the following steps:

[0068] Step SD1: Input data is H′ 1~9 ;

[0069] Step SD2: H′1 and H′ 1~8 To splice;

[0070] Step SD3: Send the concatenated results to the fully connected layer to obtain 8 one-dimensional vectors, denoted as J′ 1~8 ;

[0071] Step SD4: After concatenating the 8 one-dimensional vectors, 8 weights are obtained after passing through the fully connected layer and the softmax layer;

[0072] Step SD5, weight and H′ 2~9 After multiplication, we get the output O 1~8 .

[0073] In the above, after data calculation by the feature attention module, the temporal attention module and the spatial attention module, the specific data information such as the target vehicle's lateral coordinates, longitudinal coordinates, speed, acceleration, heading angle, vehicle length, vehicle width, vehicle category, and lane number are converted into data model simulation quantities. Through the calculation and analysis of the data model, the information required for vehicle prediction is obtained, the vehicle trajectory is predicted, and the real-time information collected every 4 seconds is compared with the predicted trajectory, and the subsequent predicted trajectory is corrected to improve the accuracy of trajectory prediction.

[0074] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings, but the above is only a preferred embodiment of the present invention. It should be noted that the technical features involved in the above embodiments and their preferred methods can be reasonably combined and matched into a variety of equivalent schemes by those skilled in the art without departing from or changing the design ideas and technical effects of the present invention; therefore, the present invention is not limited to the scope of implementation shown in the drawings, and all changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which still do not exceed the spirit covered by the specification and drawings, should be within the protection scope of the present invention.

Claims

1. A vehicle trajectory prediction method based on a three-stage attention mechanism, comprising: Step S1: Collect data information and perform data preprocessing; Step S2: dividing the data set according to the preprocessed data set; Step S3: construct a vehicle trajectory prediction model; Step S4: training the vehicle trajectory prediction model, using negative log-likelihood as the loss function; Step S5: Save the trained data model, input the data in the test set into the trained network model, and predict the trajectory coordinates of the vehicle; The feature X of each vehicle at each time i Send them to the feature attention module to get X ′ =X t ′ ;T:1 ,…,X t ′ ; X ′ Send it to LSTM to obtain the hidden state H = h at each moment t;T:1 ,…,h t ; H is fed into the temporal attention module to obtain H1 ′ =H t ′ ;T:1 ,…,H t ′ ; Get the hidden layer state H2 of each surrounding vehicle ′ ~9 ; H1 ′ Respectively with H2 ′ ~9 Splicing, sending to the spatial attention module, obtain O 1~8 Output; H1 ′ Send it to the MLP layer to get H1″; O 1~8 Add, concatenate with H1″, and send to the LSTM decoder to obtain the hidden layer output Y=Y t:1 ,…,Y t:M ; Y is mapped through the MLP layer to obtain the future trajectory of the target vehicle.

2. A vehicle trajectory prediction method based on a three-stage attention mechanism according to claim 1, characterized in that the steps S1 includes: Step S1A: Select a single vehicle at a certain time as the target vehicle, and obtain the historical information of the target vehicle and its eight surrounding vehicles at time T and the future information at time M; Step S1B: preprocess the information at each moment, use min-max to perform standardization, and use ZCA to perform standardization transformation; Step S1C: Sample the vehicle trajectories of the dataset at intervals of 4 seconds.

3. The vehicle trajectory prediction method based on the three-stage attention mechanism according to claim 2 is characterized in that: The historical information and future information include lateral coordinates, longitudinal coordinates, speed, acceleration, heading angle, vehicle length, vehicle width, vehicle type, and lane number.

4. The vehicle trajectory prediction method based on the three-stage attention mechanism according to claim 1 is characterized in that: The data set division in step S2 includes: Training set, used to train the model; Validation set, used to verify the model training results; The test set is used to test the trained model.

5. The vehicle trajectory prediction method based on the three-stage attention mechanism according to claim 4 is characterized in that: The data division ratio of the training set, validation set, and test set is 7:1:

2.

6. The vehicle trajectory prediction method based on the three-stage attention mechanism according to claim 1 is characterized in that: Step S3 includes a feature attention module, a temporal attention module and a spatial attention module.

7. The vehicle trajectory prediction method based on the three-stage attention mechanism according to claim 2 is characterized in that: The prediction model algorithm flow of step S3 is based on the T time information of the target vehicle in the sample, denoted as X=X t;T:1 ,…,X t , where X i =x1,…,x n is the feature preprocessed by S1B at the i-th moment, n represents the number of features, X t is the characteristic of the vehicle at the current moment.

Citation Information

Patent Citations

  • Unmanned vehicle trajectory prediction method based on local attention mechanism

    CN112465273A

  • Vehicle trajectory prediction method based on residual attention mechanism

    CN113313320A

Cited By

  • Method for realizing strip mine vehicle trajectory prediction through combination of random forest and Kalman

    CN120145808A

  • Method for realizing vehicle trajectory prediction in open-pit mine by combining random forest and kalman

    CN120145808B