Vehicle trajectory prediction method considering on-road driving style

Abstract

This invention belongs to the field of autonomous driving safety technology, and more particularly relates to a vehicle trajectory prediction method that takes into account on-the-go driving style. This invention comprehensively understands the state of a target vehicle from the perspectives of the scene perception layer, the pattern layer, the operation layer, and the vehicle state layer. The proposed trajectory prediction method that takes into account on-the-go driving style integrates the advantages of rule-based data-driven methods and uses a knowledge-driven approach to transform the current state of the target vehicle from an abstract data level into a driving behavior semantic space. This method can simulate humans' understanding of the real world and demonstrate the prediction results of the trajectory prediction model in different driving behavior semantic spaces and different on-the-go driving style types.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving safety technology, and in particular relates to a vehicle trajectory prediction method considering on-road driving style. Background Art

[0002] Self-driving cars have attracted widespread attention as a potentially safer and more sustainable mode of transportation. Researchers aim to improve traffic safety and passenger comfort through autonomous driving technology, thereby enhancing the travel experience of travelers. Vehicle trajectory prediction is an important research topic for self-driving cars.

[0003] Before entering the era of fully autonomous driving, a mixed traffic landscape of human-driven and autonomous vehicles will exist for a long time. Autonomous vehicles will need to interact with surrounding human-driven vehicles and predict their trajectories to ensure driving safety. Vehicle trajectory prediction already has a relatively well-established research framework. Current research focuses primarily on long-term trajectory characteristics. Trajectory prediction models aim to improve trajectory prediction accuracy, but rarely consider the impact of individual driver characteristics and the driving environment on model results. The complexity of real-world traffic environments and the uncertainty of human driving styles present new opportunities and challenges for trajectory prediction. Human drivers in the surrounding environment exhibit individual heterogeneity and inconsistency. Individual heterogeneity refers to differences in characteristics between different drivers, while inconsistency refers to differences between the same driver in different driving scenarios. Existing trajectory prediction models have achieved some success in a data-driven model. However, data-driven approaches are trained on specific driving scenarios or data, which can lead to overfitting due to data influence. While effective on the training dataset, these models lack generalization and scalability to other datasets. This presents significant challenges for autonomous vehicles in the diverse and unpredictable real-world scenarios. Summary of the Invention

[0004] To address these issues, the present invention proposes an improved bidirectional long-short-term memory (BLSTM) neural network that considers on-the-go driving, the heterogeneity and inconsistency of driving styles during the driving process, and incorporates an attention mechanism. This network incorporates an attention mechanism that assigns different weights to historical trajectory sequences. This mechanism enables the model to focus on time series data that have a greater impact on future outcomes, assigning higher weights to historical sequences that have a greater impact on outcomes. This network improves upon the BiLSTM model, improving the model's accuracy in long-term trajectory prediction by integrating the attention mechanism. It also significantly improves prediction accuracy for changes in driver driving patterns and operating states.

[0005] Compared to traditional data-driven approaches, knowledge-driven approaches enable autonomous vehicles to fully understand the target vehicle's surroundings, improving the interpretability of trajectory prediction systems and making it easier for humans to interpret autonomous vehicles' decision-making. The proposed on-the-go driving style recognition method based on driving behavior semantic understanding comprehensively understands the target vehicle's state from the scene perception layer, the pattern layer, the operation layer, and the vehicle state layer. The proposed trajectory prediction method, which considers on-the-go driving style, integrates the advantages of rule-based data-driven approaches. It uses a knowledge-driven approach to transform the target vehicle's current state from an abstract data level into a driving behavior semantic space. This method can simulate humans' understanding of the real world and demonstrate the prediction results of the trajectory prediction model in different driving behavior semantic spaces and different on-the-go driving style types.

[0006] The technical solution of the present invention:

[0007] The vehicle trajectory prediction method considering the on-road driving style has the following specific steps:

[0008] Step (1) Data acquisition and data preprocessing

[0009] Vehicle trajectory data is acquired through V2X devices at a frequency of at least 10 Hz, including vehicle position, longitudinal velocity, lateral velocity, longitudinal acceleration, lateral acceleration, time-to-headway (THW), expected time to collision (TTC), and perception data (surrounding vehicle information, weather, and road conditions). The raw vehicle trajectory data is smoothed using a Savitzky-Golay filter.

[0010] Step (2) Construction of driving behavior semantic space and on-the-road driving style recognition

[0011] The driving behavior semantic space is a multi-dimensional concept that aims to comprehensively capture and analyze the driver's behavior patterns in different driving scenarios. The present invention divides the driving behavior semantic space into four levels from the macro level to the micro level: scene perception layer, pattern layer, operation layer and vehicle status layer. Among them, the scene perception layer (macro level) mainly refers to the weather information, road environment information, surrounding vehicle information, etc. obtained by the vehicle during driving. The pattern layer refers to the status information such as the vehicle choosing close following, long-distance following or frequent lane changing in a specific scenario after determining the driving path. The operation layer is manifested as whether the driver has sudden acceleration, sudden deceleration and other behaviors in the current mode. The vehicle status layer (micro level) refers to micro-level information such as the current vehicle speed and acceleration.

[0012] The vehicle trajectory data obtained in step (1) is subjected to driving behavior semantic understanding, and the abstract vehicle trajectory data is converted into understandable driving behavior semantic information. A driving behavior semantic space is constructed to comprehensively describe and understand the driver's driving process. The converted driving behavior semantic data is input into the LDA topic model for topic identification. The number of driving style topics is determined through topic perplexity, topic consistency, and driving style topic visualization distribution. Finally, the characteristics of each driving style topic are determined through topic analysis, and the probability of the driver behaving as aggressive, moderate, conservative, or stable is obtained.

[0013] The LDA topic model is a widely used topic discovery model. It consists of a three-layer Bayesian structure consisting of documents, words, and topics. It was originally proposed in the field of natural language processing and can reveal the potential topics of documents. Its main idea is that the generation of an article follows a probability distribution, that is, each word is selected from a certain topic with a certain probability, and a certain word is selected from this topic with a certain probability. In the LDA model, each document in a document collection is generated by a mixture of multiple topics, and each topic is composed of multiple words. The mathematical representation of this model usually involves several steps, including:

[0014]

[0015] z m,n ∣θ m ~Miltinomial(θ m )(2)

[0016] θ m ~Dirichlet(α)(3)

[0017]

[0018] Among them, θ m Represents the topic distribution in document m, sampled from the Dirichlet distribution Dirichlet(α). represents the word distribution in topic k, obtained by sampling from the Dirichlet distribution (β). m,n Refers to the topic assignment of the nth word in document m, which is obtained by multinomial distribution Multinomial(θ m ) is sampled. m,n It refers to the nth word in document m, and z is assigned according to the topic of the word m,n From the multinomial distribution Formula (1) represents word w m,n The generation process of m,nSelect words from the corresponding word distribution. Formula (2) shows how to select topic z for each word m,n , which depends on the topic distribution of document m. Formula (3) shows how the topic distribution of each document is sampled, which depends on the Dirichlet parameter α. Formula (4) shows how the word distribution within each topic is sampled, which depends on the Dirichlet parameter β. The LDA topic model assumes that the topic generation process is the same for all documents. The parameters α and β of these distributions usually need to be obtained through training data.

[0019] Step (3) Design of trajectory prediction dataset integrating on-the-road driving style

[0020] Design a data input module that contains basic information about people, vehicles, and the environment to ensure that the influence of different factors can be fully considered during trajectory prediction.

[0021] The human factor (EDS) is the four driving styles identified by the LDA topic model in step (2): stable (DS0), aggressive (DS1), moderate (DS2), and conservative (DS3). The on-the-road driving style of each driver is the probability that the driver exhibits these four driving styles, and the sum of the probabilities of the driver exhibiting these four driving styles is 1.

[0022] EDS={DS0,DS1,DS2,DS3} (5)

[0023] Vehicle factors (TV) include lateral position x, longitudinal position y, vehicle speed information (including lateral speed v x and longitudinal velocity v y ) and acceleration information (including lateral acceleration a x and longitudinal acceleration a y ).

[0024] TV={x,y,v x ,v y ,a x ,a y} (6)

[0025] Environmental factors (EV) take into account that the preceding vehicle has the most significant impact on the predicted vehicle, and the headway distance (DHW), time headway (THW) and time to collision (TTC) are selected as characteristic parameters.

[0026] EV={DHW,THW,TTC} (7)

[0027] Step (4) Model network structure design

[0028] The attention mechanism is widely used in the field of artificial intelligence. It imitates the attention process in human vision and thinking. Its main purpose is to enable the model to selectively focus on specific parts of the input data instead of processing all information equally.

[0029] The EDTA-BiLSTM trajectory prediction model considering on-the-road driving style is constructed by integrating the attention mechanism. The complex dependencies of time series data are captured by the bidirectional long short-term memory neural network (BiLSTM) layer. First, the 13-dimensional feature vector {DS0, DS1, DS2, DS3, x, y, v x ,v y ,a x ,a y ,DHW,THW,TTC} as input, the characteristic factors affecting the target vehicle's driving trajectory are passed to the network. The input feature vector is first expanded through a fully connected layer to expand the feature space, and then the input feature is passed to a fully connected layer with 256 hidden units for linear transformation to improve the feature representation capability and adapt to the high-dimensional processing requirements of the subsequent BiLSTM layer. The linearly transformed feature vector is sent to the BiLSTM layer. The BiLSTM layer consists of two stacked LSTM sublayers, each of which contains 256 hidden units. It integrates forward and backward information through bidirectional transmission, optimizing the processing of time series data. The EDTA-BiLSTM network integrates the attention mechanism. By assigning different weights to different parts of the sequence, the network can pay more attention to information that is effective for prediction during prediction. The output of the BiLSTM layer is passed to the attention layer, and the weighted sum is calculated by the attention layer. The weight parameter is determined by the tanh(x) activation function and a trainable weight vector. The weight is normalized using the Softmax function to generate a probabilistic attention distribution. The attention layer can highlight the importance of the time step, focus limited attention on important information, and pass the output features to two consecutive fully connected layers. Finally, the output layer outputs the predicted results of the target vehicle's position information and speed information, including lateral position x, longitudinal position y, and lateral speed v. x and longitudinal velocity v y .

[0030] The EDTA-BiLSTM model optimizes trajectory prediction accuracy, particularly when considering both historical and future contextual information. This network architecture improves upon the existing network by incorporating an attention mechanism, enabling it to capture the complex dynamics of vehicle trajectories. The introduction of the attention mechanism further enhances sensitivity to key sequence features, enabling the network to balance the impact of past and future trajectory data points.

[0031] Step (5) Model training and prediction result evaluation

[0032] The model training set, test set, and validation set are randomly divided according to the proportion. The root mean square error (RMSE) and final displacement error (FDE) are used for evaluation.

[0033]

[0034] in, are the values of the predicted vehicle trajectory abscissa and ordinate, are the values of the actual vehicle trajectory horizontal and vertical coordinates, and n is the number of trajectory points.

[0035] The beneficial effects of the present invention are as follows: the method can realize long-term trajectory prediction for different human drivers, and at the same time, can explain the prediction results for drivers in different driving behavior semantic spaces, thereby improving the problem of poor interpretability of traditional neural network model predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 4 is a flowchart of the vehicle trajectory prediction method considering the on-road driving style of the present invention.

[0037] Figure 2 It is the semantic space structure diagram of driving behavior.

[0038] Figure 3 This is a schematic diagram of the vehicle trajectory prediction model considering on-the-road driving style.

[0039] Figure 4 、 Figure 5 、 Figure 6 This is a visualization of the vehicle trajectory prediction results for three different on-the-road driving styles. DETAILED DESCRIPTION

[0040] The following describes in detail the specific implementation of the present invention with reference to examples, and simulates the implementation effects of the invention.

[0041] The vehicle trajectory prediction method considering the on-road driving style of the present invention is as follows Figure 1 This example uses the highD vehicle trajectory open source dataset. After data screening, 9427 vehicle trajectory data are finally selected as the research object, and the Savitzky-Golay filter is used to smooth the original trajectory data.

[0042] The present invention establishes a driving behavior semantic division structure from macro level to micro level, including scene perception layer, mode layer, operation layer and vehicle status layer. Figure 2As shown in the figure, the driving behavior semantics are understood by translating the abstract trajectory data into understandable driving behavior semantic information. A driving behavior semantic space is constructed to comprehensively describe and understand the driver's driving process. The converted driving behavior semantic data is then input into the LDA topic model for topic identification. The number of driving style topics is determined through topic perplexity, topic consistency, and a visual distribution of driving style topics. Finally, topic analysis is used to identify the characteristics of each driving style topic and determine the probability of the driver behaving in an aggressive, moderate, conservative, or stable manner.

[0043] Design a data input module containing basic information of people, vehicles and environment to ensure that the influence of different factors can be fully considered in the process of trajectory prediction. The input feature is a 13-dimensional feature vector {DS0, DS1, DS2, DS3, x, y, v x ,v y ,a x ,a y ,DHW,THW,TTC}.

[0044] The EDTA-BiLSTM model of the present invention adopts a sequence-to-sequence (seq2seq) learning framework. Figure 3 As shown in Figure 2, it is possible to capture the temporal dependency of the sequence, which is crucial when processing time series data such as vehicle motion paths. The dimension of the input variables is 13, including the human factor (EDS), which is the driving style of the target vehicle on the way, and the vehicle factor (TV), which is {x, y, v x ,v y ,a x ,a y}, and environmental factors (EV), namely {DHW, THW, TTC}, the output of the prediction model is the trajectory information at the future moment, including {x, y, v x ,v y}, using 5 seconds of historical vehicle trajectory data to predict vehicle trajectory information for the next 5 seconds. The model's input layer receives a 13-dimensional feature vector, providing the model with a comprehensive view of the vehicle's current state. The input feature vector first passes through a fully connected layer, which expands the feature space to accommodate the high-dimensional processing requirements of the subsequent BiLSTM layer. After passing through the fully connected layer, the feature vector is fed into the BiLSTM layer, which consists of two stacked LSTM sublayers, each containing 256 hidden units. This bidirectional structure enables the network to consider both past and future information, optimizing the processing of time series data. Finally, the output data of the BiLSTM layer is passed to the output layer, which outputs a 4-dimensional prediction vector. The model also integrates a dropout mechanism to prevent overfitting. By randomly discarding some neuronal connections during training, the model's generalization ability is improved.

[0045] The EDTA-BiLSTM model training of the present invention is carried out under the Pytorch deep learning framework built in Pycharm. The specific experimental environment is shown in the table. The operating system is Linux, the memory is 512G, the GPU is NVIDIA GeForce RTX 3090, and the programming language used is Python.

[0046] The parameters that need to be passed during the training process of the EDTA-BiLSTM model of the present invention include n_feature, input_size, hidden_size, n_layers, and output_size. Among them, n_feature is the number of features contained in the input trajectory point. Input_size is the length of the vector input to the LSTM layer, hidden_size is the number of hidden nodes of the LSTM, n_layers is the number of hidden layers, and output_size is the dimension of the output predicted trajectory data. The batch (batch_size) is set to 128, the model optimizer is selected as Adam, the learning rate (learning rate) weight decay (weight decay) is 0.0001, and dropout is 0.2. Dropout is a regularization technique used to randomly discard (set the weight to zero) a portion of neurons during the training process of the neural network to prevent overfitting. This helps to improve the generalization ability of the model and make it perform better on new data. By randomly ignoring certain neurons, dropout forces the network not to rely on specific neurons, thereby reducing the risk of overfitting. The loss function adopts the mean square error loss function (MSELoss).

[0047] This embodiment designs a comparative experiment to verify the effect of the trajectory prediction model considering the on-the-road driving style.

[0048] (1) LSTM: Long Short-Term Memory Neural Network.

[0049] (2) BiLSTM: Bidirectional long short-term memory neural network.

[0050] (3) ED-BiLSTM: A bidirectional long short-term memory neural network considering on-the-road driving style.

[0051] (4) EDTA-BiLSTM (present invention): an improved bidirectional long short-term memory neural network integrating attention mechanism.

[0052] The root mean square error and final displacement error results of the model in the 1 to 5 s prediction time domain are shown in Tables 1 and 2.

[0053] Table 1 Comparison of root mean square error (RMSE) of prediction results of different models

[0054]

[0055] Table 2 Comparison of final displacement errors of prediction results of different models

[0056]

[0057] This example focuses on the model's trajectory prediction results for individual drivers with different on-the-road driving styles in different driving behavior semantic spaces. Figure 4 The driving semantic space of the driver in the example is: speed level is high speed, acceleration level is safe acceleration, driving mode is gradually following, operation state is normal deceleration, surrounding driving environment is obstacles ahead, and his on-road driving style is [0.1666, 0.0001, 0.7983, 0.0348]. This driver mainly exhibits a conservative driving style. The EDTA-BiLSTM prediction result of the 5-second FDE is 1.23m, and the ED-BiLSTM FDE of the next 5 seconds is 1.48m. Figure 5 The driver's speed level is moderate, the acceleration level is risky acceleration, the vehicle's driving mode is restricted lane change, the operating state is normal acceleration, and the surrounding driving environment is an obstacle ahead. The driver's on-the-road driving style probability combination is [0.1276, 0.0001, 0.0306, 0.8415]. In this case, the driver mainly exhibits an aggressive driving style. The visualization shows that the improved model shows significant improvement over the original model within 1 to 5 seconds. The EDTA-BiLSTM model predicts a final displacement error of 1.39 meters within 5 seconds, while the ED-BiLSTM model predicts a final displacement error of 1.91 meters within 5 seconds. Figure 6 The semantic space of the driver's driving behavior is as follows: speed level is high speed, acceleration level is risky acceleration, driving mode is restricted lane change, operation mode is continuous acceleration, surrounding environment is obstacle ahead and obstacle diagonally ahead. The probability combination of his on-road driving style is [0.0001, 0.0253, 0.0002, 0.9744]. At this time, the driver mainly behaves in an aggressive manner. From the visualization of the prediction results, it can be seen that the original model has poor prediction effect on the lane change position and lane change trajectory, while the improved model integrating the attention mechanism can well predict the vehicle's lane change trajectory and starting point.

[0058] The present invention integrates the attention mechanism to improve the trajectory prediction model, and by comparing the model prediction effects, evaluates the performance changes of the model before and after the introduction of the attention mechanism and the on-the-road driving style. The RMSE and FDE of the overall prediction results of the model of the present invention are improved. In the long-term trajectory prediction problem in the 5s prediction time domain, the RMSE and FDE can reach 1.28m and 1.17m respectively. The example verification results show that the improved BiLSTM model integrating the attention mechanism is superior to the original model in both accuracy and prediction details. In addition, the example explores the performance of the improved model's prediction results for different driving behavior semantic spaces and drivers with different on-the-road driving styles. Combining the advantages of data-driven and rule-driven, the knowledge-driven approach is used to improve the interpretability and scalability of the model, which not only improves the accuracy of the vehicle trajectory prediction model, but also contributes an effective tool to the field of long-term series analysis.

Claims

1. A vehicle trajectory prediction method considering on-the-road driving style, characterized by: The specific steps are as follows: Step 1: Data acquisition and data preprocessing Vehicle trajectory data is acquired through V2X devices at a frequency of no less than 10 Hz, including vehicle position, longitudinal velocity, lateral velocity, longitudinal acceleration, lateral acceleration, headway, expected collision time, and perception data. The raw vehicle trajectory data is smoothed using a Savitzky-Golay filter. Step 2: Construction of driving behavior semantic space and identification of driving style on the way From the macro level to the micro level, the driving behavior semantic space is divided into four layers: scene perception layer, mode layer, operation layer, and vehicle status layer. Among them, the scene perception layer is the macro level, including weather information, road environment information, and surrounding vehicle information obtained by the vehicle during driving. The mode layer refers to the vehicle's state information of choosing close following, long following, or frequent lane changes in specific scenarios after determining the driving path. The operation layer shows whether the driver has sudden acceleration or deceleration in the current mode. The vehicle status layer is the micro level, referring to the current vehicle speed and acceleration information. The vehicle trajectory data obtained in step 1 is used to understand driving behavior semantics. The abstract vehicle trajectory data is converted into understandable driving behavior semantic information, and a driving behavior semantic space is constructed. The converted driving behavior semantic data is input into the LDA topic model for topic identification. The number of driving style topics is determined through topic perplexity, topic consistency, and visual distribution of driving style topics. Finally, the characteristics of each driving style topic are determined through topic analysis, and the probability of a driver behaving as aggressive, moderate, conservative, or stable is obtained. Step 3: Design a trajectory prediction dataset that integrates on-the-road driving style Design a data input module that includes basic information about the person, vehicle, and environment to ensure that the impact of different factors can be fully considered during trajectory prediction; Human Factors In step 2, the LDA topic model was used to identify four driving styles: stable DS0, aggressive DS1, moderate DS2, and conservative DS3. The on-the-road driving style of each driver is the probability of the driver exhibiting these four driving styles, and the sum of the probabilities of the driver exhibiting these four driving styles is 1. (1) ， Car factors Including horizontal position x, vertical position , vehicle speed information and acceleration information; wherein the vehicle speed information includes lateral speed and longitudinal speed ; Acceleration information includes lateral acceleration and longitudinal acceleration ; (2) ， Environmental factors Considering that the front vehicle has the most significant impact on the predicted vehicle, the headway is selected , headway and collision time As characteristic parameters; (3) ， Step 4: Model network structure design The EDTA-BiLSTM trajectory prediction model considering on-the-road driving style is constructed by integrating the attention mechanism. The complex dependencies of time series data are captured by the BiLSTM layer of the bidirectional long short-term memory neural network. First, the 13-dimensional feature vector obtained in step 3 is converted into As input, the characteristic factors affecting the target vehicle's driving trajectory are passed to the network. The input feature vector is first expanded through a fully connected layer to expand the feature space, and then the input feature is passed to a fully connected layer with 256 hidden units for linear transformation; the linearly transformed feature vector is sent to the BiLSTM layer, which consists of two stacked LSTM sublayers, each sublayer contains 256 hidden units, and integrates forward and backward information through bidirectional transmission; the output of the BiLSTM layer is passed to the attention layer, and the weighted sum is calculated through the attention layer, where the weight parameter is determined by the tanh(x) activation function and a trainable weight vector. The Softmax function is used to normalize the weight to generate a probabilistic attention distribution, and the output feature is passed to two consecutive fully connected layers. Finally, the output layer outputs the predicted results of the target vehicle's position information and speed information, including the lateral position x and longitudinal position x. , lateral speed and longitudinal speed ; Step 5: Model training and prediction result evaluation The model training set, test set and validation set are randomly divided according to the proportion; the root mean square error is used And the final displacement error Conduct evaluations; (4) ， (5) ， in, , are the values of the predicted vehicle trajectory abscissa and ordinate, , are the values of the actual vehicle trajectory horizontal and vertical coordinates, and n is the number of trajectory points.

2. The vehicle trajectory prediction method considering on-road driving style according to claim 1, characterized in that: In the LDA model of step 2, each document in the document collection is generated by mixing multiple topics, and each topic is composed of multiple words; the mathematical representation of the LDA model includes: （6）， （7）， （8）， （9）， in, represents the topic distribution in document m, sampled from the Dirichlet distribution Dirichlet(α); represents the word distribution in topic k, through Dirichlet distribution Obtained by sampling; refers to the topic assignment of the nth word in document m, which is distributed by the multinomial Obtained by sampling; Refers to the nth word in document m, assigned according to the topic of the word From the multinomial distribution The formula (6) represents the word The generative process of Select words from the corresponding word distribution; Formula (7) shows how to select topics for each word , which depends on the topic distribution of document m; Formula (8) shows how the topic distribution of each document is sampled, which depends on the Dirichlet parameter ; Formula (9) shows how the word distribution in each topic is sampled, which depends on the Dirichlet parameter .

Images