Vehicle lane-changing trajectory prediction method based on random forest and improved Informer model
By combining the self-attention mechanism and Gaussian bias, the problem of insufficient accuracy in vehicle lane change trajectory prediction by random forest screening features and improved Informer model is solved, and the prediction accuracy and model robustness are improved, and the Resource consumption.
Patent Information
- Application Number
- CN202310993736.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-08
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2043-08-08
AI Technical Summary
Traditional neural networks are difficult to capture the relationship between far-time sequence data, resulting in insufficient prediction accuracy of vehicle lane change trajectory, and feature engineering relies on experience to cause model data to be invalid, increasing model time complexity.
Using a vehicle lane change trajectory prediction method based on random forest and improved Informer model, important features are screened through the random forest algorithm, combined with improved self-attention mechanism and Gaussian bias, local timing information capture is enhanced, noise is reduced, and model generalization is improved. ability.
It improves the accuracy and model robustness of vehicle lane change trajectory prediction, reduces the risk of overfitting, and compresses the model size and computing volume while maintaining high accuracy, reducing resource consumption.
Smart Images

Figure CN117807413B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traffic vehicle trajectory prediction, and in particular to a vehicle lane change trajectory prediction method based on random forest and improved Informer model. Background Art
[0002] Vehicle control technology has become an important part of modern automobile design and development, among which vehicle lane changing is a very important key technology. With the development of Internet technology and artificial intelligence technology, autonomous driving technology is becoming a hot research field. In the field of autonomous driving, vehicle lane changing is a complex problem. Accurately predicting the lane changing trajectory of the vehicle can effectively improve the safety and stability of the vehicle during the lane changing process.
[0003] At present, many researchers have devoted themselves to solving the problem of vehicle lane change trajectory prediction. However, traditional feature engineering is mostly based on feature selection based on existing experience, which may lead to invalid model data input, resulting in reduced model prediction accuracy and increased model time complexity. Efficient feature selection is the key technology to achieve accurate prediction of vehicle lane change trajectory. At the same time, prediction models based on neural networks have been widely used in the field of lane change trajectory prediction.
[0004] However, traditional neural networks find it difficult to capture the relationship between two distant time series data, resulting in insufficient accuracy in long sequence prediction problems. Summary of the invention
[0005] In order to overcome the defects of the above-mentioned existing PBFT consensus mechanism, the present invention provides a vehicle lane change trajectory prediction method based on random forest and improved Informer model.
[0006] The technical solution of the present invention is:
[0007] The vehicle lane-changing trajectory prediction method based on random forest and improved Informer model includes the following steps:
[0008] Collect vehicle driving information based on vehicle sensors and public natural driving data sets;
[0009] Preprocessing the vehicle driving information to obtain preprocessed vehicle driving information;
[0010] The random forest algorithm is used to construct a data set for preprocessing vehicle driving information;
[0011] The vehicle lane-changing trajectory is predicted by training a model based on the characteristics of vehicle data and the need for vehicle lane-changing trajectory prediction.
[0012] As a further technical solution of the present invention, the vehicle driving information is collected based on the vehicle-mounted sensors and the public natural driving data set. Specifically, the vehicle driving information includes the vehicle historical trajectory, speed, acceleration, lateral speed, lateral acceleration, and leading vehicle distance information collected based on the vehicle-mounted sensors and the public natural driving data set.
[0013] As a further technical solution of the present invention, the preprocessing of the vehicle driving information to obtain the preprocessed vehicle driving information specifically includes:
[0014] Abnormal lane-changing behaviors caused by traffic accidents are eliminated, and the Savitzky-Golay (SG) filter is used to smooth the data to reduce noise; the distance that the vehicle deviates from the lane centerline is calculated through the historical trajectory and the lane centerline.
[0015] As a further technical solution of the present invention, the pre-processing vehicle driving information adopts a random forest algorithm to construct a data set; specifically comprising:
[0016] The random forest algorithm is used to build multiple decision tree models, and the ensemble learning idea is used to integrate the prediction results of multiple models. In the regression problem, the variance reduction is used to calculate the influence of different features on the target variable (the distance of the vehicle offset from the center line of the lane). The greater the variance reduction, the greater the influence of the feature on the target variable, and the higher the importance score. The features with larger importance scores are selected as the input features of the Informer model.
[0017] Construct training sets, validation sets, and test sets based on the data obtained after random forest algorithm screening;
[0018] The mean and standard deviation of the data for the training set, validation set, and test set are calculated respectively, and the data are standardized using the standardized calculation formula.
[0019] Furthermore, the data set after the random forest algorithm is selected, the first 70% of the data set is selected as the training set, 10% as the validation set, and the last 20% as the test set. The mean and standard deviation σ of the data are respectively obtained, and the standardized data of the training set, validation set, and test set are obtained by the standardized calculation formula: X = (x-mean) / σ, where x is the sample data before standardization and X is the sample data after standardization.
[0020] As a further technical solution of the present invention, predicting the vehicle lane changing trajectory by training a model according to the characteristics of the vehicle data and the vehicle lane changing trajectory prediction requirements specifically includes:
[0021] Step 1: Set the model's hyperparameters according to the characteristics of vehicle data and the requirements for vehicle lane change trajectory prediction. The model's hyperparameters include input sequence length --seq_len, prediction sequence length --pred_len, model iteration number Epochs, batch processing sample number batch_size, learning rate --learning_rate, Dropout, and loss function.
[0022] Step 2: Generate an encoder, take out the data of Batch_size from the training set and input it into the encoder embedding layer for position information encoding, add the local timestamp (PE) and the global timestamp (SE) and map it into a vector of the length of d_model (embedding dimension);
[0023] Step 3: The encoder uses the improved self-attention mechanism to capture the global information of the input data and the sequence data, and captures the local context information of the input data and the neighboring data by adding Gaussian bias, and determines the captured dependency through the stacking of encoder layers;
[0024] Step 4: Generate a decoder. The decoder and the generator use the same structure. After the input sequence passes through a self-attention layer, it performs another attention calculation with the output of the generator in the encoder-decoder layer, and finally passes through a fully connected layer to obtain the final output. The loss function is calculated by comparing the output with the true value.
[0025] Step 5: Repeat steps 2, 3, and 4 until the model converges and the training is terminated. The vehicle's lane-changing trajectory is predicted based on the trained model.
[0026] Furthermore, the encoder is generated, specifically including: the data uniformly converted by the embedding layer is input to the encoder part of the model, the sparse self-attention (ProbSparseSelf-attention) is calculated in the attention module, and the Gaussian bias G is added to enhance the local temporal information. i,j The specific method is to select the N queries with the highest sparsity scores, only calculate the dot product results of N queries and keys, and not calculate the remaining LN queries.
[0027] Furthermore, the generation of the decoder includes: extracting the dominant features of the attention mechanism through the Distilling layer, and obtaining the input of the decoder Decoder through stacking the combination of Self-attention+Distilling layers.
[0028] Furthermore, the loss function is calculated by comparing the output with the true value, specifically: batch generative prediction is adopted by the decoder in the improved Informer model, and the final output of the long sequence is obtained by only one forward calculation, and the loss function is calculated by comparing the output with the true value.
[0029] Furthermore, the loss function uses mean square error (MSE) and mean absolute error (MAE).
[0030] The beneficial effects of the present invention are:
[0031] 1. The present invention utilizes SG filtering to effectively reduce the noise in vehicle-mounted sensors and natural driving data sets. On this basis, the random forest algorithm is used as a feature screening model, and the Bootstrap Sampling technology is adopted to increase the richness of training data and improve the generalization ability of the model. The attribute random selection technology is used to avoid the influence of other attributes on the classification effect, improve the robustness of the model, and reduce the risk of overfitting. Multiple decision trees are calculated in parallel to improve the computing efficiency. Finally, the results of multiple decision trees are integrated through the ensemble learning method to improve the data validity and randomness.
[0032] 2. In the problem of predicting vehicle lane-changing trajectories, according to the characteristics of vehicle driving time series data, the longer the interval, the smaller the correlation of the time series data. The self-attention mechanism in the traditional Informer model will cause the model to pay too much attention to long-term dependencies. By introducing Gaussian bias through the improved self-attention mechanism to strengthen the local time series information in the model, the calculation of self-attention in the long sequence prediction problem can not only capture long-term dependencies, but also strengthen local time series information, thereby improving the accuracy of the Informer model in the problem of predicting vehicle lane-changing trajectories and reducing the risk of model overfitting.
[0033] 3. Use the improved Infomer model to predict time series. The probabilistic sparse self-attention mechanism, self-attention distillation technology and generative decoder in the model can significantly reduce the size and amount of calculation of the model while maintaining high accuracy when solving complex time series problems such as vehicle lane change trajectory prediction, thereby reducing the time and resource consumption of model operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 A flow chart of a vehicle lane-changing trajectory prediction method based on random forest and improved Informer model provided by the present invention;
[0035] Figure 2 A flow chart of a specific embodiment of the present invention;
[0036] Figure 3A flow chart of the random forest algorithm proposed in the present invention for screening vehicle driving data features;
[0037] Figure 4 This is a diagram of the improved Informer model for lane change trajectory prediction proposed by the present invention. DETAILED DESCRIPTION
[0038] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.
[0039] The neural network model based on the self-attention mechanism adopted in the present invention can pay close attention to two time series data with a large distance between them and learn the characteristic relationship between them, so as to effectively extract global information. However, due to the characteristic that the correlation of vehicle time series data decreases with the increase of time interval, the traditional time series prediction model based on the self-attention mechanism, when applied to the field of vehicle lane change trajectory prediction, needs to strengthen the capture of local context information on the basis of capturing long-distance dependencies, so as to improve the accuracy of the model in solving the problem of vehicle lane change trajectory prediction and reduce the risk of model overfitting.
[0040] See also Figure 1 , a vehicle lane-changing trajectory prediction method based on random forest and improved Informer model includes the following steps:
[0041] Step 101, collecting vehicle driving information based on vehicle-mounted sensors and public natural driving data sets;
[0042] Step 102, preprocessing the vehicle driving information to obtain preprocessed vehicle driving information;
[0043] Step 103, constructing a data set using a random forest algorithm for the pre-processed vehicle driving information;
[0044] Step 104 , predicting the vehicle lane changing trajectory through a training model according to the characteristics of the vehicle data and the vehicle lane changing trajectory prediction requirements.
[0045] In step 101, the vehicle driving information is collected based on the vehicle-mounted sensors and the public natural driving data set. Specifically, the vehicle driving information includes the vehicle's historical trajectory, speed, acceleration, lateral speed, lateral acceleration, and leading vehicle distance information collected based on the vehicle-mounted sensors and the public natural driving data set.
[0046] In step 102, the vehicle driving information is preprocessed to obtain preprocessed vehicle driving information; specifically, the process includes:
[0047] Abnormal lane-changing behaviors caused by traffic accidents are eliminated, and the Savitzky-Golay (SG) filter is used to smooth the data to reduce noise; the distance that the vehicle deviates from the lane centerline is calculated through the historical trajectory and the lane centerline.
[0048] In the embodiment of the present invention, due to the influence of the vehicle-mounted sensor or the camera algorithm in the natural driving data set and other possibilities, the data has a certain amount of noise, and a Savitzky-Golay (SG) filter is used to smooth the data to reduce the noise. The SG filter is a filtering method based on the local polynomial least squares fitting in the time domain, which is usually used for signal smoothing on discrete data signals. It uses a sliding window of a fixed length to fit a polynomial curve to smooth the input signal. The biggest feature of this filter is that it can ensure that the shape and width of the signal remain unchanged while filtering out the noise.
[0049] In step 103, the pre-processed vehicle driving information is subjected to a random forest algorithm to construct a data set; specifically, the step includes:
[0050] The random forest algorithm is used to build multiple decision tree models, and the ensemble learning idea is used to integrate the prediction results of multiple models. In the regression problem, the variance reduction is used to calculate the influence of different features on the target variable (the distance of the vehicle offset from the center line of the lane). The greater the variance reduction, the greater the influence of the feature on the target variable, and the higher the importance score. The features with larger importance scores are selected as the input features of the Informer model.
[0051] The training set, validation set, and test set are constructed based on the data obtained after screening by the random forest algorithm; the mean and standard deviation of the data are calculated for the training set, validation set, and test set respectively, and the data are standardized using the standardized calculation formula.
[0052] In the embodiment of the present invention, the data after noise elimination is divided into a training set and a test set, and a random forest algorithm is used to construct multiple decision tree models according to the vehicle driving data. Each decision tree model is generated based on random sampling within the sample, and then a certain number of features are randomly selected from these samples for splitting. The results of multiple prediction models are integrated using sample averaging or voting methods. The variance reduction is used to calculate the influence of different features on the target variable (the distance of the vehicle offset from the center line of the lane) in the regression problem. The greater the variance reduction, the greater the influence of the feature on the target variable, and the higher the importance score. The features with larger importance scores are selected as the input features of the Informer model.
[0053] Among them, based on the data set screened by the random forest algorithm, the first 70% of the data set is selected as the training set, 10% as the validation set, and the last 20% as the test set. The mean and standard deviation σ of the data are calculated respectively, and the standardized data of the training set, validation set, and test set are obtained through the standardized calculation formula: X = (x-mean) / σ, where x is the sample data before standardization and X is the sample data after standardization.
[0054] In step 104, the vehicle lane changing trajectory is predicted by training a model according to the characteristics of the vehicle data and the vehicle lane changing trajectory prediction requirements, specifically including:
[0055] Step 401, according to the characteristics of vehicle data and the prediction requirements of vehicle lane change trajectory, set the hyper parameters of the model; wherein the hyper parameters of the model include input sequence length --seq_len, prediction sequence length --pred_len, model iteration number Epochs, batch processing sample number batch_size, learning rate --learning_rate, Dropout, and loss function;
[0056] Step 402, generate an encoder, take out data of Batch_size from the training set and input it into the encoder embedding layer for position information encoding, add the local timestamp (PE) and the global timestamp (SE) and then map it into a vector of the length of d_model (embedding dimension);
[0057] Step 403, the encoder uses an improved self-attention mechanism to capture the global information of the input data and the sequence data, and captures the local context information of the input data and the neighboring data by adding a Gaussian bias, and determines the captured dependency by stacking the encoder layers;
[0058] Step 404, generate a decoder. The decoder and the generator use the same structure. After the input sequence passes through a self-attention layer, it performs another attention calculation with the output of the generator in the encoder-decoder layer, and finally passes through a fully connected layer to obtain the final output. The loss function is calculated by comparing the output with the true value.
[0059] Step 405, repeat steps 402, 403 and 404 until the model converges and the training is terminated, and the lane change trajectory of the vehicle is predicted based on the trained model.
[0060] In step 402, generating an encoder specifically includes:
[0061] The data converted uniformly by the embedding layer is input to the encoder part of the model, and the sparse self-attention (ProbSparse Self-attention) is calculated in the attention module, and a Gaussian bias G is added to enhance the local temporal information. i,j The specific method is to select the N queries with the highest sparsity scores, only calculate the dot product results of N queries and keys, and not calculate the remaining LN queries.
[0062] In the embodiment of the present invention, for the long sequence prediction problem, a global timestamp SE is added, and the input embedding consists of three independent parts: scalar projection, local timestamp (PE) and global timestamp (SE).
[0063] In the embodiment of the present invention, the uniformly converted data is input to the encoder part of the model, and the sparse self-attention (ProbSparse Self-attention) calculation is performed in the attention module to improve the model calculation speed. When performing self-attention calculation, since the correlation of vehicle driving data in time series data decreases with the increase of time interval, if the model over-captures long-distance dependencies, it will cause the model to pay too much attention to the global information related to the data and the overall sequence and ignore the local time series information that is very important to the vehicle time series data, resulting in the risk of overfitting the model. Therefore, a Gaussian bias G is added to enhance the local time series information. i,j To enhance the accuracy of the traditional Informer model in the vehicle lane change trajectory prediction problem.
[0064] In step 404, generating a decoder includes: extracting the dominant features of the dominant attention mechanism through the Distilling layer, and obtaining the input of the decoder Decoder through stacking the combination of Self-attention+Distilling layers.
[0065] In an embodiment of the present invention, in order to solve the problem of high memory usage of the Informer model in predicting long sequence problems, the advantageous features of the dominant attention mechanism are extracted through the Distilling layer, and the input of the decoder is obtained by stacking the combination of Self-attention + Distilling layers.
[0066] In an embodiment of the present invention, the loss function is calculated by the output and the true value, specifically: the decoder in the improved Informer model adopts batch generative prediction, and the final output of the long sequence is obtained through only one forward calculation, and the loss function is calculated by the output and the true value.
[0067] Among them, the loss function uses mean square error MSE and mean absolute error MAE, and uses mean square error MSE and mean absolute error MAE as indicators to measure the performance of the prediction model, where the mean square error is the average of the sum of squares of the differences between all predicted values and the true values, and the mean absolute error is the average of the absolute values of the differences between all predicted values and the true values.
[0068] The present invention utilizes SG filtering to effectively reduce the noise in vehicle-mounted sensors or natural driving data sets. On this basis, a random forest algorithm is used as a feature screening model, and BootstrapSampling technology is adopted to increase the richness of training data and improve the generalization ability of the model. The attribute random selection technology is used to avoid the influence of other attributes on the classification effect, improve the robustness of the model, and reduce the risk of overfitting. Multiple decision trees are calculated in parallel to improve the computing efficiency. Finally, the results of multiple decision trees are synthesized through an integrated learning method to improve data validity and randomness.
[0069] In the problem of predicting vehicle lane-changing trajectories, according to the characteristics of vehicle driving time series data, the longer the interval, the smaller the correlation of the time series data. The self-attention mechanism in the traditional Informer model will cause the model to pay too much attention to long-term dependencies. By introducing Gaussian bias through the improved self-attention mechanism to strengthen the local time series information in the model, the calculation of self-attention in the long sequence prediction problem can not only capture long-term dependencies, but also strengthen local time series information, thereby improving the accuracy of the Informer model in the problem of predicting vehicle lane-changing trajectories and reducing the risk of model overfitting.
[0070] The improved Infomer model is used to predict time series. The probabilistic sparse self-attention mechanism, self-attention distillation technology and generative decoder in the model can significantly reduce the size and amount of calculation of the model while maintaining high accuracy when solving complex time series problems such as vehicle lane change trajectory prediction, thereby reducing the time and resource consumption of model operation.
[0071] The following will refer to the attached Figure 1 The specific steps of describing a vehicle lane change trajectory prediction method based on random forest and improved Informer model according to an embodiment of the present invention are as follows:
[0072] (1) Data extraction
[0073] Based on the public road traffic data set, the vehicle's historical trajectory, speed, acceleration, lateral speed, lateral acceleration, and time distance to the front vehicle are obtained. Due to the special road policy for motorcycles, motorcycle data is excluded. The present invention is mainly used for trajectory prediction in free lane change scenarios, so the abnormal lane change behavior information caused by traffic accidents is excluded, and the distance of the vehicle offset from the lane centerline is calculated through the historical trajectory and the lane centerline coordinates.
[0074] (2) SG filter preprocessing of vehicle driving data
[0075] Affected by the on-board sensors and the camera algorithms in the public road traffic data set and other possibilities, the data has certain noise, which may lead to the overfitting of the nonlinear model. By using the SG filtering method to smooth the original data and reduce the interference of noise, the overfitting of the nonlinear model can be effectively suppressed. The principle of SG filtering is as follows:
[0076] SG filtering is a filtering method based on local polynomial least squares fitting in the time domain. It is an improvement on the moving smoothing algorithm. The biggest feature of this filter is that it can ensure that the shape and width of the signal remain unchanged while filtering out noise. Using SG filtering can effectively reduce data noise and reduce the occurrence of model overfitting. The core idea is to perform weighted filtering on the data in the window, set the width of the filter window n = 2m + 1, the sampling points are x = (-m, -m + 1, ... 0, 1, ..., m-1, m), and use k-1 order polynomial to fit the filter window data:
[0077] y=a0+a1x+a2x 2 +…+a k-1 x k-1 ;
[0078] In the formula, x is the data to be fitted, y is the output data, and a is the parameter to be solved. The larger k is, the closer the smooth curve is to the true value.
[0079] By juxtaposing n of the above polynomials, we can obtain the following matrix operation:
[0080]
[0081] In the formula (ε x-m ...ε x ...ε x+m ) represents the prediction error vector, which is the error between the actual data point and the predicted value.
[0082] This simplifies to the following overdetermined system of equations:
[0083] Y (2m+1)×1 =X (2m+1)×k ·A K×1 +E(2m+1)×1 ;
[0084] Where Y represents the filtered value, X represents the observed value, A represents the parameter to be solved, and E represents the prediction error vector. The subscripts of each parameter represent their respective dimensions, such as A K×1 It means there are k rows and 1 column of parameters. By the least squares method, we can get A K×1 The solution is:
[0085]
[0086] The calculation formula of the filter value Y of the model is:
[0087]
[0088] Where B is the filter coefficient matrix, which is determined by and only by the X matrix. The B matrix is a (2m+1)×(2m+1)-order matrix. The SG smooth fitting equation can be obtained based on the coefficient matrix. The formula is:
[0089] B=X·(X T ·X) -1 ·X T ;
[0090] (3) Random forest feature selection
[0091] The random forest algorithm is used to construct multiple decision tree models for the preprocessed vehicle driving information. The Bootstrap Sampling technology is adopted. Each decision tree model is generated based on random sampling within the sample, which increases the generalization of the model.
[0092] The principle of Bootstrap Sampling technology is that for a given dataset D of m samples, we randomly sample it to obtain a dataset The process is to randomly select a sample from D each time, copy it and put it into Then put the sample back into the original data set D. Due to the random selection with replacement, the sample data may still be selected in the next random sampling. Finally, after m sampling processes, a random sampling data set containing m samples is obtained. After n random sampling data sets are constructed, n data sets containing different samples are obtained to build a decision tree. For a sample, the probability that it is never sampled in m samplings is Taking the limit we get:
[0093]
[0094] The result in the formula means that approximately 36.8% of the samples do not appear in the dataset after sampling. In the above example, the test set is constructed from samples that do not appear in the model. In the independent sampling, the test set accounts for 36.8% of the total data volume and is used to evaluate the generalization ability of the model.
[0095] In these samples, a certain number of features are randomly selected by using the attribute random selection technique to form a feature subset for training the decision tree. When selecting an attribute for splitting at each node, first randomly select an attribute subset containing k attributes (k < d) from the attribute set, and then select an optimal attribute from this subset for splitting. Generally, the value of k is taken as k = log2 n ; The attribute random selection technique avoids the influence of other features on the classification effect, reduces the correlation between decision trees, and since not all features are used to construct the decision tree at one time, the calculation efficiency is greatly increased.
[0096] The majority voting method is used to synthesize the results of multiple prediction models to obtain the influence degree of each feature on the target variable (the distance between the vehicle and the center line of the lane).
[0097] (4) Construction of the training set, validation set, and test set
[0098] The random forest algorithm eliminates the feature data that has little or no influence on the lane-changing trajectory. For the dataset obtained after screening based on the random forest algorithm, the first 70% is taken to construct the training set, 20% to construct the validation set, and 10% to construct the test set. To eliminate the influence of different feature dimensions on machine learning, the Z-score normalization is performed on the training set, validation set, and test set respectively. The Z-score normalization can scale the data to a standard normal distribution with a mean of 0 and a standard deviation of 1, eliminating the influence between different dimensions. The formula is:
[0099] X = (x - mean) / σ;
[0100] where mean is the mean of the feature data and σ is the standard deviation of the feature data.
[0101] By performing the Z-score normalization, all features can be scaled to the same range, making it easier for the model to learn the relationships and differences between features, accelerating the model convergence, and improving the model operation speed.
[0102] (5) Construct an improved Informer time series prediction model to predict the vehicle lane-changing trajectory
[0103] In the present invention, the improved Informer model is used to predict the vehicle lane-changing trajectory. The vehicle lane-changing trajectory multivariate dataset obtained from the standardized data is used to achieve the prediction of the vehicle lane-changing trajectory. The execution process of the model is as follows:
[0104] The hyperparameters of the model are set including the input sequence length --seq_len, the predicted sequence length --pred_len, the number of model iterations Epochs, the number of batch samples batch_size, the learning rate --learning_rate, Dropout, the loss function, the optimizer, etc.
[0105] Each time, we take out batch_size sample data from the dataset and perform input-embedding operation uniformly. The embedding operation converts the original time series data into a set of vector representations. The conversion formula of the input data is:
[0106]
[0107] Where: is the characteristic scalar of the original data, t represents the current time step, PE is the local timestamp, SE is the global timestamp, i∈{1,…,L x}, L x is the number of time steps in a sample, α is a factor that balances the size between scalar mapping and local / global embedding, and α=1 if the sequence input is normalized.
[0108] (a) For characteristic scalar The model uses a one-dimensional convolution filter to transform the scalar context Convert to 512-dimensional vector
[0109] (b) For the local timestamp (PE), the Positional Emebdding in the traditional Transformer is used, and the calculation formula is:
[0110]
[0111]
[0112] In the formula, d model is the dimension of the input vector, 2j represents an even dimension, and 2j+1 represents an odd dimension. L x is the number of time steps in the sample, and pos represents the position of a single data in the overall sequence. Based on this formula, different positions can be characterized, and the dimensions of pos (d model ), all contain certain location information.
[0113] (c) For the global timestamp (SE), a fully connected layer is used to map the input timestamp to a 512-dimensional embedding to help the model better learn the temporal features of time series data.
[0114] In an embodiment of the present invention, a specific method for generating an encoder includes:
[0115] The input data transformed uniformly by the embedding layer Input to the encoder part of the model.
[0116] The output of self-attention in the traditional Informer model is expressed as:
[0117]
[0118]
[0119]
[0120] Where x represents the input vector sequence, i is the position of the query element in the sequence, j is the position of the queried element in the sequence, and z i is the output vector, α ij is the connection weight, e ij Q i With k j The attention size is obtained by using the dot product. z For the hidden layer dimension, select It can make (x i W Q )(x j W K ) T The calculation result satisfies the distribution with expectation of 0 and variance of 1. Q , W K , W V are the learnable parameter matrices respectively.
[0121] For the problem of vehicle lane change trajectory prediction, traditional self-attention can capture long-distance dependencies very well. However, due to the characteristic of physical trajectory time series data that the shorter the time interval, the stronger the correlation may be, we need to strengthen the capture of local attention relationships while considering global information. This method adds a Gaussian bias term to self-attention. This method can make the attention weight higher when it is close to the center position and gradually decrease when it is far away from the center position. In this way, it is easier to find local attention patterns in long sequences and obtain better attention weighted results. The specific implementation method is to introduce a Gaussian bias in the traditional self-attention calculation method. The calculation of the above self-attention method can be modified as follows:
[0122]
[0123]
[0124]
[0125] in,
[0126]
[0127] Among them G ij is the Gaussian bias added in the self-attention calculation, G ik is the Gaussian bias of each point when calculating self-attention in the entire time series, P i is the center position, i.e. the position of the query vector, j is the position of the queried vector, Usually set to D i Represents the size of the window. For the window size, a fixed length of 10 is used.
[0128] Through the Gaussian bias G ij With the introduction of , the model can make full use of the characteristics of vehicle time series data in the lane change trajectory prediction problem, strengthen the capture of the correlation of short-distance time series data, and improve the performance of the model in the lane change trajectory prediction problem.
[0129] Based on the improved self-attention mechanism, sparse self-attention (ProbSparse Self-attention) calculation is performed. Each key only focuses on u main queries, Q is the query matrix, K is the key matrix, V is the value, and d is the dimension of the hidden layer vector. The calculation formula is:
[0130]
[0131] In the formula, The same size as Q, including the sparse metric M(q i ,K) to get the top-u queries with the highest sparsity scores. The specific calculation method of the sparsity metric is: c is the sampling factor, set u = clnL q ; Randomly sample c*lnL keys for each query and calculate the sparsity score M(qi, K) of each query; The calculation formula of the sparse metric M(qi, K) is:
[0132]
[0133] Where q i ,k i ,v i are Q, K, and V, respectively, and d represents q i The dimension, L K Represents the number of time steps in the sample.
[0134] Select the N queries with the highest sparsity scores. N defaults to c*lnL. Only the dot product results of the N queries and keys are calculated. The self-attention scores of the remaining points are directly replaced by the mean of the values to improve the efficiency of self-attention calculation.
[0135] In order to better capture the correlation between time series data, a multi-layer encoder stack is used. The output of the previous encoder layer calculated by the improved self-attention mechanism passes through the Distilling layer, which is specifically down-sampled through 1D maxpooling to reduce the length of the input sequence. The sampling values of the query and key are also reduced, and the computational complexity is further reduced without losing the original features. The input of the decoder is obtained by stacking the combination of Self-attention + Distilling layers.
[0136] The decoder used by the improved Informer is similar to the traditional decoder. In order to improve the computational efficiency of the model in long time series prediction problems, the decoder in the improved Informer model adopts batch generative prediction, which obtains the final output of the long sequence through only one forward calculation. Using batch generative prediction, the Informer requires the following input:
[0137]
[0138] in the formula is the original input sequence of the Decoder, is the starttoken sequence, To predict the sequence, fill it with 0. The Informer model does not use the token form in the traditional Tranformer model, but samples L in the input sequence. token In the lane change trajectory prediction problem, if we need to predict 50 trajectory points, we can use the first 50 known trajectory points as the starttoken, then the decoder input is
[0139] At the same time, in order to avoid autoregression, the self-attention layer calculation in the decoder introduces a masked mechanism, and finally passes through a fully connected layer to obtain the final output.
[0140] Using MSE (mean square error) or MAE (mean absolute error) as the loss function, the calculation formula is as follows:
[0141]
[0142] Where m is the number of samples, For real data, The model iteration is repeated until the training condition is terminated (the number of model iterations is reached or the early stopping mechanism is triggered because the loss does not decrease), and a trained model is generated, which can be used to predict the future lane change trajectory of the vehicle. The output of the model can obtain multi-step prediction values.
[0143] In order to verify the proposed vehicle lane change trajectory prediction method based on random forest and improved Informer model, a multivariate prediction experiment based on real traffic data set was conducted. The experimental environment uses Python development language and Pytorch deep learning framework. Experiment 1 verifies the improvement of the accuracy of the Informer model in predicting time series by the random forest algorithm. Experiment 2 verifies the effect of the proposed prediction method by comparing it with other neural network time series prediction methods and the Informer model without Gaussian bias.
[0144] The experimental data uses the Next Generation Simulation (NGSIM) data set of the Federal Highway Administration of the United States. It is the largest natural vehicle trajectory data set available, and contains information such as the vehicle's horizontal and vertical coordinates, vehicle speed and acceleration, and the time distance to the preceding vehicle. The vehicle's lateral speed and acceleration are calculated from the change in the vehicle's vertical coordinate value, and the distance the vehicle deviates from the lane centerline is calculated from the vehicle's horizontal coordinate and the lane centerline coordinate.
[0145] Experiment 1:
[0146] The random forest algorithm is used to screen the sample features of driving data, and the vehicle offset lane centerline distance is selected as the target variable vector. The remaining driving data features are used as feature matrices. The random forest algorithm is used to screen the features that have a greater impact on the target variable as the input features of the Informer model. The experimental environment uses the Python development language and the Random Forest Regressor method in the random forest algorithm package in the sklean library.
[0147] Table 1 The influence of different features in the sample on the distance of vehicle deviation from the lane center line
[0148]
[0149] The sum of the Permutation Importance values of each feature in Table 1 is 1, where a larger Permutation Importance value represents a higher degree of influence on the lane-changing trajectory.
[0150] The features with an influence degree less than 0.2 are eliminated. The final input sample features include vehicle speed, lateral speed, and lateral acceleration, which are used as the input of the Informer model. The comparative experiment uses the decision tree algorithm to select vehicle acceleration, vehicle speed, and lateral speed as input data, as well as all features that have not been screened as input data. The prediction sequence length is 40.
[0151] Table 2
[0152]
[0153] It can be seen from Table 2 that the mean square error MSE and mean absolute error MAE of the random forest algorithm of the present invention are both smaller than those of the decision tree and the unscreened input data.
[0154] Experiment 2:
[0155] This method will be compared with the lane change trajectory prediction model based on the LSTM network and the unimproved Informer model. The long short-term memory network (LSTM) is a special RNN that mainly solves the gradient vanishing and gradient exploding problems of RNN in the long sequence training process and is widely used in time series prediction problems. The comparison results are shown in Table 3.
[0156] Table 3
[0157]
[0158] It can be seen from Table 3 that, for different prediction sequence lengths, the mean absolute error of the mean square error of the present invention is smaller than that of the LSTM method and the traditional Informer method, and the prediction accuracy is high.
[0159] The present invention is described in detail above, but the present invention is not limited to the above-mentioned embodiments. Various changes can be made within the knowledge of ordinary technicians in the field without departing from the purpose of the present invention. Many other changes and modifications can be made without departing from the concept and scope of the present invention. It should be understood that the present invention is not limited to specific embodiments, and the scope of the present invention is defined by the appended claims.
Claims
1. Vehicle lane-changing trajectory prediction method based on random forest and improved Informer model, It is characterized in that The steps include: Collect vehicle driving information based on vehicle sensors and public natural driving data sets; Preprocessing the vehicle driving information to obtain preprocessed vehicle driving information; The random forest algorithm is used to construct a data set for preprocessing vehicle driving information; According to the characteristics of vehicle data and the need for predicting vehicle lane-changing trajectories, a training model is established to predict the vehicle lane-changing trajectories; The method of predicting the vehicle lane changing trajectory by training a model according to the characteristics of the vehicle data and the vehicle lane changing trajectory prediction requirements specifically includes: Step 1: Set the model's hyperparameters according to the characteristics of vehicle data and the requirements for vehicle lane change trajectory prediction. The model's hyperparameters include the input sequence length seq_len, the prediction sequence length pred_len, the model iteration number Epochs, the batch sample number batch_size, the learning rate learning_rate, and the loss function Dropout. Step 2: Generate an encoder, take out the data of Batch_size from the training set and input it into the encoder embedding layer for position information encoding, add the local timestamp and global timestamp and map it into a vector of length d_model; Step 3: The encoder uses the improved self-attention mechanism to capture the global information of the input data and the sequence data, and captures the local context information of the input data and the neighboring data by adding Gaussian bias, and determines the captured dependency through the stacking of encoder layers; Step 4: Generate a decoder. The decoder and the generator use the same structure. After the input sequence passes through a self-attention layer, it performs another attention calculation with the output of the generator in the encoder-decoder layer, and finally passes through a fully connected layer to obtain the final output. The loss function is calculated by comparing the output with the true value. Step 5: Repeat steps 2, 3, and 4, iterate the model, and calculate the loss value through the loss function until the loss function value converges to a smaller stable value. The improved Informer model converges, the training is terminated, and the vehicle's lane change trajectory is predicted based on the trained model.
2. The vehicle lane change trajectory prediction method based on random forest and improved Informer model according to claim 1, It is characterized in that The vehicle driving information is collected based on the vehicle-mounted sensors and the public natural driving data set. Specifically, the vehicle driving information includes the vehicle historical trajectory, speed, acceleration, lateral speed, lateral acceleration, and leading vehicle distance information collected based on the vehicle-mounted sensors and the public natural driving data set.
3. The vehicle lane change trajectory prediction method based on random forest and improved Informer model according to claim 1, It is characterized in that The preprocessing of the vehicle driving information to obtain the preprocessed vehicle driving information specifically includes: Abnormal lane-changing behaviors caused by traffic accidents are eliminated, and the SG filter is used to smooth the data to reduce noise; the distance that the vehicle deviates from the lane centerline is calculated through the historical trajectory and the lane centerline.
4. The vehicle lane change trajectory prediction method based on random forest and improved Informer model according to claim 1, It is characterized in that The method of using a random forest algorithm to construct a data set for preprocessing vehicle driving information specifically includes: The random forest algorithm is used to build multiple decision tree models, and the prediction results of multiple models are integrated using the ensemble learning idea. In the regression problem, the variance reduction is used to calculate the influence of different features on the target variable. The greater the variance reduction, the greater the influence of the feature on the target variable and the higher the importance score. The features with larger importance scores are selected as the input features of the Informer model. Construct training sets, validation sets, and test sets based on the data obtained after random forest algorithm screening; The mean and standard deviation of the data for the training set, validation set, and test set are calculated respectively, and the data are standardized using the standardized calculation formula.
5. The vehicle lane-changing trajectory prediction method based on random forest and improved Informer model according to claim 4, It is characterized in that The training set, validation set and test set are constructed based on the data obtained after screening by the random forest algorithm; specifically, the first 70% of the data set is selected as the training set, 10% as the validation set, and the last 20% as the test set; and the mean mean and standard deviation σ of the data are respectively obtained, and the standardized data of the training set, validation set and test set are obtained through the standardized calculation formula: X=(x-mean) / σ, where x is the sample data before standardization and X is the sample data after standardization.
6. The vehicle lane-changing trajectory prediction method based on random forest and improved Informer model according to claim 1, It is characterized in that The generating encoder specifically comprises: The data converted uniformly by the embedding layer is input to the encoder part of the model, sparse self-attention calculation is performed in the attention module, and a Gaussian bias G is added to enhance the local temporal information. i,j The specific method is to select the N queries with the highest sparsity scores, only calculate the dot product results of N queries and keys, and not calculate the remaining LN queries.
7. The vehicle lane-changing trajectory prediction method based on random forest and improved Informer model according to claim 6, It is characterized in that The generation of the decoder includes: extracting the dominant features of the attention mechanism through the Distilling layer, and obtaining the input of the decoder Decoder through the stacking of the Self-attention+Distilling layer combination.
8. The vehicle lane-changing trajectory prediction method based on random forest and improved Informer model according to claim 1, It is characterized in that The loss function is calculated by comparing the output with the true value, specifically: batch generative prediction is adopted by the decoder in the improved Informer model, the final output of the long sequence is obtained by only one forward calculation, and the loss function is calculated by comparing the output with the true value.
9. The vehicle lane-changing trajectory prediction method based on random forest and improved Informer model according to claim 8, It is characterized in that The loss function uses mean square error (MSE) and mean absolute error (MAE).
Citation Information
Patent Citations
Vehicle driving track type detection method and device
CN110288032A
Vehicle trajectory prediction method based on three-stage attention mechanism
CN114495036A
Driving behavior prediction method based on informer neural network in safety potential field
CN115547040A
Vehicle positioning method and device based on Informer network, and computer
CN115755134A
System and method for predicting motion state of surrounding vehicle based on driving intention
WO2021077725A1
Cited By
A Method for Predicting and Controlling the Trajectory of Goods Placement on a Clamping Cart Based on Transformer Networks
CN119960438B
