A vehicle trajectory fusion prediction method based on deep learning and kinematic model

Abstract

The present invention discloses a vehicle trajectory fusion prediction method based on deep learning and kinematic models, comprising: collecting scene information of a target vehicle when it is traveling, and using a deep learning prediction model to predict state information of the target vehicle in a future time domain; using the predicted state information of the target vehicle to establish a target vehicle trajectory prediction model through a Kalman filter, and predicting the kinematic state information of the target vehicle at the next moment; using the comprehensive state information of the target vehicle at the current moment, the predicted state information of the target vehicle at the next moment, the predicted kinematic state information of the target vehicle at the next moment, etc. as inputs of a Weight_LSTM network model, predicting a weight coefficient of a deep learning prediction model at the next moment and a weight coefficient of a target vehicle trajectory prediction model, weighting the predicted state information of the target vehicle and the predicted kinematic state information of the target vehicle, and predicting the comprehensive state information of the target vehicle at the next moment.

Description

Technical Field

[0001] The present invention relates to the technical field of trajectory prediction of autonomous driving vehicles, and in particular to a vehicle trajectory fusion prediction method based on deep learning and kinematic models. Background Art

[0002] In autonomous driving and advanced driver assistance systems, vehicle trajectory prediction is a key task. The purpose of vehicle trajectory prediction is to predict the future movement path of the vehicle based on its historical state and surrounding environment. Accurate vehicle trajectory prediction can improve the safety and reliability of autonomous driving systems and provide basic support for path planning and decision-making.

[0003] Vehicle prediction methods based on physical models - based on kinematic and dynamic equation modeling to achieve short-term accurate modeling, but in complex scenarios and long-term prediction tasks, accuracy is difficult to guarantee. Vehicle prediction methods based on deep learning can achieve predictions in multiple feature dimensions by capturing complex nonlinear relationships in model data, building network models, and using a large amount of data for training. Under the premise of a large amount of data, they can understand the implicit interactive relationships between multiple data and improve the accuracy of trajectory prediction, but they require a large amount of training data and have limited ability to handle abnormal situations; in addition, deep learning models usually only rely on data for training, lack an explicit understanding of vehicle dynamics and kinematic constraints, and may generate trajectories that do not conform to actual physical laws, especially in complex dynamic environments. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides a vehicle trajectory fusion prediction method based on deep learning and kinematic models, which combines the accuracy of physical models and the flexibility of machine learning models. It can provide more reliable vehicle trajectory prediction in complex dynamic environments and provide support for subsequent path planning and decision-making of intelligent vehicles.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution: a vehicle trajectory fusion prediction method based on deep learning and kinematic model, specifically comprising the following steps:

[0006] Step S1, collecting scene information when the target vehicle is driving, the scene information includes: state information of the target vehicle and state information of surrounding vehicles;

[0007] Step S2: predict the state information of the target vehicle in the future time domain using a deep learning prediction model based on the collected scene information, and obtain the state information of the target vehicle at each moment in the future time domain;

[0008] Step S3, using the state information of the target vehicle at each moment in the future time domain to establish a target vehicle trajectory prediction model through a Kalman filter, and predicting the kinematic state information of the target vehicle at the next moment;

[0009] Step S4, taking the comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment as the input of the Weight_LSTM network model, predicting the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model;

[0010] Step S5: weight the state information of the target vehicle at the next moment predicted by step S2 and the kinematic state information of the target vehicle at the next moment predicted by step S3 by using the weight coefficient of the predicted deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model to predict the comprehensive state information of the target vehicle at the next moment.

[0011] Furthermore, step 2 includes the following sub-steps:

[0012] Step S2.1, filtering and standardizing the collected scene information to obtain standardized state information;

[0013] Step S2.2, the deep learning prediction model includes an LSTM encoder and a Transformer decoder connected in sequence;

[0014] Step S2.3, input the standardized state information into the LSTM encoder for encoding to generate the hidden state of each time step and the hidden state of the last time step;

[0015] Step S2.4, repeatedly map the hidden state of the last time step to the input dimension of the Transformer decoder, add position encoding for each repeated time step, obtain the hidden state space with position encoding, and process the hidden state space with position encoding and the hidden state of each time step output by the LSTM encoder in parallel through the multi-head self-attention mechanism to generate the state information of the target vehicle in the future time domain;

[0016] Step S2.5, repeating steps S2.3-S2.4 until the mean square error loss function composed of the state information of the target vehicle in the future time domain and the actual state information of the target vehicle converges, completing the training of the deep learning prediction model;

[0017] Step S2.6: After filtering and standardizing the real-time scene information of the target vehicle, the information is input into the trained deep learning prediction model to predict the state information of the target vehicle in the future time domain.

[0018] Furthermore, the standardization process in step S2.1 is specifically as follows:

[0019]

[0020] Among them, I diff represents the differential scene information, I diff =I denoise t -I denoise t-1 , I denoise t represents the filtered scene information at time t, I max ,I min Respectively represent the maximum and minimum values ​​in the differential scene information.

[0021] Furthermore, step S2.3 includes the following sub-steps:

[0022] Step S2.3.1, the normalized state information is passed to the first layer of the LSTM encoder to generate a hidden state sequence for each time step;

[0023] Step S2.3.2: Use the generated hidden state sequence of each time step as the input of the next layer of the LSTM encoder to continue generating the hidden state sequence of each time step;

[0024] Step S2.3.3: Repeat step S2.3.2 until the hidden state of each time step and the hidden state of the last time step of the last layer in the LSTM encoder are obtained.

[0025] Furthermore, step S2.4 includes the following sub-steps:

[0026] Step S2.4.1 repeatedly maps the hidden state of the last time step to the hidden state space through the embedding layer of the Transformer decoder, generates a position code for each time step in the hidden state space through sine and cosine functions, and obtains a hidden state space vector with added position code;

[0027] Step S2.4.2, by using a multi-head self-attention mechanism, the hidden state space of the position encoding and the hidden state of each time step output by the LSTM encoder are processed in parallel to obtain the output semantics of the Transformer decoder;

[0028] Step S2.4.3: The output semantics of the Transformer decoder is passed through the fully connected layer to generate the state information of the target vehicle in the future time domain.

[0029] Furthermore, the calculation process of the hidden state space vector embeddingPE of the position encoding added in step S2.4.1 is:

[0030] embedding PE =embedding+PE

[0031] Where, embedding represents the hidden state space vector, embedding = We h′, We represents the weight matrix of the embedding layer, h′ represents the time step matrix of the hidden state repetition of the last time step; PE represents the position encoding matrix, which is composed of the 2xth component PE(s, 2x) of the encoding vector at position k and the 2x+1th component PE(s, 2x+1) of the encoding vector at position k. d represents the dimension of position encoding,

[0032] Furthermore, the specific process of step S3 is as follows: by obtaining the state information of the target vehicle at a certain moment in the future time domain, obtaining the prior state estimation value of the state information of the target vehicle at the next moment, combining the state information of the target vehicle at the next moment in the future time domain, constructing the target vehicle trajectory prediction model, and predicting the kinematic state information of the target vehicle at the next moment

[0033]

[0034] in, represents the prior state estimate of the target vehicle state information at time k, represents the state information of the target vehicle at time k-1 in the future time domain, A represents the coefficient matrix; K [k] represents the Kalman gain value at time k, H m represents the observation constant matrix, R represents the covariance matrix of the measurement noise, represents the covariance matrix of the prior state estimate at time k, P [k-1] represents the covariance matrix of the posterior state estimate at time k-1, Q represents the covariance matrix of the process noise; z [k] Represents the state information of the target vehicle at time k in the future time domain.

[0035] Furthermore, step S4 includes the following sub-steps:

[0036] Step S4.1, taking the comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment as the input of the Weight_LSTM network model, taking the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model as the label of the Weight_LSTM network model, and predicting the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model;

[0037] Step S4.2, repeat step S4.1 until the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model converge with the mean square error loss function of the corresponding label, and the training of the Weight_LSTM network model is completed;

[0038] Step S4.3, input the comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment into the trained Weight_LSTM network model to predict the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model.

[0039] Furthermore, the prediction uncertainty coefficient σ of the deep learning prediction model in step S4.1 is DL The calculation process is:

[0040]

[0041] The prediction uncertainty coefficient σ of the target vehicle trajectory prediction model KF The calculation process is:

[0042]

[0043] Among them, P k,ii The covariance matrix P represents the posterior state estimate at time k [k] The diagonal elements of I represents the identity matrix and n represents the number of diagonal elements.

[0044] Furthermore, the weight coefficient W of the deep learning prediction model at the next moment in step S4.1 labelDL and the weight coefficient W of the target vehicle trajectory prediction model KF The calculation process is:

[0045]

[0046] Among them, inv_e DL represents the inverse error of the deep learning prediction model, e DL represents the computational error of the deep learning prediction model, e DL =||z [k] -X [k] ||,X [k] represents the true value of the target vehicle's state information at time k, ∈ is a normal number; inv_e KF represents the inverse error of the target vehicle trajectory prediction model, e KF represents the calculation error of the target vehicle trajectory prediction model,

[0047] Compared with the prior art, the present invention has the following beneficial effects: the vehicle trajectory fusion prediction method based on deep learning and kinematic model of the present invention combines the advantages of deep learning and Kalman filtering, and can provide more accurate and reliable trajectory prediction in complex dynamic traffic environments to ensure safe driving of vehicles. First, the deep learning prediction model can capture the complex nonlinear relationships and temporal dependencies in the target vehicle scene information, which helps to make accurate predictions in various complex scenarios; the target vehicle trajectory prediction model is established through the Kalman filter to estimate the kinematic state information of the target vehicle, which can provide robust short-term predictions. The deep learning prediction model uses historical data for learning, while the target vehicle trajectory prediction model adds physical constraints for short-term predictions, which can provide more robust predictions when data is insufficient or abnormal. By integrating the advantages of the two, the prediction accuracy of the comprehensive state information of the target vehicle can be improved. Secondly, when fusing the two prediction models, the weights are dynamically adjusted based on the prediction uncertainty of each model, and the contribution of each model can be flexibly adjusted according to the actual situation. This adaptive adjustment capability is particularly important in a rapidly changing environment. The dynamic weight allocation method combines the data-driven deep learning prediction model and the Kalman filter based on the physical model, which reduces the dependence on high-quality data and improves the robustness of the prediction of the target vehicle's comprehensive state information, thereby obtaining better prediction results in various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a framework diagram of the vehicle trajectory fusion prediction method based on deep learning and kinematic model of the present invention;

[0049] Figure 2It is a schematic diagram of the Weight_LSTM network model in the present invention. DETAILED DESCRIPTION

[0050] The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0051] like Figure 1 The framework diagram of the vehicle trajectory fusion prediction method based on deep learning and kinematic model of the present invention, the vehicle trajectory fusion prediction method specifically includes the following steps:

[0052] Step S1, by installing a positioning module, a speed sensor, an acceleration sensor, a laser radar, a camera and a radar sensor on the target vehicle to collect scene information when the target vehicle is driving, including: state information of the target vehicle and state information of surrounding vehicles; track position information of the target vehicle in the present invention target = {x,y,v x , v y , a x , a y}, where x represents the horizontal coordinate of the target vehicle's position, y represents the vertical coordinate of the target vehicle's position, and v x 、v y Respectively represent the velocity components of the target vehicle in the horizontal and vertical directions, a x 、a y Respectively represent the acceleration components of the target vehicle in the horizontal and vertical directions; the trajectory position information of the surrounding vehicles in the present invention, neighbors ={x i ,y i , v x i , v y i , a x i , a y i}, where x i Represents the horizontal coordinate of the position of the i-th vehicle around the target vehicle, y i represents the ordinate of the position of the i-th vehicle around the target vehicle, v x i 、v y i They represent the velocity components of the i-th vehicle around the target vehicle in the horizontal and vertical directions, respectively. x i 、a y i They respectively represent the acceleration components of the i-th vehicle around the target vehicle in the horizontal and vertical directions. In the present invention, there are at most 6 surrounding vehicles, including: one in front and one behind the target vehicle lane, two in the left lane of the target vehicle, and two in the right lane of the target vehicle.

[0053] Step S2: Based on the collected scene information, use the deep learning prediction model to predict the state information of the target vehicle in the future time domain, and obtain the state information of the target vehicle at each moment in the future time domain. The deep learning prediction model has a strong nonlinear mapping ability, can fit complex functional relationships according to the scene information, can learn the high-order features and complex patterns of the scene information in the training data, has a strong generalization ability, and generates high-quality prediction results. Specifically, it includes the following sub-steps:

[0054] Step S2.1, filtering and standardizing the collected scene information to obtain standardized state information;

[0055] The filtering process in the present invention is to filter the collected scene information by wavelet decomposition method to avoid the error of collected data from having a significant impact on subsequent prediction work;

[0056] In order to eliminate the influence of data of different dimensions in the filtered scene information on the deep learning prediction model, the present invention performs standardization processing, that is, differentiates the filtered scene information in the time dimension, eliminates the trend in the scene information, and makes it stable. The specific process of standardization processing is as follows:

[0057]

[0058] Among them, I diff represents the differential scene information, I diff =Id enoise t -I denoise t-1 , I denoise t represents the filtered scene information at time t, I max ,I min Respectively represent the maximum and minimum values ​​in the differential scene information.

[0059] Step S2.2, the deep learning prediction model in the present invention includes an LSTM encoder and a Transformer decoder connected in sequence, the LSTM encoder is used to capture the time dependency in the standardized state information, can effectively learn and remember the important features in the standardized state information, and provide rich time series features for the subsequent Transformer decoder; the Transformer decoder processes data in parallel through a multi-head self-attention mechanism, can independently learn different dependencies in different subspaces, capture global information, and improve computational efficiency. The deep learning prediction model combining the LSTM encoder and the Transformer decoder can make full use of the advantages of both, and efficiently learn and predict the state information of the target vehicle.

[0060] Step S2.3: Input the standardized state information into the LSTM encoder for encoding to generate the hidden state of each time step and the hidden state of the last time step; specifically, the following sub-steps are included:

[0061] Step S2.3.1, the normalized state information is passed to the first layer of the LSTM encoder to generate a hidden state sequence for each time step;

[0062] Step S2.3.2: Use the generated hidden state sequence of each time step as the input of the next layer of the LSTM encoder to continue generating the hidden state sequence of each time step;

[0063] Step S2.3.3: Repeat step S2.3.2 until the hidden state of each time step and the hidden state of the last time step of the last layer in the LSTM encoder are obtained.

[0064] Step S2.4, repeatedly mapping the hidden state of the last time step to the input dimension of the Transformer decoder, adding position encoding to each repeated time step, obtaining the hidden state space with added position encoding, and processing the hidden state space with added position encoding and the hidden state of each time step output by the LSTM encoder in parallel through the multi-head self-attention mechanism to generate the state information of the target vehicle in the future time domain; specifically including the following sub-steps:

[0065] Step S2.4.1 repeatedly maps the hidden state of the last time step to the hidden state space through the embedding layer of the Transformer decoder, generates a position code for each time step in the hidden state space through sine and cosine functions, and obtains a hidden state space vector with added position code. The position code adds a unique code to each time step, so that the Transformer decoder can distinguish data at different time steps and naturally capture the order information of the data. At the same time, the position code uses fixed sine and cosine functions, which has good scalability and multi-scale information capture capabilities, which enables the Transformer decoder to understand that there are both local dependencies and global dependencies in the hidden state, significantly improving the performance and understanding of the deep learning prediction model. The calculation process of the hidden state space vector embeddingPE with added position code in the present invention is:

[0066] embedding PE =embedding+PE

[0067] Where, embedding represents the hidden state space vector, embedding = We h′, We represents the weight matrix of the embedding layer, h′ represents the time step matrix of the hidden state repetition of the last time step; PE represents the position encoding matrix, which is composed of the 2xth component PE(s, 2x) of the encoding vector at position k and the 2x+1th component PE(s, 2x+1) of the encoding vector at position k. d represents the dimension of position encoding,

[0068] Generate a hidden state space vector embedding with positional encoding added PE The upper triangular matrix mask matrix M i，j , ensuring that the Transformer decoder can only see the output before that time step at each time step:

[0069]

[0070] Here, i and j are time step indices. The mask matrix ensures that when calculating the output of the current time step i, only the output of the current time step and before can be seen, thereby preventing information leakage and improving the training effect of the deep learning prediction model.

[0071] Step S2.4.2, by parallel processing the hidden state space of the added position encoding and the hidden state of each time step output by the LSTM encoder through the multi-head self-attention mechanism, the output semantics mutl_head of the Transformer decoder is obtained; specifically,

[0072] Step S2.4.2.1. Calculate the query matrix Q = embedding by adding the positional encoding to the hidden state space PE W Q , calculate the key matrix K = encoder_out W through the hidden state encoder_out of each time step output by the LSTM encoder K , value matrix V = encoder_out W v , where W Q , W K , W v The weight matrices representing the linear transformation of the query matrix, key matrix, and value matrix respectively;

[0073] Step S2.4.2.2, based on the query matrix Q, key matrix K, value matrix V and upper triangular matrix mask matrix M i，j Calculate the attention score att(Q, K, V):

[0074]

[0075] Among them, d k is the dimension of the key matrix;

[0076] Step S2.4.2.3, perform multi-head attention parallel calculation according to the attention score to obtain the output semantics mutl_head of the Transformer decoder:

[0077] mutl_head=Concat(head1,...,head h )W O

[0078] in, are the weight matrices for each attention head, h is the number of attention heads, W O It is a linear transformation matrix for combination and mapping, and Concat is a concatenation function.

[0079] Step S2.4.3: The output semantics of the Transformer decoder mutl_head is passed through the fully connected layer to generate the state information of the target vehicle in the future time domain

[0080]

[0081] Among them, σ represents the activation function, W m represents the weight matrix of the fully connected layer, b m Represents the bias vector.

[0082] Step S2.5, repeating steps S2.3-S2.4 until the mean square error loss function composed of the state information of the target vehicle in the future time domain and the actual state information of the target vehicle converges, completing the training of the deep learning prediction model;

[0083] Step S2.6: After filtering and standardizing the real-time scene information of the target vehicle, the information is input into the trained deep learning prediction model to predict the state information of the target vehicle in the future time domain.

[0084] Step S3, the state information of the target vehicle at each moment in the future time domain is used to establish a target vehicle trajectory prediction model through a Kalman filter to predict the kinematic state information of the target vehicle at the next moment; the deep learning prediction model learns complex patterns and dependencies from a large amount of historical scene information to generate a preliminary predicted trajectory of the target vehicle. However, these predicted trajectories may lack physical constraints and consistency, and because the prediction process of the pure deep learning prediction model is often "black box", it is difficult to explain the prediction results. The kinematic prediction introduces the physical constraints of the target vehicle's motion through the target vehicle trajectory prediction model, making the overall prediction process more transparent and explainable, and further correcting and optimizing the prediction results of the deep learning prediction model, which is easy to understand and verify, can improve the accuracy and credibility of the overall prediction. By combining the flexibility of the deep learning prediction model and the reliability of the kinematic prediction, a more accurate and robust target vehicle trajectory prediction is achieved. Specifically, by obtaining the prior state estimate of the target vehicle state information at a certain moment in the future time domain, combined with the state information of the target vehicle at the next moment in the future time domain, a target vehicle trajectory prediction model is constructed to predict the kinematic state information of the target vehicle at the next moment

[0085]

[0086] in, represents the prior state estimate of the target vehicle state information at time k, represents the state information of the target vehicle at time k-1 in the future time domain. Since there is no state information of the target vehicle predicted by the deep learning prediction model at the initial moment, the state information of the target vehicle in the current state collected is used instead; A represents the coefficient matrix, Δt represents the interval between two adjacent moments; K [k] represents the Kalman gain value at time k, H m represents the observation constant matrix, R represents the covariance matrix of the measurement noise, represents the covariance matrix of the prior state estimate at time k, P [k-1] represents the covariance matrix of the posterior state estimate at time k-1, Q represents the covariance matrix of the process noise; z [k] Represents the state information of the target vehicle at time k in the future time domain.

[0087] Step S4: The comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment are used as inputs of the Weight_LSTM network model, such as Figure 2 , the encoder of the Weight_LSTM network model outputs the hidden state of the last layer and the hidden information of the last time step. The decoder of the Weight_LSTM network model recursively decodes the output of the encoder of the Weight_LSTM network model to predict the weight coefficient of the deep learning prediction model and the weight coefficient of the target vehicle trajectory prediction model at the next moment; the Weight_LSTM network is used to optimize the weight coefficients of the deep learning prediction model and the target vehicle trajectory prediction model. Its powerful learning ability and time-dependent processing ability can be used to fit the contribution of deep learning prediction and kinematic prediction to future trajectories, so as to more accurately predict the target vehicle trajectory. It includes the following sub-steps:

[0088] Step S4.1, taking the comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment as the input of the Weight_LSTM network model, taking the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model as the label of the Weight_LSTM network model, and predicting the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model;

[0089] The prediction uncertainty coefficient σ of the deep learning prediction model in the present invention DL The calculation process is:

[0090]

[0091] The prediction uncertainty coefficient σ of the target vehicle trajectory prediction model in the present invention is KF The calculation process is:

[0092]

[0093] Among them, P k,ii The covariance matrix P represents the posterior state estimate at time k [k] The diagonal elements of I represents the identity matrix and n represents the number of diagonal elements.

[0094] The weight coefficient W of the deep learning prediction model at the next moment in the label DL and the weight coefficient W of the target vehicle trajectory prediction model KF The calculation process is:

[0095]

[0096] Among them, inv_e DL represents the inverse error of the deep learning prediction model, e DL represents the computational error of the deep learning prediction model, e DL =||z [k] -X [k] ||,X [k] represents the true value of the target vehicle's state information at time k, ∈ is a normal number; inv_e KF represents the inverse error of the target vehicle trajectory prediction model, e KF represents the calculation error of the target vehicle trajectory prediction model,

[0097] Step S4.2, repeat step S4.1 until the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model converge with the mean square error loss function of the corresponding label, and the training of the Weight_LSTM network model is completed;

[0098] Step S4.3, input the comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment into the trained Weight_LSTM network model to predict the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model.

[0099] Step S5: The state information of the target vehicle at the next moment predicted by step S2 and the kinematic state information of the target vehicle at the next moment predicted by step S3 are calculated by the weight coefficient of the deep learning prediction model at the next moment. and the weight coefficient of the target vehicle trajectory prediction model Weighted, predict the comprehensive state information X of the target vehicle at the next moment k :

[0100]

[0101] The present invention is based on the vehicle trajectory fusion prediction method of deep learning and kinematic model. The motion trajectory of the target vehicle usually depends on historical data within a long time range. The LSTM encoder can effectively capture and memorize the dependencies in the long time series; the Transformer decoder can process the output of the LSTM encoder in parallel through the multi-head self-attention mechanism to capture the global dependencies of the target vehicle. Combining the LSTM encoder with the Transformer decoder can take advantage of the advantages of both to achieve accurate prediction of the target vehicle trajectory. In addition, the kinematic target vehicle trajectory prediction model introduced in the present invention ensures that the target vehicle prediction result is physically reasonable and consistent through physical constraints, corrects the target vehicle state predicted by the deep learning prediction model, and improves the accuracy and credibility of the prediction. In addition, since the deep learning prediction model uses rich training data to train the network, it can adaptively optimize the model parameters, filter out noise data to a certain extent, and improve the adaptability to different types and complexity data. The kinematic target vehicle trajectory prediction model can provide correction for the deep learning prediction method when the data is abnormal or extreme, ensuring the robustness of the target vehicle trajectory prediction.

[0102] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0103] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0104] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A vehicle trajectory fusion prediction method based on deep learning and kinematic model, characterized in that: The specific steps include: Step S1, collecting scene information when the target vehicle is driving, the scene information includes: state information of the target vehicle and state information of surrounding vehicles; Step S2: predict the state information of the target vehicle in the future time domain using a deep learning prediction model based on the collected scene information, and obtain the state information of the target vehicle at each moment in the future time domain; Step S3, using the state information of the target vehicle at each moment in the future time domain to establish a target vehicle trajectory prediction model through a Kalman filter, and predicting the kinematic state information of the target vehicle at the next moment; Step S4, taking the comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment as the input of the Weight_LSTM network model, predicting the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model; comprising the following sub-steps: Step S4.1, taking the comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment as the input of the Weight_LSTM network model, taking the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model as the label of the Weight_LSTM network model, and predicting the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model; The prediction uncertainty coefficient σ of the deep learning prediction model DL The calculation process is: The prediction uncertainty coefficient σ of the target vehicle trajectory prediction model KF The calculation process is: Among them, P k,ii The covariance matrix P represents the posterior state estimate at time k [k] The diagonal elements of I represents the identity matrix, n represents the number of diagonal elements; The weight coefficient W of the deep learning prediction model at the next moment in the label DL and the weight coefficient W of the target vehicle trajectory prediction model KF The calculation process is: Among them, inv_e DL represents the inverse error of the deep learning prediction model, e DL represents the computational error of the deep learning prediction model, e DL =||z [k] -X [k] ||,X [k] represents the true value of the target vehicle's state information at time k, ∈ is a normal number; inv_e KF represents the inverse error of the target vehicle trajectory prediction model, e KF represents the calculation error of the target vehicle trajectory prediction model, Step S4.2, repeat step S4.1 until the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model converge with the mean square error loss function of the corresponding label, and the training of the Weight_LSTM network model is completed; Step S4.3, input the comprehensive state information of the target vehicle at the current moment, the state information of the target vehicle at the next moment predicted by step S2, the kinematic state information of the target vehicle at the next moment predicted by step S3, the prediction uncertainty coefficient of the deep learning prediction model, the prediction uncertainty coefficient of the target vehicle trajectory prediction model, the weight coefficient of the deep learning prediction model at the current moment, and the weight coefficient of the target vehicle trajectory prediction model at the current moment into the trained Weight_LSTM network model, and predict the weight coefficient of the deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model; Step S5: weight the state information of the target vehicle at the next moment predicted by step S2 and the kinematic state information of the target vehicle at the next moment predicted by step S3 by using the weight coefficient of the predicted deep learning prediction model at the next moment and the weight coefficient of the target vehicle trajectory prediction model to predict the comprehensive state information of the target vehicle at the next moment.

2. The vehicle trajectory fusion prediction method based on deep learning and kinematic model according to claim 1 is characterized in that: Step 2 includes the following sub-steps: Step S2.1, filtering and standardizing the collected scene information to obtain standardized state information; Step S2.2, the deep learning prediction model includes an LSTM encoder and a Transformer decoder connected in sequence; Step S2.3, input the standardized state information into the LSTM encoder for encoding to generate the hidden state of each time step and the hidden state of the last time step; Step S2.4, repeatedly map the hidden state of the last time step to the input dimension of the Transformer decoder, add position encoding for each repeated time step, obtain the hidden state space with position encoding, and process the hidden state space with position encoding and the hidden state of each time step output by the LSTM encoder in parallel through the multi-head self-attention mechanism to generate the state information of the target vehicle in the future time domain; Step S2.5, repeating steps S2.3-S2.4 until the mean square error loss function composed of the state information of the target vehicle in the future time domain and the actual state information of the target vehicle converges, completing the training of the deep learning prediction model; Step S2.6: After filtering and standardizing the real-time scene information of the target vehicle, the information is input into the trained deep learning prediction model to predict the state information of the target vehicle in the future time domain.

3. The vehicle trajectory fusion prediction method based on deep learning and kinematic model according to claim 2 is characterized in that: The specific process of standardization in step S2.1 is as follows: Among them, I diff represents the differential scene information, I diff =I denoise t -I denoise t-1 , I denoise t represents the filtered scene information at time t, I max ,I min Represent the maximum and minimum values ​​in the differential scene information respectively.

4. The vehicle trajectory fusion prediction method based on deep learning and kinematic model according to claim 2 is characterized in that: Step S2.3 includes the following sub-steps: Step S2.3.1, the normalized state information is passed to the first layer of the LSTM encoder to generate a hidden state sequence for each time step; Step S2.3.2: Use the generated hidden state sequence of each time step as the input of the next layer of the LSTM encoder to continue generating the hidden state sequence of each time step; Step S2.3.3: Repeat step S2.3.2 until the hidden state of each time step and the hidden state of the last time step of the last layer in the LSTM encoder are obtained.

5. The vehicle trajectory fusion prediction method based on deep learning and kinematic model according to claim 2 is characterized in that: Step S2.4 includes the following sub-steps: Step S2.4.1 repeatedly maps the hidden state of the last time step to the hidden state space through the embedding layer of the Transformer decoder, generates a position code for each time step in the hidden state space through sine and cosine functions, and obtains a hidden state space vector with added position code; Step S2.4.2, by using a multi-head self-attention mechanism, the hidden state space of the position encoding and the hidden state of each time step output by the LSTM encoder are processed in parallel to obtain the output semantics of the Transformer decoder; Step S2.4.3: The output semantics of the Transformer decoder is passed through the fully connected layer to generate the state information of the target vehicle in the future time domain.

6. The vehicle trajectory fusion prediction method based on deep learning and kinematic model according to claim 5 is characterized in that: Add the hidden state space vector embedding of the position encoding in step S2.4.1 PE The calculation process is: embedding PE =embedding+PE Where, embedding represents the hidden state space vector, embedding = We h′, We represents the weight matrix of the embedding layer, h′ represents the time step matrix of the hidden state repetition of the last time step; PE represents the position encoding matrix, which is composed of the 2xth component PE(s, 2x) of the encoding vector at position k and the 2x+1th component PE(s, 2x+1) of the encoding vector at position k. d represents the dimension of position encoding, 7. The vehicle trajectory fusion prediction method based on deep learning and kinematic model according to claim 2 is characterized in that: The specific process of step S3 is as follows: by obtaining the prior state estimation value of the target vehicle state information at the next moment in the future time domain through the state information of the target vehicle at a certain moment in the future time domain, and combining the state information of the target vehicle at the next moment in the future time domain, constructing the target vehicle trajectory prediction model, and predicting the kinematic state information of the target vehicle at the next moment in, represents the prior state estimate of the target vehicle state information at time k, represents the state information of the target vehicle at time k-1 in the future time domain, A represents the coefficient matrix; K [k] represents the Kalman gain value at time k, H m represents the observation constant matrix, R represents the covariance matrix of the measurement noise, represents the covariance matrix of the prior state estimate at time k, P [k-1] represents the covariance matrix of the posterior state estimate at time k-1, Q represents the covariance matrix of the process noise; z [k] Represents the state information of the target vehicle at time k in the future time domain.

Images