Vehicle trajectory prediction method, system, computer device and storage medium

Abstract

The present invention belongs to the field of autonomous driving, and discloses a vehicle trajectory prediction method, system, computer equipment and storage medium, including: extracting historical motion trajectory data of a to-be-predicted vehicle and each of its adjacent vehicles to be predicted, and obtaining a multi-dimensional dynamic scene feature vector of the to-be-predicted vehicle; extracting the multi-dimensional dynamic scene feature vector of the to-be-predicted vehicle, and obtaining traffic perception information of the to-be-predicted vehicle; encoding the traffic perception information and historical motion state data of the to-be-predicted vehicle, and obtaining hidden state information of the to-be-predicted vehicle; obtaining a mixed attention matrix of the to-be-predicted vehicle according to the hidden state information of the to-be-predicted vehicle, and assigning weights to the hidden state information of the to-be-predicted vehicle through the mixed attention matrix of the to-be-predicted vehicle, and then obtaining a trajectory prediction value of the to-be-predicted vehicle through maximum pooling processing and full connection processing in sequence, thereby effectively improving the accuracy of vehicle trajectory prediction.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving and relates to a vehicle trajectory prediction method, system, computer equipment and storage medium. Background Art

[0002] Autonomous vehicles and connected cars will play a key role in the development of future road traffic and transportation. In order to travel quickly and safely in complex traffic environments, connected cars must use path planning algorithms and driving strategies to decide when to accelerate and change lanes based on the future trajectories of surrounding vehicles. However, due to the influence of the driver's subjective driving intentions and the objective dynamic interactions between vehicles, vehicle trajectories are usually highly nonlinear, so predicting the future trajectory of vehicles is a very challenging problem.

[0003] At present, when performing trajectory prediction, only the interaction between vehicles is considered, while the influence of the driver's subjective driving intention on the future trajectory of the vehicle is ignored, resulting in low trajectory prediction accuracy, especially when the vehicle is maneuvering laterally. In order to solve this problem, trajectory prediction methods based on driving strategy classification have become a research direction. This type of method first predicts the future driving strategy of the vehicle, such as going straight, changing lanes left and right, and then predicts the micro-trajectory based on the driving strategy. Although this type of method solves the prediction problem of some lane change trajectories, it also makes the trajectory prediction accuracy completely dependent on the prediction accuracy of the driving strategy, so that when the model predicts the driving strategy incorrectly, the trajectory prediction accuracy drops significantly. In addition, the long-term prediction ability of the LSTM-based model is poor, and its prediction error increases sharply with the increase of the prediction time. For the Transformer model with strong long-term prediction ability, the model parameters and calculation amount are huge, making the model too complicated. Summary of the invention

[0004] The purpose of the present invention is to overcome the disadvantage of low vehicle trajectory prediction accuracy in the above-mentioned prior art, and to provide a vehicle trajectory prediction method, system, computer device and storage medium.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect of the present invention, a vehicle trajectory prediction method comprises:

[0007] Obtain historical motion trajectory data of the vehicle to be predicted and each of the adjacent vehicles of the vehicle to be predicted;

[0008] The historical motion trajectory data of the vehicle to be predicted and its adjacent vehicles are extracted by a preset multi-dimensional dynamic scene feature extraction function to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted;

[0009] The multi-dimensional dynamic scene feature vector of the vehicle to be predicted is extracted through a preset information extraction neural network to obtain traffic perception information of the vehicle to be predicted;

[0010] The traffic perception information and historical motion state data of the vehicle to be predicted are encoded through a preset time feature encoder to obtain the hidden state information of the vehicle to be predicted;

[0011] According to the hidden state information of the vehicle to be predicted, the hybrid attention matrix of the vehicle to be predicted is obtained, and weights are assigned to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted. Then, the trajectory prediction value of the vehicle to be predicted is obtained through maximum pooling processing and full connection processing in sequence.

[0012] Optionally, the adjacent vehicles of the vehicle to be predicted include adjacent vehicles in eight directions: front, rear, left, right, left front, left rear, right front and right rear of the vehicle to be predicted.

[0013] Optionally, the extracting of historical motion trajectory data of the vehicle to be predicted and each of the adjacent vehicles of the vehicle to be predicted by a preset multi-dimensional dynamic scene feature extraction function to obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted includes:

[0014] Through the preset multi-dimensional dynamic scene feature extraction function, the historical motion trajectory data of the vehicle to be predicted and its adjacent vehicles are extracted, and the position data, speed data and acceleration data of the vehicle to be predicted and its adjacent vehicles in the driving direction and the direction perpendicular to the driving direction are obtained to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted.

[0015] Optionally, the information extraction neural network includes a first convolutional layer, a second convolutional layer, a maximum pooling layer and a fully connected layer connected in sequence;

[0016] The first convolutional layer is used to fuse the position data, speed data and acceleration data of the vehicle to be predicted and its adjacent vehicles in the driving direction and the vertical direction of the driving direction to obtain the first fusion features of the vehicle to be predicted and its adjacent vehicles;

[0017] The second convolutional layer is used to fuse the first fused features of the vehicle to be predicted and any adjacent vehicles of the adjacent vehicles of the vehicle to be predicted to obtain the second fused features;

[0018] The maximum pooling layer is used for maximum pooling processing of the second fusion feature;

[0019] The fully connected layer is used to fully connect the second fusion features after the maximum pooling process to obtain the traffic perception information of the vehicle to be predicted.

[0020] Optionally, the preset time feature encoder is a long short-term memory network encoder.

[0021] Optionally, obtaining a mixed attention matrix of the vehicle to be predicted according to the hidden state information of the vehicle to be predicted includes:

[0022] The mixed attention matrix α of the vehicle to be predicted is obtained by the following formula:

[0023] α t =softmax(g t (H,h t+1 ))

[0024] α f =softmax(g f (H,h t+1 ))

[0025] α=α t α f

[0026] Among them, softmax is a normalized exponential function, α t is the time weight vector, α f is the feature weight vector, g t (H,h t+1 )=Hh T , g f (H,h t+1 )=h t+1 (W f H), g f is the feature weight cosine correlation function, g t is the time-weighted cosine correlation function, W f For history h The feature matrix of the vehicle to be predicted in the frame, H is the history T h The hidden state information of the vehicle to be predicted in the frame, h t is the hidden state data of the vehicle to be predicted at time t, h t+1 It is the hidden state data of the vehicle to be predicted at time t+1.

[0027] Optionally, the step of assigning weights to the hidden state information of the vehicle to be predicted by using the hybrid attention matrix of the vehicle to be predicted, and then sequentially performing maximum pooling processing and full connection processing to obtain the trajectory prediction value of the vehicle to be predicted includes:

[0028] The predicted trajectory value y of the vehicle to be predicted is obtained by the following formula: t+1 :

[0029] O=α⊙H

[0030]

[0031]

[0032] h' t+1 =contact(h t+1 ,o t ,o f )

[0033] y t+1 =h' t+1 W 2 W 1

[0034] Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o t To improve the prediction accuracy, To perform maximum pooling in the time dimension, O i,j is the hidden state information of the vehicle to be predicted after assigning weights, o f is the most beneficial feature for improving prediction accuracy. To perform maximum pooling in the feature dimension, h t ' +1 For o t , o f and h t+1 The hidden state information of the vehicle to be predicted at time t+1 is obtained by performing full connection processing, contact is fully connected, W 1 and W 2 The preset weights.

[0035] A second aspect of the present invention is a vehicle trajectory prediction system, comprising:

[0036] A data acquisition module is used to acquire historical motion trajectory data of the vehicle to be predicted and the vehicles adjacent to the vehicle to be predicted;

[0037] A data preprocessing module is used to extract the historical motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted;

[0038] An information extraction module is used to extract a multi-dimensional dynamic scene feature vector of the vehicle to be predicted through a preset information extraction neural network to obtain traffic perception information of the vehicle to be predicted;

[0039] The encoding module is used to encode the traffic perception information and historical motion state data of the vehicle to be predicted through a preset time feature encoder to obtain the hidden state information of the vehicle to be predicted;

[0040] The prediction module is used to obtain the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted, and assign weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then obtain the trajectory prediction value of the vehicle to be predicted through maximum pooling processing and full connection processing in sequence.

[0041] In a third aspect of the present invention, a computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned vehicle trajectory prediction steps when executing the computer program.

[0042] A fourth aspect of the present invention is a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned vehicle trajectory prediction steps are implemented.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The vehicle trajectory prediction method of the present invention utilizes the historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted, performs feature extraction through a preset multi-dimensional dynamic scene feature extraction function, obtains a multi-dimensional dynamic scene feature vector of the vehicle to be predicted, and based on the multi-dimensional dynamic scene feature vector of the vehicle to be predicted, obtains the traffic perception information of the vehicle to be predicted through an information extraction neural network, thereby characterizing the dynamic dependency relationship between the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted and the subjective driving intention of the driver, then encodes the traffic perception information of the vehicle to be predicted and the historical motion state data through a time feature encoder to obtain the hidden state information of the vehicle to be predicted, and then obtains the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted, and finally assigns weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, selectively reuses the historical trajectory information through the hybrid attention mechanism, improves the long-term prediction ability of the model, and then obtains the trajectory prediction value of the vehicle to be predicted through maximum pooling processing and full connection processing in sequence, so that the vehicle trajectory prediction is more reasonable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart of a vehicle trajectory prediction method according to an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a traffic scene and a static coordinate system according to an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of constructing a multi-dimensional dynamic scene feature map according to an embodiment of the present invention;

[0048] Figure 4A schematic diagram of the information extraction neural network structure of an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the principle of the hybrid attention mechanism according to an embodiment of the present invention;

[0050] Figure 6 A schematic diagram of the prediction principle based on the hybrid attention mechanism according to an embodiment of the present invention;

[0051] Figure 7 A schematic diagram of the principle of a vehicle trajectory prediction method according to an embodiment of the present invention;

[0052] Figure 8 is a data preprocessing flow chart of an embodiment of the present invention;

[0053] Fig. 9 Schematic diagram of data distribution before normalization according to an embodiment of the present invention;

[0054] Fig.10 This is a schematic diagram of normalized data distribution according to an embodiment of the present invention;

[0055] Fig.11 A schematic diagram of the relationship between the initial learning rate and the number of iterations in an embodiment of the present invention;

[0056] Fig.12 A trajectory prediction diagram of the vehicle trajectory prediction method of the present invention according to an embodiment of the present invention;

[0057] Fig.13 It is a trajectory prediction diagram of the existing transformer model in an embodiment of the present invention;

[0058] Fig.14 This is a schematic diagram of the relationship between the RMSE of a sample trajectory and the number of predicted frames according to an embodiment of the present invention;

[0059] Fig.15 Schematic diagram of the relationship between MAEx and the predicted number of frames for a sample trajectory according to an embodiment of the present invention;

[0060] Fig.16 Schematic diagram of the relationship between MAEy and the number of predicted frames for a sample trajectory according to an embodiment of the present invention;

[0061] Fig.17 A schematic diagram of the relationship between RMSE and the number of predicted frames of another sample trajectory according to an embodiment of the present invention;

[0062] Fig.18 A schematic diagram of the relationship between MAEx and the predicted frame number of another sample trajectory according to an embodiment of the present invention;

[0063] Fig.19 FIG. 4 is a schematic diagram showing the relationship between MAEy and the predicted frame number of another sample trajectory according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0065] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0066] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0067] See also Figure 1 In one embodiment of the present invention, a vehicle trajectory prediction method is provided to solve the problem that the existing method does not consider the subjective driving intention of the driver, resulting in low prediction accuracy and poor long-term prediction ability. Specifically, the vehicle trajectory prediction method includes the following steps:

[0068] S1: Obtain historical motion trajectory data of the vehicle to be predicted and its adjacent vehicles.

[0069] S2: Through a preset multi-dimensional dynamic scene feature extraction function, the historical motion trajectory data of the vehicle to be predicted and the vehicles adjacent to the vehicle to be predicted are extracted to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted.

[0070] S3: extracting the multi-dimensional dynamic scene feature vector of the vehicle to be predicted through a preset information extraction neural network to obtain traffic perception information of the vehicle to be predicted.

[0071] S4: Encode the traffic perception information and historical motion state data of the vehicle to be predicted through a preset time feature encoder to obtain the hidden state information of the vehicle to be predicted.

[0072] S5: According to the hidden state information of the vehicle to be predicted, a hybrid attention matrix of the vehicle to be predicted is obtained, and weights are assigned to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then the trajectory prediction value of the vehicle to be predicted is obtained through maximum pooling processing and full connection processing in sequence.

[0073] Specifically, the vehicle trajectory prediction method of the present invention utilizes the historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted, performs feature extraction through a preset multi-dimensional dynamic scene feature extraction function, obtains a multi-dimensional dynamic scene feature vector of the vehicle to be predicted, and based on the multi-dimensional dynamic scene feature vector of the vehicle to be predicted, obtains the traffic perception information of the vehicle to be predicted through an information extraction neural network, thereby characterizing the dynamic dependency relationship between the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted and the subjective driving intention of the driver, and then encodes the traffic perception information of the vehicle to be predicted and the historical motion state data through a time feature encoder to obtain the hidden state information of the vehicle to be predicted, and then obtains the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted, and finally assigns weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, selectively reuses the historical trajectory information through the hybrid attention mechanism, improves the long-term prediction ability of the model, and then obtains the trajectory prediction value of the vehicle to be predicted through maximum pooling processing and full connection processing in sequence, so that the vehicle trajectory prediction is more reasonable and accurate.

[0074] First, see Figure 2 , when the vehicle trajectory prediction method of the present invention is implemented, a feasible traffic scene and static coordinate system are provided. The traffic scene is a two-way eight-lane road, and a fixed reference system is used to determine the position of each vehicle. The starting points of the x-axis and the y-axis are both the upper left corner of the road, the x-axis is parallel to the direction of vehicle movement on the highway, and the y-axis is perpendicular to the direction of travel. Along the y-axis, there are 1-8 lanes. Among them, vehicles in lanes 1-4 travel along the positive direction of the x-axis, and vehicles in lanes 5-8 travel along the negative direction of the x-axis. The following embodiments are schematically illustrated using traffic scenes as examples, but are not limited thereto.

[0075] In a possible implementation manner, the neighboring vehicles of the to-be-predicted vehicle include neighboring vehicles in eight directions: front, rear, left, right, left front, left rear, right front, and right rear of the to-be-predicted vehicle.

[0076] Optionally, the extracting of historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted includes: extracting the position data, speed data and acceleration data of the vehicle to be predicted and each adjacent vehicle to be predicted in the driving direction and in the direction perpendicular to the driving direction from the historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted.

[0077] For details, see Figure 3 , the multi-dimensional dynamic scene feature vector of the vehicle to be predicted is extracted through a preset information extraction neural network, and the specific extraction process can be completed by constructing a multi-dimensional dynamic scene feature map. Among them, the multi-dimensional dynamic scene feature map is centered on the vehicle to be predicted, and the vehicles in the eight directions around it, that is, the front, rear, left, right, left front, left rear, right front and right rear of the predicted vehicle, construct a 3*3 rectangular area and divide it into 9 units. Adjacent vehicles are mapped to the corresponding units according to their relative positions. According to this rule, the position layer, velocity layer and acceleration layer in the X and Y directions are constructed respectively, forming a total of 6 rectangles of 3*3 dimensions, and finally superimposed to form a multi-dimensional dynamic scene feature map of 6*3*3 dimensions. Assume that the definition function of the feature map is Then the multi-dimensional dynamic scene characteristics at time t can be expressed as:

[0078]

[0079] Among them, x t Represents the motion trajectory data of the vehicle to be predicted and its adjacent vehicles at time t.

[0080] In a possible implementation, the information extraction neural network includes a first convolutional layer, a second convolutional layer, a maximum pooling layer, and a fully connected layer that are connected in sequence.

[0081] Among them, the first convolution layer is used to fuse the position data, speed data and acceleration data of the vehicle to be predicted and its adjacent vehicles in the driving direction and the perpendicular direction of the driving direction, and obtain the first fusion feature of the vehicle to be predicted and its adjacent vehicles; the second convolution layer is used to fuse the first fusion feature of the vehicle to be predicted and any adjacent vehicles of the adjacent vehicles to be predicted, and obtain the second fusion feature; the maximum pooling layer is used to process the second fusion feature by maximum pooling; the fully connected layer is used to fully connect the second fusion feature after maximum pooling processing, and obtain the traffic perception information of the vehicle to be predicted.

[0082] For details, see Figure 4For the 6*3*3 multi-dimensional dynamic scene volume, the 6-dimensional features of each vehicle are fused using the first 6*1*1 convolution layer to capture the intention of the driver of each vehicle. Then, each vehicle interacts with the surrounding vehicles through the second 2*2 convolution layer to extract the interaction between vehicles. Finally, the traffic force constraint information p that reflects the driver's driving intention and the interaction between vehicles is obtained through the maximum pooling layer Maxpool and the fully connected layer FC. The above process is defined as a convolution operation with the mapping function traf. The multi-dimensional dynamic scene feature vector F at time t is obtained through the traf function. t Extract traffic perception information p t :

[0083] p t = traf(F t ).

[0084] In a possible implementation, the preset time feature encoder is a long short-term memory network (LSTM) encoder. Specifically, when the traffic perception information and historical motion state data of the vehicle to be predicted are encoded by the preset time feature encoder to obtain the hidden state information of the vehicle to be predicted, the traffic perception information p t The historical trajectory data of the vehicle to be predicted Splicing, as the input of the LSTM encoder, implements the encoding of the historical state through the following formula:

[0085]

[0086] f t =σ(W fz z t +W fh h t-1 +b f )

[0087] i t =σ(W iz z t +W ih h t-1 +b i )

[0088] o t =σ(W oz z t +W oh h t-1 +b o )

[0089]

[0090]

[0091] Among them, the forget gate f in the LSTM encoder t Decide which information to discard, input gate i t Controls which new information is updated and stored, the output gate o t Then control the output of the candidate layer, the value is the candidate unit, the cell state c of the LSTM encoder t is the cell state c at the previous moment t-1 and the current candidate status The final output gate of the LSTM encoder is o t Will determine which cell states are output.

[0092] Then the hidden state information of the vehicle to be predicted is obtained by the following formula:

[0093] h t =o t ⊙tanh(c t )

[0094] Among them, σ is the sigmoid function, ⊙ is the multiplication of the corresponding elements of the matrix, and x t and h t are the input vector and hidden layer state at time t respectively. f, i and o are the forget gate, input gate and output gate respectively, W and b are model parameters, W fz Input weight matrix for forget gate, W fh is the weight matrix of the hidden layer of the forget gate, W iz Input weight matrix for the input gate, W ih is the input gate hidden layer weight matrix, W oz Input weight matrix for output gate, W oh is the output gate hidden layer weight matrix, W cz Input weight matrix for cell state, W ch is the cell state hidden layer weight matrix, b f is the forget gate bias, b i is the input gate bias, b o is the output gate bias, b c Bias for cell state.

[0095] In a possible implementation, obtaining the hybrid attention matrix of the vehicle to be predicted according to the hidden state information of the vehicle to be predicted includes: obtaining the hybrid attention matrix α of the vehicle to be predicted by the following formula:

[0096] α t =softmax(g t (H,h t+1 ))

[0097] α f =softmax(g f(H,h t+1 ))

[0098] α=α t α f

[0099] Among them, softmax is a normalized exponential function, α t is the time weight vector, α f is the feature weight vector, g t (H,h t+1 )=Hh T , g f (H,h t+1 )=h t+1 (W f H), g f is the feature weight cosine correlation function, g t is the time-weighted cosine correlation function, W f For history h The feature matrix of the vehicle to be predicted in the frame, H is the history T h The hidden state information of the vehicle to be predicted in the frame, h t is the hidden state data of the vehicle to be predicted at time t, h t+1 It is the hidden state data of the vehicle to be predicted at time t+1.

[0100] For details, see Figure 5 The vehicle trajectory prediction method of the present invention combines time attention and feature attention. The hybrid attention mechanism can comprehensively consider the moments and features that have a great impact on the output accuracy, and independently assign attention weights to each feature at each moment. H is the history T h The hidden state information of the vehicle to be predicted in the frame is the hidden state of the LSTM encoder. If the hidden state of the LSTM encoder is set to n dimensions, then Assume that the hidden state at time t+1 is Then the time weighted cosine correlation function g t The cosine correlation function g with the feature weight f To calculate H and h t+1 Correlation in time and feature dimensions.

[0101] Then the cosine correlation is converted into a time weight vector through the Softmax function With the feature weight vector And finally get the mixed attention matrix through vector multiplication

[0102] In a possible implementation, the step of assigning weights to the hidden state information of the vehicle to be predicted by using the hybrid attention matrix of the vehicle to be predicted, and then sequentially performing maximum pooling processing and full connection processing to obtain the trajectory prediction value of the vehicle to be predicted includes: obtaining the trajectory prediction value y of the vehicle to be predicted by the following formula: t+1 :

[0103] O=α⊙H

[0104]

[0105]

[0106] h' t+1 =contact(h t+1 ,o t ,o f )

[0107] y t+1 =h' t+1 W 2 W 1

[0108] Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o t The most favorable time to improve the prediction accuracy is To perform maximum pooling in the time dimension, O i,j is the hidden state information of the vehicle to be predicted after assigning weights, o f is the most beneficial feature for improving prediction accuracy. To perform maximum pooling in the feature dimension, h t ' +1 For o t , o f and h t+1 The hidden state information of the vehicle to be predicted at time t+1 is obtained by performing full connection processing, contact is fully connected, W 1 and W 2 They are the preset weights for the two fully connected layers respectively.

[0109] For details, see Figure 6 , first use the mixed attention matrix α to assign weights to H; then perform maximum pooling of the time dimension and feature dimension to obtain and Finally, h t+1 , o t and f Connect and obtain the trajectory prediction value y at time t+1 through the fully connected layer t+1 .

[0110] See also Figure 7,The implementation principle of the vehicle trajectory prediction method of the present invention integrates the traffic force-aware LSTM encoder and the LSTM decoder based on the hybrid attention mechanism, x pre is the historical motion trajectory data of the vehicle to be predicted, z t and h t They represent the vector concatenation result and hidden state of LSTM at time t respectively. Finally, the LSTM decoder gives the prediction result y at time t+1 t+1 .

[0111] Optionally, in the vehicle trajectory prediction method of the present invention, the preset multi-dimensional dynamic scene feature extraction function, the preset information extraction neural network, the preset time feature encoder, and the network layer for maximum pooling processing and full connection processing are all pre-trained to determine their specific parameters. In the pre-training process, the vehicle's motion trajectory data (position data, speed data, and acceleration data in the x and y directions of the vehicle) for the past 50 frames are used to predict the trajectory of the next 50 frames, so a total of 100 frames of complete data are required, and vehicle data that appear for less than 100 frames are deleted through data cleaning.

[0112] In order to verify the effectiveness of the vehicle trajectory prediction method of the present invention, the German high-speed public dataset highD is used for testing. 110,660 trajectories are selected for model training and 50,787 trajectories are used for model testing.

[0113] The data preprocessing process before the experiment is as follows Figure 8 As shown, Fig. 9 , Fig.10 That is, the data distribution before and after normalization in data preprocessing. Fig.11 Different initial learning rates are used in different training stages during model training. Fig.12 and Fig.13 The prediction results of 50 consecutive frames of the vehicle trajectory prediction method of the present invention and the existing Transformer model are shown in Figure 2. Figures 14 to 19 This is a comparison between the vehicle trajectory prediction method of the present invention and the existing Transformer model in terms of various evaluation indicators. The specific description is as follows:

[0114] See also Fig.11 , is the initial learning rate used by the model in different training stages. This method can reduce fluctuations when the model performance is close to the optimal point and reach the optimal point at a faster speed.

[0115] See also Fig.12 and 13 , which is a comparison chart of the vehicle trajectory prediction method of the present invention and the trajectory prediction of the existing Transformer model. Fig.12It can be seen from the figure that the prediction results of this method are more accurate when the vehicle makes lateral maneuvers, while the Transformer model tends to offset the left and right maneuvers in the lateral direction. Fig.13 It can be seen that as the prediction length increases, the predicted points of the Transformer model are increasingly different from the actual trajectory points, while the vehicle trajectory prediction method of the present invention can still make relatively accurate predictions. Among them, hist represents the historical trajectory coordinates, gt represents the future actual trajectory coordinates, ours represents the vehicle trajectory prediction method of the present invention, tf represents the Transformer model, the horizontal coordinate is the vehicle x-direction coordinate, and the vertical coordinate is the vehicle y-direction coordinate.

[0116] See also Figures 14 to 19 , shows the changes of evaluation indicators RMSE, MAEx and MAEy with frame length, where RMSE, MAEx and MAEy represent the root mean square error, the mean absolute error in the x direction and the mean absolute error in the y direction respectively, and the horizontal axis frame represents the time frame. Fig.14 , Fig.15 and Fig.16 They are the changes of RMSE, MAEx and MAEy of a sample trajectory with frame length. Fig.17 , Fig.18 and Fig.19 They are the changes of RMSE, MAEx and MAEy of another sample trajectory with frame length. It can be seen that with the increase of prediction frame length, the RMSE of the Transformer model grows almost exponentially, MAEx and MAEy also grow linearly, while the RMSE of the vehicle trajectory prediction method of the present invention can fluctuate within a relatively low level, which shows that the long-term prediction ability of the vehicle trajectory prediction method of the present invention is strong; at the same time, the MAEx of the vehicle trajectory prediction method of the present invention is smaller than that of the Transformer, which also shows that the prediction result of the vehicle trajectory prediction method of the present invention is more accurate when the vehicle is moving sideways.

[0117] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0118] In yet another embodiment of the present invention, a vehicle trajectory prediction system is provided, which can be used to implement the above-mentioned vehicle trajectory prediction method. Specifically, the vehicle trajectory prediction system includes a data acquisition module, a data preprocessing module, an information extraction module, a coding module and a prediction module.

[0119] Among them, the data acquisition module is used to obtain the historical motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted; the data preprocessing module is used to extract the historical motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function, and obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted; the information extraction module is used to extract the multi-dimensional dynamic scene feature vector of the vehicle to be predicted through a preset information extraction neural network, and obtain the traffic perception information of the vehicle to be predicted; the encoding module is used to encode the traffic perception information and historical motion state data of the vehicle to be predicted through a preset time feature encoder, and obtain the hidden state information of the vehicle to be predicted; the prediction module is used to obtain the mixed attention matrix of the vehicle to be predicted according to the hidden state information of the vehicle to be predicted, and assign weights to the hidden state information of the vehicle to be predicted through the mixed attention matrix of the vehicle to be predicted, and then obtain the trajectory prediction value of the vehicle to be predicted through maximum pooling processing and full connection processing in sequence.

[0120] In a possible implementation manner, the neighboring vehicles of the to-be-predicted vehicle include neighboring vehicles in eight directions: front, rear, left, right, left front, left rear, right front, and right rear of the to-be-predicted vehicle.

[0121] In a possible implementation, the method of extracting historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted includes: extracting position data, speed data and acceleration data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted in the driving direction and in a direction perpendicular to the driving direction from the historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted.

[0122] In a possible implementation, the information extraction neural network includes a first convolutional layer, a second convolutional layer, a maximum pooling layer and a fully connected layer connected in sequence; wherein the first convolutional layer is used to fuse the position data, speed data and acceleration data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted in the driving direction and the direction perpendicular to the driving direction, so as to obtain the first fused features of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted; the second convolutional layer is used to fuse the first fused features of any adjacent vehicle of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted, so as to obtain the second fused features; the maximum pooling layer is used to process the second fused features by maximum pooling; the fully connected layer is used to fully process the second fused features after the maximum pooling processing, so as to obtain the traffic perception information of the vehicle to be predicted.

[0123] In a possible implementation, the preset time feature encoder is a long short-term memory network encoder.

[0124] In a possible implementation, obtaining the hybrid attention matrix of the vehicle to be predicted according to the hidden state information of the vehicle to be predicted includes: obtaining the hybrid attention matrix α of the vehicle to be predicted by the following formula:

[0125] α t =softmax(g t (H,h t+1 ))

[0126] α f =softmax(g f (H,h t+1 ))

[0127] α=α t α f

[0128] Among them, softmax is a normalized exponential function, α t is the time weight vector, α f is the feature weight vector, g t (H,h t+1 )=Hh T , g f (H,h t+1 )=h t+1 (W f H), g f is the feature weight cosine correlation function, g t is the time-weighted cosine correlation function, W f For history h The feature matrix of the vehicle to be predicted in the frame, H is the history T h The hidden state information of the vehicle to be predicted in the frame, h t is the hidden state data of the vehicle to be predicted at time t, h t+1 It is the hidden state data of the vehicle to be predicted at time t+1.

[0129] In a possible implementation, the step of assigning weights to the hidden state information of the vehicle to be predicted by using the hybrid attention matrix of the vehicle to be predicted, and then sequentially performing maximum pooling processing and full connection processing to obtain the trajectory prediction value of the vehicle to be predicted includes: obtaining the trajectory prediction value y of the vehicle to be predicted by the following formula: t+1 :

[0130] O=α⊙H

[0131]

[0132]

[0133] h t ' +1=contact(h t+1 ,o t ,o f )

[0134] y t+1 =h t ' +1 W 2 W 1

[0135] Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o t The most favorable time to improve the prediction accuracy is To perform maximum pooling in the time dimension, O i,j is the hidden state information of the vehicle to be predicted after assigning weights, o f is the most beneficial feature for improving prediction accuracy. To perform maximum pooling in the feature dimension, h' t+1 For o t , o f and h t+1 The hidden state information of the vehicle to be predicted at time t+1 is obtained by performing full connection processing, contact is fully connected, W 1 and W 2 The preset weights.

[0136] All relevant contents of each step involved in the embodiment of the aforementioned vehicle trajectory prediction method can be referred to the functional description of the functional module corresponding to the vehicle trajectory prediction system in the embodiment of the present invention, and will not be repeated here. The division of modules in the embodiment of the present invention is schematic, which is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention can be integrated into a processor, or it can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules.

[0137] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding functions; the processor described in the embodiment of the present invention can be used for the operation of the vehicle trajectory prediction method.

[0138] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the vehicle trajectory prediction method in the above embodiment.

[0139] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A vehicle trajectory prediction method, characterized in that: include: Obtain historical motion trajectory data of the vehicle to be predicted and each of the adjacent vehicles of the vehicle to be predicted; The historical motion trajectory data of the vehicle to be predicted and its adjacent vehicles are extracted by a preset multi-dimensional dynamic scene feature extraction function to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted; The multi-dimensional dynamic scene feature vector of the vehicle to be predicted is extracted through a preset information extraction neural network to obtain traffic perception information of the vehicle to be predicted; The traffic perception information and historical motion state data of the vehicle to be predicted are encoded through a preset time feature encoder to obtain the hidden state information of the vehicle to be predicted; According to the hidden state information of the vehicle to be predicted, a hybrid attention matrix of the vehicle to be predicted is obtained, and a weight is assigned to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then the trajectory prediction value of the vehicle to be predicted is obtained through maximum pooling processing and full connection processing in sequence; The method of obtaining the mixed attention matrix of the vehicle to be predicted according to the hidden state information of the vehicle to be predicted includes: The mixed attention matrix α of the vehicle to be predicted is obtained by the following formula: α t =softmax(g t (H,h t+1 )) α f =softmax(g f (H,h t+1 )) α＝α t α f Among them, softmax is a normalized exponential function, α t is the time weight vector, α f is the feature weight vector, g t (H,h t+1 )=Hh T , g f (H,h t+1 )=h t+1 (W f H), g f is the feature weight cosine correlation function, g t is the time-weighted cosine correlation function, W f For history h The feature matrix of the vehicle to be predicted in the frame, H is the history T h The hidden state information of the vehicle to be predicted in the frame, h t is the hidden state data of the vehicle to be predicted at time t, h t+1 is the hidden state data of the vehicle to be predicted at time t+1; The method of assigning weights to the hidden state information of the vehicle to be predicted by the hybrid attention matrix of the vehicle to be predicted, and then sequentially performing maximum pooling processing and full connection processing to obtain the trajectory prediction value of the vehicle to be predicted includes: The predicted trajectory value y of the vehicle to be predicted is obtained by the following formula: t+1 : O=α⊙H h t ' +1 =contact(h t+1 ,o t ,o f ) the t+1 =h t ' +1 W2W1 Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o t The most favorable time to improve the prediction accuracy is To perform maximum pooling in the time dimension, O i,j is the hidden state information of the vehicle to be predicted after assigning weights, o f is the most beneficial feature for improving prediction accuracy. To perform maximum pooling in the feature dimension, h t ' +1 For o t , o f and h t+1 The hidden state information of the vehicle to be predicted at time t+1 is obtained by performing full connection processing. Contact is fully connected, and W1 and W2 are preset weights.

2. The vehicle trajectory prediction method according to claim 1, characterized in that: The adjacent vehicles of the to-be-predicted vehicle include adjacent vehicles in eight directions: front, rear, left, right, left front, left rear, right front and right rear of the to-be-predicted vehicle.

3. The vehicle trajectory prediction method according to claim 1, characterized in that: The method of extracting the historical motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted by using a preset multi-dimensional dynamic scene feature extraction function to obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted includes: Through the preset multi-dimensional dynamic scene feature extraction function, the historical motion trajectory data of the vehicle to be predicted and its adjacent vehicles are extracted, and the position data, speed data and acceleration data of the vehicle to be predicted and its adjacent vehicles in the driving direction and the direction perpendicular to the driving direction are obtained to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted.

4. The vehicle trajectory prediction method according to claim 3, characterized in that: The information extraction neural network includes a first convolutional layer, a second convolutional layer, a maximum pooling layer and a fully connected layer connected in sequence; The first convolutional layer is used to fuse the position data, speed data and acceleration data of the vehicle to be predicted and its adjacent vehicles in the driving direction and the vertical direction of the driving direction to obtain the first fusion features of the vehicle to be predicted and its adjacent vehicles; The second convolutional layer is used to fuse the first fused features of the vehicle to be predicted and any adjacent vehicles of the adjacent vehicles of the vehicle to be predicted to obtain the second fused features; The maximum pooling layer is used for maximum pooling processing of the second fusion feature; The fully connected layer is used to fully connect the second fusion features after the maximum pooling process to obtain the traffic perception information of the vehicle to be predicted.

5. The vehicle trajectory prediction method according to claim 1, characterized in that: The preset time feature encoder is a long short-term memory network encoder.

6. A vehicle trajectory prediction system, characterized in that: include: A data acquisition module is used to acquire historical motion trajectory data of the vehicle to be predicted and the vehicles adjacent to the vehicle to be predicted; A data preprocessing module is used to extract the historical motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted; An information extraction module is used to extract a multi-dimensional dynamic scene feature vector of the vehicle to be predicted through a preset information extraction neural network to obtain traffic perception information of the vehicle to be predicted; The encoding module is used to encode the traffic perception information and historical motion state data of the vehicle to be predicted through a preset time feature encoder to obtain the hidden state information of the vehicle to be predicted; A prediction module is used to obtain a hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted, and to assign weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then obtain a trajectory prediction value of the vehicle to be predicted through maximum pooling processing and full connection processing in sequence; The method of obtaining the mixed attention matrix of the vehicle to be predicted according to the hidden state information of the vehicle to be predicted includes: The mixed attention matrix α of the vehicle to be predicted is obtained by the following formula: α t =softmax(g t (H,h t+1 )) α f =softmax(g f (H,h t+1 )) α＝α t α f Among them, softmax is a normalized exponential function, α t is the time weight vector, α f is the feature weight vector, g t (H,h t+1 )=Hh T , g f (H,h t+1 )=h t+1 (W f H), g f is the feature weight cosine correlation function, g t is the time-weighted cosine correlation function, W f For history h The feature matrix of the vehicle to be predicted in the frame, H is the history T h The hidden state information of the vehicle to be predicted in the frame, h t is the hidden state data of the vehicle to be predicted at time t, h t+1 is the hidden state data of the vehicle to be predicted at time t+1; The method of assigning weights to the hidden state information of the vehicle to be predicted by the hybrid attention matrix of the vehicle to be predicted, and then sequentially performing maximum pooling processing and full connection processing to obtain the trajectory prediction value of the vehicle to be predicted includes: The predicted trajectory value y of the vehicle to be predicted is obtained by the following formula: t+1 : O=α⊙H h t ' +1 =contact(h t+1 ,o t ,o f ) the t+1 =h t ' +1 W2W1 Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o t The most favorable time to improve the prediction accuracy is To perform maximum pooling in the time dimension, O i,j is the hidden state information of the vehicle to be predicted after assigning weights, o f is the most beneficial feature for improving prediction accuracy. To perform maximum pooling in the feature dimension, h t ' +1 For o t , o f and h t+1 The hidden state information of the vehicle to be predicted at time t+1 is obtained by performing full connection processing. Contact is fully connected, and W1 and W2 are preset weights.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of vehicle trajectory prediction according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of vehicle trajectory prediction according to any one of claims 1 to 5 are implemented.

Images