A trajectory prediction method for two-way interactive vehicles based on long short-term memory network

By combining long short-term memory networks and self-attention networks, the problem of existing trajectory prediction methods not considering two-way vehicle interaction is solved, achieving higher accuracy in predicting future vehicle trajectories and improving the performance of the prediction algorithm.

CN114565191BActive Publication Date: 2024-11-08JILIN UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210356398.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-30
Publication Date
2024-11-08
Estimated Expiration
2042-03-30

AI Technical Summary

Technical Problem

Existing trajectory prediction methods fail to consider the impact of the predicted vehicle on surrounding vehicles, resulting in low prediction accuracy and an inability to truly reflect traffic scenarios.

Method used

A bidirectional interactive vehicle trajectory prediction method based on long short-term memory network is adopted. The context vector of the vehicle is obtained through encoder, and the interaction weight is calculated by combining self-attention network. The bidirectional interaction between the predicted vehicle and the surrounding vehicles is considered, and the future trajectory is decoded.

Benefits of technology

It improves the accuracy of trajectory prediction and the performance of prediction algorithms, making the prediction results closer to real traffic scenarios and enhancing the accuracy of future vehicle trajectories.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114565191B_ABST
    Figure CN114565191B_ABST
Patent Text Reader

Abstract

The present invention provides a trajectory prediction method for bidirectional interactive vehicles based on a long short-term memory network, including: S1, determining the historical trajectory information of the predicted vehicle and its surrounding vehicles; S2, respectively sending the historical trajectory information of the predicted vehicle and its surrounding vehicles into multiple encoders with long short-term memory networks as basic units, and obtaining the context vector of each vehicle after encoding; S3, position encoding the context vector to obtain a vector containing position information; S4, sending the vector obtained in S3 into the self-attention network to obtain #imgabs0#S5, using #imgabs1# as the input of the self-attention sub-network to obtain the interaction weight between the predicted vehicle and the surrounding vehicles, and then weighting to obtain the final encoding vector of the predicted vehicle; S6, decoding the final encoding vector to obtain the predicted value of the future trajectory of the predicted vehicle. The present invention solves the problem that the existing method fails to consider the real traffic scene and has a low accuracy rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle intelligent driving, and in particular relates to a trajectory prediction method for bidirectional interactive vehicles based on a long short-term memory network. Background Art

[0002] With the rapid development of machine learning technology and in-depth research in the field of artificial intelligence, the field of autonomous driving is undergoing revolutionary changes, and people have seen the possibility of realizing autonomous driving.

[0003] In the field of driverless technology, in order to safely and efficiently navigate complex traffic composed of human drivers, autonomous vehicles need to have proactive capabilities, such as determining when to change lanes, overtake, or slow down to allow other vehicles to merge. This requires autonomous vehicles to be able to accurately predict the future movements of surrounding vehicles in a timely manner.

[0004] At present, people are mainly innovating trajectory prediction in interactive modeling. When doing trajectory prediction, there is a two-way interactive process between two vehicles, that is, there is a game process in the interaction between vehicles. The predicted vehicle will not only be affected by the surrounding vehicles, but it will also affect the trajectory of the surrounding vehicles. However, the interaction currently studied only considers the impact of surrounding vehicles on the predicted vehicle, and does not consider that the predicted vehicle will also affect the trajectory of surrounding vehicles. Therefore, trajectory prediction fails to consider the real traffic scene and has a low accuracy rate.

[0005] In this regard, the present invention provides a trajectory prediction method for bidirectional interactive vehicles based on a long short-term memory network, the input of which is the historical motion trajectory information of the predicted vehicle and its surrounding vehicles, and the output is the predicted value of the future trajectory of the predicted vehicle. Summary of the invention

[0006] The purpose of the present invention is to provide a trajectory prediction method for bidirectional interactive vehicles based on long short-term memory networks, which solves the problem that the existing interactive methods fail to take into account real traffic scenarios and have low accuracy.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is a trajectory prediction method for two-way interactive vehicles based on a long short-term memory network, comprising the following steps:

[0008] S1. Determine the historical trajectory information of the predicted vehicle and its surrounding vehicles;

[0009] S2: The historical trajectory information of the predicted vehicle and its surrounding vehicles is sent to multiple encoders based on long short-term memory networks. The encoders encode the context vectors of each vehicle. Wherein, when i=1, i represents the predicted vehicle, and when i≠1, i represents the surrounding vehicles;

[0010] S3. Give context vector Do position encoding to get a vector containing position information

[0011] S4. Send it to the self-attention network to get the corresponding vectors

[0012] S5, then As the input of a self-attention sub-network, the interaction weights between the predicted vehicle and the surrounding vehicles are obtained, and then the final encoding vector of the predicted vehicle is obtained after weighting.

[0013] S6. The final encoding vector The input decoder decodes and obtains the predicted value of the future trajectory of the predicted vehicle.

[0014] Furthermore, the specific method for determining the historical trajectory information of the predicted vehicle and its surrounding vehicles in S1 is:

[0015] The historical trajectory information of the predicted vehicle and its surrounding vehicles at each moment is estimated through the measurement values ​​obtained by the sensor, and then the corresponding terminal device or server sends this information to the execution vehicle to determine the historical trajectory information of the predicted vehicle and the surrounding vehicles. The historical trajectory information of the i-th vehicle is:

[0016]

[0017] Among them, O i contains the trajectory information of the i-th vehicle at T moments, represents the historical trajectory information of the i-th vehicle at time t, i = 1, 2, ..., n; i = 1 represents the predicted vehicle, i ≠ 1 represents the surrounding vehicles of the predicted vehicle, and the size of n depends on the lateral and longitudinal detection range of the executing vehicle, which is determined by the effective detection range of the sensor on the executing vehicle;

[0018] is the lateral coordinate of the i-th vehicle at time t, is the longitudinal coordinate of the i-th vehicle at time t;

[0019] is the lateral speed of the i-th vehicle at time t, is the longitudinal speed of the ith vehicle at time t;

[0020] is the lateral acceleration of the ith vehicle at time t, is the longitudinal acceleration of the ith vehicle at time t.

[0021] Furthermore, the working process of the long short-term memory network in S2 is: the input at time t in the long short-term memory network is the output value h at the previous time t-1 and the input value x at this moment t , x t Refers to the historical trajectory information in step S1, represented by h t-1 and x t First enter the forget gate and get f t Then enter the input gate and get i t as well as Finally, enter the output gate and get the state value o of the output gate t , the output value f of the forget gate, input gate, and output gate t 、i t , o t Combine them to get long memory C t and short memory h t The information of the i-th vehicle is represented by the long memory C t as the context vector; where f t Decide which historical trajectory features to discard, Represents the trajectory characteristics of the current moment, i t Determine which trajectory features at the current moment should be added to the historical trajectory features.

[0022] Furthermore, the vector representations of the forget gate, input gate, and output gate are as follows:

[0023] Forget Gate:

[0024] f t =σ(W f ·[h t-1 ,x t ]+b f );

[0025] Input Gate:

[0026] i t =σ(W i ·[h t-1 ,x t ]+b i );

[0027]

[0028] Output Gate:

[0029] o t =σ(W o ·[h t-1 ,xt ]+b o );

[0030] The vector representation of the long memory network and the short memory network is as follows:

[0031] Long Memory Networks:

[0032]

[0033] Short Memory Networks:

[0034] h t =o t *tanh(C t );

[0035] Where: W f , W i , W o are the weight matrices of the forget gate, input gate, and output gate respectively; b f 、b i 、b o are the bias terms of the forget gate, input gate, and output gate respectively; σ is the sigmoid activation function; tanh is the activation function; W C is the weight matrix of the cell state, b C is the bias term of the cell state; C t-1 Represents the long memory information of the previous moment.

[0036] Furthermore, the specific steps of S3 are:

[0037] S31. Get the position vector first:

[0038] A local reference system is used with the origin fixed on the predicted vehicle. The vehicle at the origin represents the predicted vehicle, and the rest represent the surrounding vehicles. The y-axis points to the vehicle's forward direction, and the x-axis is the direction perpendicular to the forward direction.

[0039] The vehicle's position vector: e i ={Δ x , Δ y};

[0040] Where: i = 1 represents the predicted vehicle, i ≠ 1 represents the surrounding vehicles; Δ x , Δ y is the relative lateral distance and longitudinal distance between the i-th surrounding vehicle and the predicted vehicle at the last moment of the historical trajectory;

[0041] S32, respectively, the position vector e i and context vector Directly splice them together to get a vector containing position information

[0042] Furthermore, the sequence input into the self-attention network in S4 Output sequence The specific calculation steps are:

[0043] S41. Calculate three matrices: query vector Q, key vector K, value vector V

[0044] Q=W Q *X

[0045] K=W K *X

[0046] V=W V *X

[0047] Among them, W Q , W K , W V is the parameter obtained by each matrix in its own training process, Q = [q1, q2, ...q i ,…q n ],K=[k1,k2,…k i ,…k n ],V=[v1,v2,…v i ,…v n ], q i ,k i ,v i are the values ​​of the i-th vehicle in the corresponding input sequence X. The query vector, key vector, and value vector of

[0048] S42. Calculate attention weight

[0049]

[0050] Among them, D k represents the dimension of Q; j is a subscript used to explain the calculation process, j∈[1,n]; i∈[1,n], i=1 represents the predicted vehicle, and i≠1 represents the surrounding vehicles; k j represents the jth vehicle The key vector of ;

[0051] S43. Calculate the output vector

[0052]

[0053] in, v j represents the jth vehicle A vector of values ​​of .

[0054] Furthermore, the specific steps of S5 are:

[0055] S51. Obtain the interaction weights between the predicted vehicle and its surrounding vehicles:

[0056] The predicted vehicle The query vector is:

[0057]

[0058] The i-th vehicle The key vector is:

[0059]

[0060] Among them, W Q , W K are the parameters obtained during the training process;

[0061] right The correlation coefficient α 1,i for:

[0062] α 1,i =q 1 ·k i

[0063] Then all the correlations α 1,i The interaction weights are obtained through a softmax layer:

[0064]

[0065] Among them, α′ 1,i is between 0 and 1, and e is a natural constant;

[0066] S52, weighting the interaction weights to obtain the final encoding vector of the predicted vehicle

[0067]

[0068] The beneficial effects of the present invention are:

[0069] The trajectory prediction method proposed in the present invention solves the problem that the encoder-decoder structure based on the recursive neural network cannot obtain the interaction between vehicles. In the method of the present invention, the interaction between vehicles not only considers the influence of the surrounding vehicles on the predicted vehicle, but also considers the influence of the predicted vehicle on the trajectory of the surrounding vehicles. This makes the trajectory prediction of the present invention closer to the real traffic scene, and the predicted future trajectory of the vehicle is more accurate, which improves the accuracy of the prediction and the performance of the prediction algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 4 is a flow chart of a trajectory prediction method according to an embodiment of the present invention.

[0072] Figure 2 It is a structural diagram of a trajectory prediction method according to an embodiment of the present invention.

[0073] Figure 3 4 is a structural diagram of a long short-term memory network according to an embodiment of the present invention.

[0074] Figure 4 This is a three-lane highway scene diagram according to an embodiment of the present invention.

[0075] Figure 5 Schematic diagram of a self-attention network according to an embodiment of the present invention.

[0076] Figure 6 Schematic diagram of a self-attention sub-network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] like Figure 1-Figure 2 As shown, the present invention provides a trajectory prediction method for bidirectional interactive vehicles based on a long short-term memory network, comprising the following steps:

[0079] S1. Determine historical trajectory information of the predicted vehicle and its surrounding vehicles, where the historical trajectory information includes position information, speed information, and acceleration information.

[0080] S2: The historical trajectory information of the predicted vehicle and its surrounding vehicles is sent to multiple encoders based on long short-term memory networks. The encoders encode the context vectors of each vehicle. Among them, when i=1, i represents the predicted vehicle, and when i≠1, i represents the surrounding vehicles.

[0081] S3. Context vector Do positional encoding to get a vector containing position information

[0082] S4. Send it to the Self-attention network to get the corresponding vectors

[0083] S5. As the input of a self-attention sub-network, the interaction weights between the predicted vehicle and its surrounding vehicles are obtained, and then the interaction weights are weighted to obtain the final encoding vector of the predicted vehicle.

[0084] S6. The final encoding vector The predicted value of the future trajectory of the predicted vehicle is obtained by decoding with the decoder.

[0085] The present invention introduces the trajectory prediction method of the present application in detail through the following embodiments:

[0086] For ease of understanding, this embodiment completes the trajectory testing method of the present application through the following four modules:

[0087] 1. Historical trajectory acquisition module, used to obtain the historical trajectory information of the predicted vehicle and its surrounding vehicles

[0088] The predicted vehicle and its surrounding vehicles described in this embodiment can be any motor vehicle, including self-driving vehicles and vehicles with assisted driving functions, etc.

[0089] In this embodiment, the autonomous driving vehicle that executes the trajectory prediction method described in this specification is called an execution vehicle. A stationary global reference system is used, and the origin is fixed on the execution vehicle.

[0090] This embodiment makes a prediction at time t based on the historical trajectory at time T before.

[0091] Since the predicted vehicle and its surrounding vehicles are around the executing vehicle, the position information, speed information and acceleration information of the predicted vehicle and its surrounding vehicles at each moment can be estimated through the measurement values ​​obtained by the sensors mounted on the executing vehicle (such as cameras, lidar and radar sensors), or can be obtained through sensor devices mounted at other locations. The corresponding terminal device or server then sends this information to the executing vehicle to determine the historical trajectory information of the predicted vehicle and its surrounding vehicles. The historical trajectory information of the i-th vehicle is:

[0092]

[0093] Among them, O i contains the trajectory information of the i-th vehicle at T moments, It represents the historical trajectory information of the i-th vehicle at time t, i=1, 2, ..., n; when i=1, i represents the predicted vehicle, and the other i represent the surrounding vehicles of the predicted vehicle. n represents the total number of vehicles around the executing vehicle, and the size of n depends on the lateral and longitudinal detection range of the executing vehicle, which is determined by the effective detection range of the sensor on the executing vehicle.

[0094] is the lateral coordinate of the i-th vehicle at time t, is the longitudinal coordinate of the i-th vehicle at time t;

[0095] is the lateral speed of the i-th vehicle at time t, is the longitudinal speed of the ith vehicle at time t;

[0096] is the lateral acceleration of the ith vehicle at time t, is the longitudinal acceleration of the ith vehicle at time t.

[0097] It should be noted that the trajectory of the executing vehicle is planned by itself, and its trajectory is known and does not need to be predicted. However, since the executing vehicle will also participate in traffic, the executing vehicle can be considered as a surrounding vehicle of the predicted vehicle, and the present invention is not limited to this.

[0098] 2. Encoding module, used to obtain the context vector of the predicted vehicle and its surrounding vehicles encoded by the encoder

[0099] The historical trajectory information of the predicted vehicle and its surrounding vehicles is sent to multiple encoders based on long short-term memory networks. The encoders encode the context vectors of the predicted vehicle and its surrounding vehicles. This vector is used to capture the features of past trajectories, i=1, 2, ..., n; in this embodiment, when i=1, it represents the predicted vehicle, and when i≠1, it represents the surrounding vehicles.

[0100] like Figure 2 As shown in the historical trajectory acquisition module and the encoding module, the historical trajectory information O corresponding to the i-th vehicle i Send it to the LSTM Encoder to get the context vector of the i-th vehicle

[0101] The principle of LSTM Encoder is as follows Figure 3As shown, the input of the LSTM Network at time t is the output value h at the previous time. t-1 and the input value x at this moment t , x t Refers to the historical track information above These two parameters first enter the forget gate and get f t Then enter the input gate and get i t as well as Finally, enter the output gate and get the state value o of the output gate t , the output value f of the forget gate, input gate, and output gate t 、i t , o t Combine them to get long memory C t and short memory h t Information. Where f t Decide which historical trajectory features to discard, Represents the trajectory characteristics of the current moment, i t Determine which trajectory features at the current moment should be added to the historical trajectory features. The vector expression of each gate is as follows:

[0102] Forget Gate:

[0103] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0104] Input Gate:

[0105] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0106]

[0107] Output Gate:

[0108] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0109] Two kinds of memory:

[0110] Long Memory:

[0111] Short memory: h t =ot *tanh(C t )

[0112] in:

[0113] W f , W i , W o are the weight matrices of the forget gate, input gate, and output gate respectively; b f 、b i 、b o are the bias terms of the forget gate, input gate, and output gate respectively; σ is the sigmoid activation function; tanh is the activation function; W C is the weight matrix of the cell state, b C is the bias term of the cell state; C t-1 Represents the long memory information of the previous moment.

[0114] In order to fully consider the characteristics of historical trajectories, this embodiment uses a long memory network C t As the context vector to get the final encoding vector.

[0115] If the context vector of each vehicle is directly used for decoding, the interaction between vehicles cannot be obtained. Therefore, in order to make the trajectory prediction model not only consider the interaction between vehicles, but also consider the two-way interaction process between two vehicles and be closer to the real traffic scene, this embodiment also inputs the context vector of the predicted vehicle and its surrounding vehicles into the encoding vector processing module to obtain the final encoding vector of the predicted vehicle considering the two-way interaction.

[0116] 3. Encoding vector processing module, used to obtain the final encoding vector of the predicted vehicle considering the two-way interaction

[0117] (1) The context vector for the i-th vehicle Do positional encoding to get a vector containing position information

[0118] Since the self-attention network does not consider the position information, if we directly get the vector sequence As the input of self-attention, the distance between any two vectors will be the same, and the position information cannot be reflected. Therefore, positional encoding should be performed before doing self-attention.

[0119] There are many ways to do positional encoding. This example is as follows:

[0120] ①First get the position vector.

[0121] like Figure 4 As shown, a local reference system with the origin fixed on the predicted vehicle is used. The vehicle at the origin represents the predicted vehicle, and the rest represent the surrounding vehicles. The y-axis points to the forward direction of the vehicle, and the x-axis is the direction perpendicular to the forward direction.

[0122] The vehicle's position vector e i ={Δ x , Δ y}

[0123] Where: i=1 represents the predicted vehicle, and other i represent surrounding vehicles; Δ x , Δ y It is the relative horizontal and vertical distance between the i-th surrounding vehicle and the predicted vehicle at the last moment of the historical trajectory, which can be easily obtained from the historical trajectory information of the predicted vehicle and the surrounding vehicles in the historical trajectory acquisition module.

[0124] It should be noted that Δ x , Δ y It can be positive, negative or 0, where e1 = {0, 0}.

[0125] ②Respectively change the position vector e i and the context vector considering historical trajectory features Directly splice them together to get a vector containing position information

[0126] (2) Figure 5 As shown, Send it to the Self-attention network to get the corresponding vectors

[0127] The calculation steps of the Self-attention network are:

[0128] Input sequence Output sequence

[0129] ① Calculate three matrices: Q (query vector Query), K (key vector Keys), V (value vector Value)

[0130] Q=W Q *X

[0131] K=W K *X

[0132] V=W V *X

[0133] Among them, W Q , W K , W V are the parameters of each matrix obtained during its training process; Q = [q1, q2, ...q i ,…q n ],K=[k1,k2,…k i ,…k n ],V=[v1,v2,…v i ,…v n ], q i ,k i ,v i They are the values ​​of the i-th vehicle in the input sequence X. The query vector, key vector, and value vector of .

[0134] ②Calculate attention weight

[0135]

[0136] Among them, D k represents the dimension of Q; j is a subscript used to explain the calculation process, j∈[1,n]; i∈[1,n], k j represents the jth vehicle The key vector of .

[0137] ③Calculate the output vector sequence

[0138]

[0139] in, v j represents the jth vehicle A vector of values ​​of .

[0140] (3) Then As the input of a self-attention sub-network, the interaction weights between the predicted vehicle and its surrounding vehicles are obtained, and then the final encoding vector of the predicted vehicle is obtained after weighting.

[0141] Since the input of the previous self-attention network has been positionally encoded and already contains position information, the self-attention sub-network here does not need positional encoding.

[0142] ① Such as Figure 6 As shown, the interaction weights between the predicted vehicle and its surrounding vehicles are obtained:

[0143] The predicted vehicle The query vector is:

[0144]

[0145] The i-th vehicle The key vector is:

[0146]

[0147] Among them, W Q , W K are the parameters obtained during the training process.

[0148] but right The correlation coefficient α 1,i for:

[0149] α 1,i =q 1 ·k i

[0150] Then all the correlations α 1,i Through a softmax layer we get:

[0151]

[0152] The interaction weight α′ 1,i is between 0 and 1, and e is a natural constant.

[0153] ②After weighting, the final encoding vector of the predicted vehicle is obtained

[0154]

[0155] So obtained Not only the impact of the predicted vehicle on its surrounding vehicles is considered, but also the impact of the predicted vehicle on the surrounding vehicles is considered.

[0156] because The interaction of the surrounding vehicles of the predicted vehicle with the predicted vehicle is included in contains the interaction of the predicted vehicle with it, that is, except for i=1 contains the impact of the predicted vehicle on it, then use To obtain the final encoding vector of the predicted vehicle So asked This includes the two-way interaction between the predicted vehicle and its surrounding vehicles.

[0157] In this embodiment, To obtain the final encoding vector of the predicted vehicle The weighted method is used. 1,i Represents the influence of the ith vehicle on the predicted vehicle, and to a certain extent reflects the closeness of the interaction between the predicted vehicle and the surrounding vehicles. This degree of interaction can be large or small. For example, the surrounding vehicles close to the predicted vehicle will be given priority, so the degree of interaction is large; on the contrary, the surrounding vehicles far from the predicted vehicle have a small degree of interaction. It can be considered to reflect the degree of interaction between the predicted vehicle and its surrounding vehicles. Of course, this interaction is bidirectional.

[0158] 4. Decoding module, used to decode and obtain the predicted value of the future trajectory of the predicted vehicle

[0159] The final encoding vector of the predicted vehicle is obtained by The data is sent to a decoder based on a long short-term memory network as the basic unit, and the decoder decodes the predicted value of the future trajectory of the predicted vehicle.

[0160] In this embodiment, the encoder and decoder use a layer of long short-term memory network (LSTMNetwork) at each moment. Of course, multiple layers can also be stacked up deeply, and the present invention does not limit this.

[0161] Simulation experiment:

[0162] The present invention uses the disclosed US-101 dataset from the United States to perform simulation experiments. The dataset includes real highway traffic trajectories captured at 10Hz within a time span of 45 minutes. The dataset consists of 15 minutes of light, moderate and congested traffic conditions. The entire dataset is divided into a training set and a test set. The test set consists of a quarter of the trajectory from each of the three subsets of the US-101 dataset. The trajectory is divided into 8-second segments, where 3 seconds of trajectory history and 5 seconds of prediction range are used. This 8s part is sampled at a dataset sampling rate of 10Hz.

[0163] In this experiment, the ordinary encoder-decoder structure based on the long short-term memory network is called the basic encoder-decoder structure, and the bidirectional interactive encoder-decoder structure based on the long short-term memory network of the present invention is called the improved encoder-decoder structure.

[0164] The basic encoding and decoding structure and the improved encoding and decoding structure are set with the same parameters (the hidden layer dimension of the LSTM network in the encoder is set to 64 dimensions, and the hidden layer dimension of the LSTM network in the decoder is set to 128 dimensions) and trained. After training, the root mean square error (RMSE) of the predicted output is calculated using the test set, and the performance of the model is judged based on the comparison of the root mean square error value. The smaller the root mean square error value, the more accurate the vehicle trajectory prediction and the better the model performance.

[0165] In the simulation experiment, we sampled at a sampling rate of 10 Hz, so there are 10 frames of data in 1 second, and each frame has a root mean square error (RMSE). For the convenience of representation, we calculate the average RMSE value for every 10 frames, i.e. 1 second. The average RMSE value is the sum of the 10 RMSE values ​​within 1 second, and then divided by 10. The smaller the average RMSE value, the more accurate the vehicle trajectory prediction and the better the model performance.

[0166] Table 1 Simulation results

[0167]

[0168] It can be seen from the simulation experiment results that no matter in any prediction period, the average RMSE value of the improved codec structure is always smaller than that of the basic codec structure, which shows that the better the performance of the improved codec structure model, the more accurate the prediction of the vehicle trajectory.

[0169] There are many methods for trajectory prediction. In recent years, data-driven algorithms have been widely used in trajectory prediction methods, and have strong fitting capabilities for complex problems. Among them, the encoder-decoder structure based on the long short-term memory network has good processing capabilities for time series problems and has good characterization capabilities for trajectory information with diversity and uncertainty. However, this structure only considers the impact of the predicted vehicle's own information parameters on the future trajectory of the vehicle itself, and does not take into account the interaction of its surrounding vehicles with it. In this case, the predicted vehicle's future driving trajectory can also be obtained, but the predicted trajectory position error value is relatively high and the accuracy is insufficient. The trajectory prediction method of a two-way interactive vehicle based on a long short-term memory network of the present invention adds a coding vector processing module on the basis of this structure, so that the coding vector finally sent to the decoder for decoding not only considers the impact of surrounding vehicles on the predicted vehicle, but also considers the impact of the predicted vehicle on surrounding vehicles. This makes the present invention closer to the real traffic scene, and the predicted future trajectory of the vehicle is more accurate, which improves the accuracy of the prediction and the performance of the prediction algorithm.

[0170] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0171] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A trajectory prediction method for bidirectional interactive vehicles based on long short-term memory network, characterized in that: The steps include: S1. Determine the historical trajectory information of the predicted vehicle and its surrounding vehicles; S2: The historical trajectory information of the predicted vehicle and its surrounding vehicles is sent to multiple encoders based on long short-term memory networks. The encoders encode the context vectors of each vehicle. Wherein, when i=1, i represents the predicted vehicle, and when i≠1, i represents the surrounding vehicles; S3. Give context vector Do position encoding to get a vector containing position information S4. Send it to the self-attention network to get the corresponding vectors S5, then As the input of a self-attention sub-network, the interaction weights between the predicted vehicle and the surrounding vehicles are obtained, and then the final encoding vector of the predicted vehicle is obtained after weighting. S6. The final encoding vector The input decoder decodes and obtains the predicted value of the future trajectory of the predicted vehicle; The working process of the long short-term memory network in S2 is as follows: the input at time t in the long short-term memory network is the output value h at the previous time t-1 and the input value x at this moment t , x t Refers to the historical trajectory information in step S1, represented by h t-1 and x t First enter the forget gate and get f t Then enter the input gate and get i t as well as Finally, enter the output gate and get the state value o of the output gate t , the output value f of the forget gate, input gate, and output gate t 、i t , o t Combine them to get long memory C t and short memory h t The information of the i-th vehicle is represented by the long memory C t as the context vector; where f t Decide which historical trajectory features to discard, Represents the trajectory characteristics of the current moment, i t Decide which trajectory features at the current moment should be added to the historical trajectory features; The vector representation of the forget gate, input gate, and output gate is as follows: Forget Gate: f t =σ(W f ·[h t-1 ,x t ]+b f ); Input Gate: i t =σ(W i ·[h t-1 ,x t ]+b i ); Output Gate: the t =σ(W o ·[h t-1 ,x t ]+b o ); The vector representation of the long memory network and the short memory network is as follows: Long Memory Networks: Short Memory Networks: h t =o t *tanh(C t ); Where: W f , W i , W o are the weight matrices of the forget gate, input gate, and output gate respectively; b f , b i , b o are the bias terms of the forget gate, input gate, and output gate respectively; σ is the sigmoid activation function; tanh is the activation function; W C is the weight matrix of the cell state, b C is the bias term of the cell state; C t-1 Indicates the information of the long memory of the previous moment; The specific steps of S3 are: S31. Get the position vector first: A local reference system is used with the origin fixed on the predicted vehicle. The vehicle at the origin represents the predicted vehicle, and the rest represent the surrounding vehicles. The y-axis points to the vehicle's forward direction, and the x-axis is the direction perpendicular to the forward direction. The vehicle's position vector: e i ={Δ x , Δ y }; Where: i = 1 represents the predicted vehicle, i ≠ 1 represents the surrounding vehicles; Δ x , Δ y is the relative lateral distance and longitudinal distance between the i-th surrounding vehicle and the predicted vehicle at the last moment of the historical trajectory; S32, respectively, the position vector e i and context vector Directly splice them together to get a vector containing position information The sequence input to the self-attention network in S4 Output sequence The specific calculation steps are: S41. Calculate three matrices: query vector Q, key vector K, value vector V Q=W Q *X K=W K *X V=W V *X Among them, W Q , W K , W V is the parameter obtained by each matrix in its own training process, Q = [q1, q2, ...q i ,…q n ],K=[k1,k2,…k i ,…k n ],V=[v1,v2,…v i ,…v n ], q i ,k i ,v i are the values ​​of the i-th vehicle in the corresponding input sequence X. The query vector, key vector, and value vector of S42. Calculate attention weight Among them, D k represents the dimension of Q; j is a subscript used to explain the calculation process, j∈[1,n]; i∈[1,n], i=1 represents the predicted vehicle, and i≠1 represents the surrounding vehicles; k j represents the jth vehicle The key vector of ; S43. Calculate the output vector in, v j represents the jth vehicle A vector of values ​​of .

2. The trajectory prediction method of a two-way interactive vehicle based on a long short-term memory network according to claim 1 is characterized in that: The specific method for determining the historical trajectory information of the predicted vehicle and its surrounding vehicles in S1 is: The historical trajectory information of the predicted vehicle and its surrounding vehicles at each moment is estimated through the measurement values ​​obtained by the sensor, and then the corresponding terminal device or server sends this information to the execution vehicle to determine the historical trajectory information of the predicted vehicle and the surrounding vehicles. The historical trajectory information of the i-th vehicle is: Among them, O i contains the trajectory information of the i-th vehicle at T moments, represents the historical trajectory information of the i-th vehicle at time t, i = 1, 2, ..., n; i = 1 represents the predicted vehicle, i ≠ 1 represents the surrounding vehicles of the predicted vehicle, and the size of n depends on the lateral and longitudinal detection range of the executing vehicle, which is determined by the effective detection range of the sensor on the executing vehicle; is the lateral coordinate of the i-th vehicle at time t, is the longitudinal coordinate of the i-th vehicle at time t; is the lateral speed of the i-th vehicle at time t, is the longitudinal speed of the ith vehicle at time t; is the lateral acceleration of the ith vehicle at time t, is the longitudinal acceleration of the ith vehicle at time t.

3. The trajectory prediction method of a two-way interactive vehicle based on a long short-term memory network according to claim 1 is characterized in that: The specific steps of S5 are: S51. Obtain the interaction weights between the predicted vehicle and its surrounding vehicles: The predicted vehicle The query vector is: The i-th vehicle The key vector is: Among them, W Q , W K are the parameters obtained during the training process; right The correlation coefficient α 1,i for: α 1,i =q 1 ·k i Then all the correlations α 1,i The interaction weights are obtained through a softmax layer: in, is between 0 and 1, and e is a natural constant; S52, weighting the interaction weights to obtain the final encoding vector of the predicted vehicle