Vehicle track prediction method based on intelligent driving
By building a three-channel network model, combining LSTM, GNN and CNN, the interaction state between vehicles, lane line information and target vehicle trajectory are extracted and jointly predicted, which solves the problem of inaccurate trajectory prediction in different environments of intelligent driving technology, and improves driving safety and perception capabilities.
Patent Information
- Application Number
- CN202411788279.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-28
AI Technical Summary
Existing intelligent driving technology is difficult to accurately predict vehicle trajectories in different road environments, affecting driving safety.
A three-channel network model is adopted, combining long and short-term memory network (LSTM), graph neural network (GNN) and convolutional neural network (CNN), to extract inter-vehicle interaction status information, lane line information and target vehicle trajectory information for joint prediction.
The accuracy and driving safety of vehicle trajectory prediction are improved, allowing the car to adaptively adjust weights in different scenarios and improve perception capabilities.
Smart Images
Figure CN119723495A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a vehicle trajectory prediction method based on intelligent driving, and belongs to the field of intelligent driving. Background Art
[0002] With the rapid development of informatization and intelligence, the application and implementation of artificial intelligence in the automotive industry has gained favor, prompting a significant improvement in the level of vehicle intelligence. Scholars have begun to devote themselves to the further development of automotive intelligence.
[0003] Predicting vehicle trajectories is essential for intelligent driving, and trajectory prediction directly affects the behavior of the car. In order to operate safely, the car's safety system must be able to predict what will happen to the vehicle's surroundings in the near future and implement advance plans. If the vehicle's predicted trajectory is available, the system can make driving decisions and timely prevent dangers to avoid or warn of potential collisions, thereby achieving safer and more efficient driving operations. Therefore, accurate trajectory prediction is the key to safe driving of cars. Summary of the invention
[0004] The present invention designs and develops a vehicle trajectory prediction method based on intelligent driving. By constructing a three-channel network model, it can simultaneously extract the interaction status information between vehicles, lane line information and trajectory information of the target vehicle for joint prediction, thereby improving the prediction accuracy and driving safety of the vehicle.
[0005] The technical solution provided by the present invention is:
[0006] A vehicle trajectory prediction method based on intelligent driving, comprising:
[0007] Step 1: Build a three-channel neural network model to extract vehicle trajectory information features based on long short-term memory network;
[0008] Step 2: Extract the interaction features between different vehicles through graph neural network, and extract the lane structure features through convolutional neural network;
[0009] Step 3: Calculate the weights of the three-channel feature vectors to obtain the predicted trajectory of the target vehicle. Preferably, a three-channel data processing method is used to predict the trajectory of the target vehicle, extract vehicle interaction information through GNN, extract target vehicle trajectory information through LSTM channel, and extract lane structure features based on CNN channel.
[0010] Preferably, the step 1 comprises:
[0011] By using the known vehicle trajectory To predict the trajectory of the vehicle in the future time period
[0012] Assume the known vehicle trajectory is:
[0013] The predicted trajectory is:
[0014] The historical trajectory data of length n Input into the LSTM encoder, after updating the parameters, the historical trajectory context vector H of N vehicles is obtained t-1 ;
[0015]
[0016] Generate a trajectory prediction sequence of target vehicle with expected length m through LSTM decoder: {Y t ,Y t+1 ,...,Y t+m};
[0017] In the formula, is the state of vehicle i at time t, N is the number of target vehicles, m is the time length of the predicted trajectory, n is the time length of the known trajectory, and h t is the hidden vector parameter of the future trajectory of the target vehicle.
[0018] Preferably, the step 2 comprises:
[0019] Set the target vehicle as the center node, and the node value of the center node is V t As the feature vector of the target vehicle extracted by the long short-term memory network, the vehicles around the target vehicle are used as edge nodes E t Construct the graph data structure model G t =(V t , E t );
[0020] The interaction between vehicles is expressed as: hour;
[0021] In the formula, A t-k is tk, k∈{0,1,...,T obs The adjacency matrix of the spatial interaction between vehicles at time .
[0022] Preferably, in step 2, map features are extracted based on a convolutional neural network, and the selection gate in the threshold mechanism is used to select the map features. Select the corresponding map feature according to the state of the target vehicle in the map:
[0023]
[0024] Where M is the map feature information extracted from the map by CNN. is the current position of the target vehicle on the map, W z is the weight parameter, b z is a deviation, σ is the sigmoid function;
[0025] Whether the target vehicle i is affected by the map information at time t is expressed as:
[0026]
[0027] Preferably, the step three comprises:
[0028] Different vehicle trajectory models are evaluated by the root mean square error between the predicted trajectory sequence and the actual trajectory sequence. The root mean square error of each predicted time step in the next 5 seconds is calculated. The calculation formula of the evaluation method is:
[0029]
[0030] In the formula, a is the size of the test set, is the predicted position value of the target vehicle i at time t, is the true value of the target vehicle i at time t.
[0031] Preferably, the long short-term memory network includes: a forget gate f t , Update gate i t And the output gate o t ;
[0032] Among them, the forget gate f t Control the internal state c of the previous moment t-1 How much information do you need to forget? Update gate i t Control the current candidate state c t As valid information, the output gate o t Select the current internal state c t-1 Part of the information is used as output information;
[0033] f t =δ(ω f [h t-1 ,x t ]+b f );
[0034] i t =δ(ω i [h t-1 ,x t ]+b i );
[0035] c t =f*c t-1 +i t *tanh(ωc [h t-1 ,x t ]+b c );
[0036] h t =o t *tanh(c t );
[0037] o t =δ(ω o [h t-1 ,x t ]+b o );
[0038] In the formula, δ is the Sigmoid function, c t-1 and c t is the memory cell state, h t-1 is the hidden state at time t-1, x t is the step input at time t, [h t-1 ,x t ] is the hidden layer state h of the LSTM network at time t-1 t-1 With x t To splice, f ,ω i ,ω c ,ω o are all network self-learning weight matrices, b f 、b i 、b c 、b o are all network bias terms.
[0039] Beneficial effects of the present invention: The vehicle trajectory prediction method based on intelligent driving provided by the present invention, by constructing a three-channel network model, can simultaneously extract the interaction status information between vehicles, lane line information and trajectory information of the target vehicle for joint prediction, so that the vehicle can adaptively adjust the weight under different road environments, effectively improve the accuracy of trajectory prediction in different scenarios, further enhance the perception ability of the car in different scenarios, and improve driving safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 This is a trajectory model diagram of the vehicle trajectory prediction method based on intelligent driving described in the present invention. DETAILED DESCRIPTION
[0041] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.
[0042] like Figure 1As shown, the present invention provides a vehicle trajectory prediction method based on intelligent driving. By constructing a three-channel network model, the interaction state information between vehicles, lane line information and trajectory information of the target vehicle can be simultaneously extracted for joint prediction, so that the vehicle can adaptively adjust the weight under different road environments, thereby improving the perception ability and driving safety of the vehicle in different scenarios, including:
[0043] Step 1: Build a three-channel neural network model to extract vehicle trajectory information features based on long short-term memory network;
[0044] Step 2: Extract the interaction features between different vehicles through graph neural network, and extract the lane structure features through convolutional neural network;
[0045] Step 3: Calculate the weights of the three-channel feature vectors to obtain the predicted trajectory of the target vehicle.
[0046] A three-channel data processing method is used to predict the trajectory of the target vehicle. The interaction information between vehicles is extracted through GNN, the target vehicle trajectory information is extracted through LSTM channel, and the lane structure features are extracted based on CNN channel.
[0047] Specifically include:
[0048] By using the known vehicle trajectory To predict the trajectory of the vehicle in the future time period
[0049] Assume the known vehicle trajectory is:
[0050] The predicted trajectory is:
[0051] It is assumed that the trajectories of vehicles on the road are consistent before time t, and the trajectories of vehicles on the road are unknown after time t.
[0052] In the formula, is the state of vehicle i at time t, N is the number of target vehicles, m is the time length of the predicted trajectory, n is the time length of the known trajectory,
[0053] Vehicles are divided into: the vehicle whose trajectory is to be predicted is called the target vehicle; the adjacent vehicles around the target vehicle are called related vehicles. Trajectory prediction can be characterized as Generate predicted trajectory sequence
[0054] Based on LSTM, an encoder-decoder architecture is used to handle the problem of generating the trajectory sequence of the target vehicle. The trajectory sequences of the target vehicle and related vehicles are encoded through the LSTM encoder, and the interaction relationship between the vehicles is encoded through the graph neural network. The results of the two encodings are cut and input into the LSTM decoder together to parse the trajectory of the target vehicle in the future.
[0055] The LSTM encoder generates the feature vector [h0,h1,h2,...h8] of the corresponding vehicle through the input vector [X0,X1,X2,...X8], calculates the relationship between different feature vectors through the graph neural network, and takes the relationship between different feature vectors and the trajectory of the vehicle as the input of the decoder, and the decoder inputs the predicted trajectory of the target vehicle.
[0056] Long short-term memory network includes: forget gate f t , Update gate i t And the output gate o t ;
[0057] Among them, the forget gate f t Control the internal state c of the previous moment t-1 How much information do you need to forget? Update gate i t Control the current candidate state c t As valid information, the output gate o t Select the current internal state c t-1 Part of the information is used as output information;
[0058] f t =δ(ω f [h t-1 ,x t ]+b f );
[0059] i t =δ(ω i [h t-1 ,x t ]+b i );
[0060] c t =f*c t-1 +i t *tanh(ω c [h t-1 ,x t ]+b c );
[0061] h t =o t *tanh(c t );
[0062] o t =δ(ω o [h t-1 ,x t ]+b o );
[0063] In the formula, δ is the Sigmoid function, c t-1 and c t is the memory cell state, h t-1 is the hidden state at time t-1, x t is the step input at time t, [h t-1 ,x t ] is the hidden layer state h of the LSTM network at time t-1 t-1 With x t To splice, f ,ω i ,ω c ,ω o are all network self-learning weight matrices, b f 、b i 、b c 、b o are all network bias terms.
[0064] The historical trajectory of a vehicle is closely related to its driving characteristics. The LSTM encoder is used to extract features from the historical trajectory of the vehicle. The extracted feature vector is used to establish the connection between the trajectory before and after the lane change, and the characteristics of the vehicle before the lane change are analyzed more deeply to generate a trajectory sequence.
[0065] The historical trajectory data of length n Input into the LSTM encoder and pass through the forget gate f t , Update gate i t And the output gate o t Each formula updates each parameter to obtain the historical trajectory context vector H of N vehicles t-1 ;
[0066]
[0067] The GNN network is used to extract the influence of N-1 vehicles around the target vehicle on the target vehicle trajectory, and the hidden vector parameter h of the target vehicle's future trajectory is obtained. t , the decoder generates an estimated prediction sequence {Y t ,Y t+1 ,...,Y t+m}.
[0068] In the formula, is the state of vehicle i at time t, N is the number of target vehicles, m is the time length of the predicted trajectory, n is the time length of the known trajectory, and h t is the hidden vector parameter of the future trajectory of the target vehicle.
[0069] Set the target vehicle as the center node, and the node value of the center node is V t As the feature vector of the target vehicle extracted by the long short-term memory network, the vehicles around the target vehicle are used as edge nodes E t Construct the graph data structure model G t =(V t , E t );
[0070] The interaction between vehicles is expressed as: hour;
[0071] In the formula, A t-k is tk, k∈{0,1,...,T obs The adjacency matrix of the spatial interaction between vehicles at time .
[0072] Based on the convolutional neural network to extract map features, the threshold mechanism is used to select the gate Select the corresponding map feature according to the state of the target vehicle in the map:
[0073]
[0074] Where M is the map feature information extracted from the map by CNN. is the current position of the target vehicle on the map, W z is the weight parameter, b z is a deviation, σ is the sigmoid function;
[0075] Whether the target vehicle i is affected by the map information at time t is expressed as:
[0076]
[0077] In the present invention, as a preferred embodiment, different vehicle trajectory models are evaluated by the root mean square error between the predicted trajectory sequence and the actual trajectory sequence, and the root mean square error of each predicted time step in the next 5 seconds is calculated. The calculation formula of the evaluation method is:
[0078]
[0079] In the formula, a is the size of the test set, is the predicted position value of the target vehicle i at time t, is the true value of the target vehicle i at time t.
[0080] Example
[0081] The vehicle trajectory data in the public dataset NGSIM is used, which is the trajectory data of passing vehicles collected on Highway 101. The road section consists of 5 main lanes, 1 auxiliary lane, 1 entrance ramp and 1 exit ramp. This dataset records the vehicle data on the highway at a frequency of 10Hz, including: vehicle coordinate points, vehicle speed, acceleration, vehicle number and other data. Most of the data is execution data, and there is less lane change data. Therefore, the data in the dataset is preprocessed to remove some unnecessary data and increase the proportion of lane change data.
[0082] During the preprocessing process, in order to ensure the stability and reliability of the data, only the vehicle trajectory data in the middle 5 lanes and the trajectory data of the vehicles that changed lanes were selected. Among them, the number of frames recorded when the lane change occurred was between the 300th frame and the 1900th frame, and the distance before and after the lane change process needed to be greater than 3m. After processing, 370 target vehicle trajectories were screened out.
[0083] The trajectory of the screened target vehicle is intercepted with 13 seconds before and after the lane change, a total of 26 seconds of data, and data blocks are selected from the trajectory data of these target vehicles as the trajectory prediction data set, that is, the historical trajectory data before 3 seconds and the predicted trajectory data after 5 seconds. A total of 82,736 trajectory data fragments are screened out, and the trajectory data of each fragment sets the initial position of the target vehicle as the coordinate origin; the historical trajectory data part in the data fragment is sampled at a sampling rate of 2, and the predicted trajectory data of the data fragment is sampled at a sampling rate of 5. 10,000 data are randomly jumped out as the test data set, and the remaining 72,736 data are used as the training data set.
[0084] In the present invention, as a preferred embodiment, in the operating environment of the Windows 10 system, based on the pytorch model, the encoder of the historical trajectory data is implemented using an LSTM model with a 32-dimensional hidden state. The decoder of the predicted trajectory data is implemented using a long short-term memory network with a 64-dimensional hidden state, and the interactive encoder is implemented using a graph attention network, and a three-head attention mechanism is used to stabilize the training process. The model was trained for 140 epochs, using the Adam optimizer with a learning rate of 0.001 to minimize the root mean square error, and Mutiprocessing was used to perform multi-core parallel operations on multiple models to ensure the running time. Different vehicle trajectory models are evaluated by the root mean square error between the predicted trajectory sequence and the actual trajectory sequence, and the root mean square error of each predicted time step in the next 5 seconds is calculated. The calculation formula of the evaluation method is:
[0085]
[0086] Where a is the size of the test set, a=10000, is the predicted position value of the target vehicle i at time t, is the true value of the target vehicle i at time t.
[0087] The target vehicle trajectory is predicted based on different models, and the comparison results are shown in Table 1:
[0088] Table 1 Prediction accuracy of different models
[0089]
[0090]
[0091] As can be seen from the table, compared with the CNN-LSTM network, the mean square error of trajectory prediction at 1s, 2s, 3s, 4s, and 5s is increased by approximately 10.9%, 19.8%, 23.8%, 24.2%, and 23.4%, respectively. This shows that the graph neural network extracts the relationship features between the target vehicle and the surrounding vehicles due to the convolutional neural network. Compared with the network with only GNN, the mean square error of the predicted trajectory is increased by approximately 3.4%, 6.1%, 23.1%, 30.5%, and 18.4%, respectively. Compared with the network results without lane information, the mean square error of the predicted trajectory of this method is increased by 58% at 5s.
[0092] The vehicle trajectory prediction method based on intelligent driving provided by the present invention can simultaneously extract the interaction status information between vehicles, lane line information and trajectory information of the target vehicle for joint prediction by constructing a three-channel network model, so that the vehicle can adaptively adjust the weight under different road environments, effectively improve the accuracy of trajectory prediction under different scenarios, further enhance the perception ability of the vehicle in different scenarios, and improve driving safety.
[0093] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.
Claims
1. A vehicle trajectory prediction method based on intelligent driving, characterized in that: include: Step 1: Build a three-channel neural network model to extract vehicle trajectory information features based on long short-term memory network; Among them, the three channels include: interaction status information between vehicles, lane line information, and trajectory information of the target vehicle; Step 2: Extract the interaction features between different vehicles through graph neural network, and extract the lane structure features through convolutional neural network; Step 3: Calculate the weights of the three-channel feature vectors to obtain the predicted trajectory of the target vehicle; Different vehicle trajectory models are evaluated by the root mean square error between the predicted trajectory sequence and the actual trajectory sequence. The root mean square error of each predicted time step in the next 5 seconds is calculated. The calculation formula of the evaluation method is: In the formula, a is the size of the test set, is the predicted position value of the target vehicle i at time t, is the true value of the target vehicle i at time t.
2. The vehicle trajectory prediction method for intelligent driving according to claim 1, characterized in that: The step one comprises: By using known vehicle trajectories To predict the trajectory of the vehicle in the future time period Assume the known vehicle trajectory is: The predicted trajectory is: The historical trajectory data of length n Input into the LSTM encoder, after updating various parameters, the historical trajectory context vector H of N vehicles is obtained t-1 ; Generate a trajectory prediction sequence of target vehicle with expected length m through LSTM decoder: {Y t ,Y t+1 ,...,Y t+m }; In the formula, is the state of vehicle i at time t, N is the number of target vehicles, m is the time length of the predicted trajectory, n is the time length of the known trajectory, and h t is the hidden vector parameter of the target vehicle's future trajectory.
3. The vehicle trajectory prediction method for intelligent driving according to claim 2, characterized in that: The second step comprises: Set the target vehicle as the center node, and the node value of the center node is V t As the feature vector of the target vehicle extracted by the long short-term memory network, the vehicles around the target vehicle are used as edge nodes E t Construct the graph data structure model G t =(V t , E t ); The interaction between vehicles is expressed as: In the formula, A t-k is tk, k∈{0,1,...,T obs The adjacency matrix of the spatial interaction between vehicles at time .
4. The vehicle trajectory prediction method for intelligent driving according to claim 3 is characterized in that: In the step 2, map features are extracted based on the convolutional neural network, and the selection gate in the threshold mechanism is used to select the map features. Select the corresponding map feature according to the state of the target vehicle in the map: Where M is the map feature information extracted from the map by CNN. is the current position of the target vehicle on the map, W z is the weight parameter, b z is a deviation, σ is the sigmoid function; Whether the target vehicle i is affected by the map information at time t is expressed as:
5. The vehicle trajectory prediction method for intelligent driving according to claim 4, characterized in that: The long short-term memory network includes: a forget gate f t , Update gate i t And the output gate o t ; Among them, the forget gate f t Control the internal state c of the previous moment t-1 How much information do you need to forget? Update gate i t Control the current candidate state c t As effective information, the output gate o t Select the current internal state c t-1 Part of the information is used as output information; f t =δ(ω f [h t-1 ,x t ]+b f ); I t =δ(ω i [h t-1 ,x t ]+b i ); c t =f*c t-1 +i t *tanh(ω c [h t-1 ,x t ]+b c ); h t =o t *tanh(c t ); the t =δ(ω o [h t-1 ,x t ]+b o ); In the formula, δ is the Sigmoid function, c t-1 and c t is the memory cell state, h t-1 is the hidden state at time t-1, x t is the step input at time t, [h t-1 ,x t ] is the hidden layer state h of the LSTM network at time t-1 t-1 With x t To splice, f ,ω i ,ω c ,ω o are all network self-learning weight matrices, b f 、b i 、b c 、b o are all network bias terms.