Knowledge data joint driving automatic driving vehicle lane-changing trajectory prediction method

Abstract

The application discloses a knowledge data joint driving automatic driving vehicle lane changing trajectory prediction method. The prediction method comprises the following steps: constructing a knowledge data joint driving automatic driving vehicle lane changing trajectory prediction model, the model comprising a lane changing driving knowledge base composed of lane changing knowledge, a joint prediction module for predicting a lane changing trajectory according to the characteristic state quantity of a host vehicle and surrounding vehicles and the lane changing knowledge, and a knowledge summarizing module for summarizing the prediction results to obtain a final prediction trajectory, wherein the joint prediction module comprises a dynamic space module constructed based on a multi-head graph attention network, a dynamic time sequence module constructed based on a multi-head self-attention mechanism, and a trajectory prediction module. The application can integrate the knowledge about the driving process of a person into a deep learning model, and compared with a deep learning model driven only by data, the prediction accuracy, model transparency and maintainability are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology. Background Technology

[0002] In autonomous driving systems, vehicle trajectory prediction is an important auxiliary function. By predicting the future trajectory of the vehicle, autonomous vehicles can more accurately assess driving risks and make safe and effective interactive driving decisions in complex traffic scenarios such as urban roads, thereby improving traffic efficiency and traffic safety.

[0003] In recent years, with the rapid development of artificial intelligence and vehicle networking technologies, deep learning-based vehicle trajectory prediction methods have achieved good application results due to their powerful prediction performance. However, most of them are based on pure data-driven black box models, lacking interpretability and transparency, which limits the continued development and industrialization of such methods.

[0004] For example, patent application CN117141517 discloses a method for constructing a vehicle trajectory prediction model that combines data-driven and knowledge-guided approaches. This method proposes a vehicle trajectory prediction model based on an intent-aware spatiotemporal attention network within an encoder-decoder framework, and introduces a traffic rule-assisted loss function in the loss function layer to improve the performance of the data-driven model. Alternatively, patent application CN118306403 discloses a vehicle trajectory prediction and behavior decision-making method and system that considers driving style. This method extracts the vehicle state during the vehicle's driving process by constructing a long short-term memory network based on Bayesian optimization and fusion of discrete cosine transform attention mechanism to predict the vehicle's trajectory, while also introducing driver style to improve the model's prediction performance. Or, patent application CN117493424 discloses a vehicle trajectory prediction method that does not rely on map information. This method inputs the processed data into different Transformer temporal encoders to encode the temporal information of vehicles in each scene. Then, it uses a graph convolutional neural network combined with an attention mechanism to obtain the interaction relationship between vehicles. Finally, it uses multiple parallel linear residual layers, i.e., residual network decoders, to complete the trajectory prediction. The existing prediction methods lack spatial relationships between vehicles, resulting in significant errors in predicting future trajectories, especially in long-term time-domain prediction tasks. Furthermore, they all employ data-driven, purely black-box models with poor transparency. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to propose a knowledge- and data-driven method for predicting lane-changing trajectories of autonomous vehicles. This method fully considers the changing spatial relationships between vehicles and integrates knowledge about the human driving process into a deep learning model. Compared to deep learning models that only use data-driven approaches, this significantly improves the model's predictive performance, transparency, and maintainability.

[0006] The technical solution of the present invention is as follows:

[0007] A knowledge-data jointly driven method for predicting lane-changing trajectories of autonomous vehicles includes: predicting trajectories using a trained knowledge-data jointly driven lane-changing trajectory prediction model, wherein the knowledge-data jointly driven lane-changing trajectory prediction model includes: an input module for inputting the current feature state quantities of the vehicle and its nearest surrounding vehicles, and a system of n driver lane-changing knowledge points K. n The constructed lane-change driving knowledge base, based on the input from the input module and each driver lane-change knowledge entry in the knowledge base, performs a lane-change operation on any given lane-change knowledge K. i The system comprises n joint prediction modules for predicting the lane-changing trajectory of a self-driving vehicle under guidance, a knowledge aggregation module that summarizes the prediction results of the n joint prediction modules to obtain the final trajectory prediction result, and an output module that outputs the final trajectory prediction result obtained by the knowledge aggregation module. The joint prediction modules include a dynamic spatial module, a dynamic temporal module, and a trajectory prediction module connected in sequence. The dynamic spatial module extracts spatial interaction features between the self-driving vehicle and the nearest surrounding vehicles, and is constructed based on a multi-head graph attention network, including integrating the input with any lane-changing knowledge K. i The system includes an encoder layer for joint encoding and a decoder layer for decoding; the dynamic timing module can extract the temporal interaction features between the vehicle and the nearest surrounding vehicles, and it is constructed based on a multi-head self-attention mechanism.

[0008] According to some preferred embodiments of the present invention, the nearest surrounding vehicles include the vehicle in the current lane that is in the same lane as the vehicle and is the closest vehicle in front of the vehicle; the vehicle in the target lane that is in the adjacent lane of the vehicle's lane, with its front end behind the vehicle's rear end and the closest vehicle to the vehicle; and the vehicle in the target lane that is in the adjacent lane of the vehicle's lane, with its rear end in front of the vehicle's front end and the closest vehicle to the vehicle; the characteristic state quantities include the type, position, speed, acceleration, spatial distance and time distance between the vehicle and the nearest surrounding vehicles.

[0009] According to some preferred embodiments of the present invention, the lane-changing driving knowledge base includes lane-changing knowledge containing the driver's lane-changing tendencies.

[0010] According to some preferred embodiments of the present invention, the construction of the dynamic space module includes:

[0011] Each vehicle is treated as an independent node v i The spatial interaction relationship between vehicles is edge e i The values ​​of the characteristic state quantities of vehicles recorded by the nodes constitute the traffic graph G = {V, E, A}, where v i Let e ​​represent the i-th independent node. i V represents the i-th edge, with a value of 0 or 1, indicating whether there is an interaction relationship between vehicles. V = {v1, v2, ..., v...} n} represents the set of independent nodes, E = {e1, e2, ..., e} n2} represents the set of edges, and n represents the total number of vehicles;

[0012] The interaction relationship between the vehicle and its nearest surrounding vehicles is set as follows: the vehicle will pay attention to the state of all its nearest surrounding vehicles, the following vehicles will actively pay attention to the state of the vehicle in front, the vehicle in front will not actively pay attention to the state of the following vehicles, and there is a mutual attention state between the vehicles in front and between the following vehicles. Then, the edge E is obtained as shown in the following formula:

[0013]

[0014] Based on the traffic map G = {V, E, A}, spatial interaction features are extracted through the encoder layer and the decoder layer, as follows:

[0015]

[0016] Where G1 represents the spatial interaction features updated by the encoder layer, G2 represents the spatial interaction features parsed by the decoder layer; A represents the set of node features of all nodes; KGANENC represents encoder computation, and GANDEC represents decoder computation.

[0017] The encoder calculation and the decoder calculation include:

[0018] (1) Solve for the attention value between any i-th node and j-th node in the encoder layer and decoder layer respectively, as follows:

[0019] Encoder layer:

[0020] Decoder layer:

[0021] Among them, eij Let be the attention value between the i-th node and the j-th node; and Let a, W, and W be the node features of the i-th and j-th nodes, respectively; LeakyReLU represents the LeakyReLU activation function; || represents the vector concatenation operation; k Let be the weight coefficients to be learned; represent the interaction features between the i-th node and the j-th node under knowledge guidance, as shown in the following formula:

[0022]

[0023] Among them, K s This represents the knowledge matrix corresponding to the s-th lane-changing knowledge, derived from the lane-changing knowledge K. s Weights are assigned to different characteristic state variables to form the state;

[0024] (2) The attention score between the i-th node and the j-th node is calculated as follows:

[0025]

[0026] In the formula: α ij N represents the attention score between the i-th node and the j-th node. i Let represent the set of all neighboring nodes of the i-th node; k represents any neighboring node of the i-th node; exp(·) represents an exponential function with base e.

[0027] (3) Perform operations (1)-(2) on all nodes that have an edge relationship with the i-th node. After the operation is completed, perform node aggregation and update using the following formula:

[0028]

[0029] in, σ(·) represents the updated node features; W0 represents the weight coefficients to be learned; and H represents the number of attention heads in the spatial graph.

[0030] According to some preferred embodiments of the present invention, the dynamic timing module includes, in sequence: an input layer, a position encoding module, a multi-head self-attention layer, a first residual connection and normalization module, a fully connected feedforward neural network layer, a second residual connection and normalization module, and an output layer, wherein the position encoding module can add position encoding to the input, as follows:

[0031]

[0032] Where pos represents the position of a single node in the time series, and d modelLet be the dimension of the input vector, and 2l and 2l+1 represent the positions of the 2l-th and 2l+1-th elements in the input vector.

[0033] According to some preferred embodiments of the present invention, the output of the multi-head self-attention layer is:

[0034] Multihead(Q,K,V)=Concat(head1,...,head h W o

[0035] head i =Attention(QW i Q ,KW i K VW i V )

[0036] Where Q, K, and V represent the query vector Q, key vector K, and value vector V, respectively, Contact() represents the Contact function, and head i This represents the i-th attention head. The parameter matrix that acts as a projection, where R represents the dimensional space, h represents the total number of attention heads, and d v d represents the dimension of the value vector V. k Let K represent the dimension of the key vector. Attention() represents the attention function as follows:

[0037]

[0038] Where T represents the transpose operation.

[0039] According to some preferred embodiments of the present invention, the fully connected feedforward neural network layer comprises two fully connected neural network layers and a linear activation layer in between.

[0040] According to some preferred embodiments of the present invention, both the first residual connection and normalization module and the second residual connection and normalization module include a residual connection layer and a normalization layer.

[0041] According to some preferred embodiments of the present invention, the trajectory prediction module is formed by two fully connected neural networks.

[0042] According to some preferred embodiments of the present invention, the knowledge aggregation module is formed by a single fully connected neural network.

[0043] According to some preferred embodiments of the present invention, the training includes optimizing and adjusting the lane change trajectory prediction model based on the root mean square error loss function.

[0044] This invention integrates driving lane-changing knowledge, including traffic rules and driving lane-changing tendencies, into the network structure, realizing the joint driving of knowledge and data, and improving model transparency, maintainability, and prediction accuracy. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the interactive motion prediction model of the present invention.

[0046] Figure 2 This is a schematic diagram of the overall structure of the knowledge data-driven lane change trajectory prediction model of the present invention.

[0047] Figure 3 This is a schematic diagram of the dynamic timing module in the lane change trajectory prediction model of the present invention.

[0048] Figure 4 This is a diagram showing the predicted trajectory of a vehicle traveling from left to right and merging into the left lane in the example.

[0049] Figure 5 This is a trajectory prediction result diagram of a vehicle traveling from left to right and merging into the right lane in the embodiment.

[0050] Figure 6 This is a diagram showing the predicted trajectory of a vehicle traveling from right to left and merging into the left lane in an example.

[0051] Figure 7 This is a trajectory prediction result diagram of a vehicle traveling from right to left and merging into the right lane in the embodiment. Detailed Implementation

[0052] The present invention will now be described in detail with reference to embodiments and accompanying drawings. However, it should be understood that the embodiments and drawings are for illustrative purposes only and do not constitute any limitation on the scope of protection of the present invention. All reasonable modifications and combinations included within the inventive spirit of the present invention fall within the scope of protection of the present invention.

[0053] According to the technical solution of the present invention, some specific embodiments include the following steps:

[0054] S1 establishes an interactive motion prediction model containing the vehicle whose lane-changing trajectory needs to be predicted and the surrounding vehicles.

[0055] More specifically, see attached Figure 1 As shown, the interactive motion prediction model includes a simulated self-vehicle (i.e., the self-vehicle) and three nearest surrounding vehicles (i.e., the nearest surrounding vehicles). The future driving trajectory of the self-vehicle is determined by the current characteristic state of the self-vehicle and the nearest surrounding vehicles.

[0056] Preferably, the nearest surrounding vehicles include the vehicle in front of the vehicle in the current lane that is in the same lane as the vehicle and at the closest distance in front of the vehicle (e.g., the vehicle in front of ... Figure 1 (As shown in the red car in the middle), the vehicle located in the adjacent lane of the vehicle's current lane, i.e., the target lane that the vehicle can change lanes to, and whose front is behind the vehicle's rear and is the closest vehicle to the vehicle in the target lane (e.g., the vehicle behind the vehicle in the red car). Figure 1 (As shown in the purple car in the middle), and the vehicle in front of you in the target lane that you can change lanes to, located in the adjacent lane of your own vehicle (i.e., the target lane where your rear is in front of your own vehicle and the closest vehicle to you in the target lane) (e.g. Figure 1 (As shown in the yellow car).

[0057] Preferably, the characteristic state quantities, i.e., the characteristic quantities, include the type, position, speed, acceleration, spatial distance and time distance between the vehicle and the nearest surrounding vehicles, and more specifically, as shown in Table 1 below.

[0058] Table 1 Characteristic State Quantities

[0059]

[0060]

[0061] This invention fully considers the numerous factors influencing vehicle lane-changing trajectories in complex dynamic traffic environments. It considers not only the driving status of the vehicle to be predicted (i.e., the vehicle changing lanes) but also the driving status of surrounding vehicles. This requires considering the characteristic information of surrounding vehicles, the interaction characteristics between the vehicle to be predicted and surrounding vehicles, and the interaction characteristics among surrounding vehicles, comprehensively analyzing the vehicle's lane-changing decision-making process. Based on this, this invention establishes a motion prediction model that understands the interactive behavior between vehicles. This motion prediction model can obtain the interaction relationships between the vehicle to be predicted and surrounding vehicles during vehicle movement, thereby generating a more reliable and safer future trajectory for autonomous vehicles during lane-changing processes.

[0062] S2 establishes a model training dataset based on the feature state quantities required by the interactive motion prediction model.

[0063] More specifically, the model training dataset uses the HighD Natural Vehicle Trajectories dataset released by the Automotive Engineering Institute of RWTH Aachen University, Germany (Reference: Krajewski R, Bock J, Kloeker L, et al. The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems[C]. 2018 21st International Conference on Intelligent Transportation Systems (ITSC). 2018). In this dataset, the HighD trajectory data is sampled at a frequency of 25Hz, collecting driving data of over 110,000 vehicles over a 420m long highway section at six locations over 16.5 hours. The raw data was collected synchronously by cameras carried by drones and directly output through video image processing technology.

[0064] When using the above dataset to train, validate, and test the knowledge-data joint-driven lane-changing trajectory prediction model for autonomous vehicles according to this invention, if the number of vehicles m around the vehicle is less than 3, 3-m virtual vehicles can be introduced to ensure the consistency of the characteristics of the surrounding vehicles. Considering the length of the data collection segment is 420m, if the virtual vehicle is in front of the vehicle, the longitudinal distance between the virtual vehicle and the vehicle is set to 500m, the speed to 0m / s, and the acceleration to 0m / s². 2 If the virtual car is behind the owner's car, set the longitudinal distance between the virtual car and the owner's car to -500m, the speed to 0m / s, and the acceleration to 0m / s². 2 .

[0065] S3 establishes a knowledge-data-driven lane-change trajectory prediction model for autonomous vehicles based on an interactive motion prediction model.

[0066] For more details, please refer to the appendix. Figure 2 The knowledge-data jointly driven lane-changing trajectory prediction model for autonomous vehicles established in this invention includes:

[0067] An input module for inputting the current state values ​​of the vehicle and the nearest surrounding vehicles, consisting of n driver lane-changing knowledge points K. n The resulting lane-changing driving knowledge base, based on the input from the input module and each driver lane-changing knowledge entry in the knowledge base, performs a lane-changing operation on any given lane-changing knowledge K. iThe system comprises n joint prediction modules for predicting the lane-changing trajectory of a self-driving vehicle under guidance, a knowledge aggregation module that summarizes the prediction results of the n joint prediction modules to obtain the final trajectory prediction result, and an output module that outputs the final trajectory prediction result obtained by the knowledge aggregation module. The joint prediction modules are based on graph networks and self-attention mechanisms, and include a dynamic spatial module, a dynamic temporal module, and a trajectory prediction module connected in sequence. The dynamic spatial module can extract spatial interaction features between the self-driving vehicle and its nearest surrounding vehicles, including combining the input with any lane-changing knowledge K. i The encoder module performs joint encoding and the decoder module performs decoding. The dynamic timing module can extract the temporal interaction features of the vehicle and the nearest surrounding vehicles.

[0068] Furthermore, in some more specific embodiments, S3 includes:

[0069] S31 constructs the lane-changing driving knowledge base

[0070] This invention fully considers that the driver lane-changing knowledge base should maximize the use of expert prior knowledge to collect data on the driver's actions before and after lane changes, as well as the surrounding environment, and use reasonable methods to interpret and organize the driver's behavior. More specifically, based on driving regulations and driver psychology, this invention constructs a lane-changing driving knowledge base containing the nine lane-changing knowledge items shown in Table 2. Each lane-changing knowledge item is associated with one or more of the characteristic state variables, as shown in Table 2.

[0071] Table 2. Lane-changing driving knowledge base and its associated feature state quantities

[0072]

[0073] According to the lane change trajectory prediction model of the present invention, when changing lanes, the driver will identify the associated feature state variables in the driving scenario under the guidance of lane change knowledge and pay special attention to them.

[0074] S32 constructs the dynamic space module

[0075] The dynamic space module of this invention models the spatial relationship between vehicles into a directed graph structure and uses a graph neural network to extract the spatial interaction features between vehicles.

[0076] In a directed graph, each vehicle is an independent node vi, and they form a set of nodes V, where V = {v1, v2, ..., v...} n The spatial interaction relationship between vehicles is represented by edge e. i Its constituent edge set E, Each node records the values ​​of the vehicle's characteristic state variables, i.e., node features. These features collectively form the node feature set A. Therefore, the spatial relationships between different vehicles can be abstracted into a simple and efficient non-Euclidean traffic graph G = {V, E, A}, where v i Let e ​​represent the i-th independent node. i Let represent the i-th edge, with a value of 0 or 1, indicating whether there is an interaction relationship between vehicles, and n represents the total number of vehicles.

[0077] Furthermore, we treat the vehicle itself, the vehicle in front of it in the current lane, the vehicle in front of it in the target lane, and the vehicle behind it in the target lane as nodes, and set that the vehicle will pay attention to the status of all nearby surrounding vehicles, the vehicle behind will actively pay attention to the status of the vehicle in front, the vehicle in front will not actively pay attention to the status of the vehicle behind, and there is a mutual attention state between vehicles in front and between vehicles behind. Then the edge E of the traffic graph is as shown in equation (2):

[0078]

[0079] Based on the traffic map above, this invention constructs a dynamic spatial module containing an encoder layer and a decoder layer based on a graph attention network (GAT). The aggregation and update process of the encoder layer and the decoder layer is as follows:

[0080] (1) Solve for the attention value between any i-th node and j-th node in the encoder layer and decoder layer respectively, as follows:

[0081] Encoder layer:

[0082] Decoder layer:

[0083] In the formula: e ij Let be the attention value between the i-th node and the j-th node; and ... k represents the weight coefficients to be learned. ij The interaction characteristics between the i-th node and the j-th node under knowledge guidance are shown in the following formula:

[0084]

[0085] Among them, K s Let represent the knowledge matrix corresponding to the s-th lane-changing knowledge.

[0086] The knowledge matrix is ​​formed by assigning weights to different feature state quantities based on lane-changing knowledge. For example, in one specific implementation, the lane-changing knowledge K... i The corresponding knowledge matrix is: In all feature state variables, a lane-changing knowledge K is set. i The corresponding weight value of the associated feature state quantity is 5, and the weight value of other feature state quantities is 1. This yields the weight matrix of all feature state quantities after the settings are obtained, which is then normalized to obtain K. i The knowledge matrix.

[0087] (2) The attention score between the i-th node and the j-th node is calculated as follows:

[0088]

[0089] In the formula: α ij N represents the attention score between the i-th node and the j-th node. i Let represent the set of all neighboring nodes of the i-th node (excluding the i-th node itself); k represents any neighboring node of the i-th node; and exp(·) represents an exponential function with base e.

[0090] (3) Perform operations (1)-(2) on all nodes that have an edge relationship with the i-th node. After the operation is completed, perform node aggregation and update using the following formula:

[0091]

[0092] In the formula: σ represents the updated node features; σ(·) represents the sigmoid activation function; W0 is the weight coefficient to be learned;

[0093] Preferably, the present invention can introduce a multi-head mechanism through GAT, applying the graph attention mechanism in parallel to multiple subspaces, so that each head learns different representations, thereby improving the model's ability to capture different attention features. Then, step (3) performs node aggregation and updating through the following formula:

[0094]

[0095] Where H represents the number of attention heads in the spatial graph.

[0096] The encoder structure based on equations (3), (6), and (8) above is KGAN. ENC The decoder structure constructed according to equations (4), (6), and (8) above is a GAN. DEC Therefore, the process of extracting vehicle spatial interaction features can be represented as follows:

[0097]

[0098] Wherein, G1 represents the spatial interaction features after being updated by the encoder layer, and G2 represents the spatial interaction features after being parsed by the decoder layer.

[0099] S33 constructs the dynamic timing module

[0100] For details, please refer to the appendix. Figure 3 The present invention constructs the dynamic temporal module based on the self-attention mechanism, which can capture the motion relationship between the vehicle and the nearest surrounding vehicles in the time series, i.e., the temporal interaction features.

[0101] The dynamic timing module comprises, in sequence: an input layer, a position encoding module, a multi-head self-attention layer, a first residual connection and normalization module, a fully connected feedforward neural network layer, a second residual connection and normalization module, and an output layer.

[0102] in:

[0103] The input layer takes the output of the dynamic spatial module constructed in S32 as its input. Compared with the original input, the output incorporates spatial interaction information between different features through the dynamic spatial module.

[0104] The position encoding module can add position codes to the input to ensure that the temporal order of information sent to the dynamic timing module at different times is preserved. The specific method of adding position codes is shown in the following formula:

[0105]

[0106] Where pos represents the position of a single node in the time series, and d model Let be the dimension of the input vector, and 2l and 2l+1 represent the positions of the 2l-th and 2l+1-th elements in the input vector.

[0107] Compared to general self-attention models, multi-head self-attention layers can more efficiently focus on global information of the time series. They linearly project the query vector Q, key vector K, and value vector V into the input using different learned projection parameter matrices, and then input the following attention function, Attention(Q,K,V), in the following formula:

[0108]

[0109] Where, d k Let K denote the dimension of the key vector K, and T denote the matrix transpose operation.

[0110] After repeating the process h times or more, the output values ​​of each head are concatenated to obtain the final multi-head self-attention output, as follows:

[0111] Multihead(Q,K,V)=Concat(head1,...,head h W o (12)

[0112] head i =Attention(QW i Q ,KW i K VW i V (13)

[0113] in, The parameter matrix that acts as a projection, where R represents the dimensional space, h represents the total number of attention heads, and d v This represents the dimension of the value vector V.

[0114] The fully connected feedforward neural network layer FFN(X) consists of two fully connected neural network layers and a linear activation layer in between. The linear activation layer uses the ReLU activation function, as shown in equation (14):

[0115]

[0116] Among them W1 FFN , This is the weight matrix of the fully connected layer. This is the bias vector for the fully connected layer.

[0117] The first residual connection and normalization module and the second residual connection and normalization module have the same structure, both including a residual connection layer and a normalization layer. The residual connection layer is used to improve the gradient vanishing problem, as shown in Equation (15):

[0118] out res =in res +FFN(in res (15)

[0119] Among them, in res ,out res These represent the input and output of the residual connection layer, respectively.

[0120] The normalization layer is used to improve the gradient explosion problem, as shown in equation (16):

[0121]

[0122] Where, μ,σ 2 The residual connection layer outputs out as input. res The mean and variance of each layer, ε is an infinitesimal, γ and β are the parameters to be learned, and outnor These represent the outputs of the normalization layer.

[0123] S34 Constructs the trajectory prediction module

[0124] More specifically, the trajectory prediction module consists of two fully connected neural networks Ω, which convert the output of the dynamic timing module into... As input, obtain the knowledge K about lane changing. i Guided vehicle trajectory prediction results L k ,as follows:

[0125]

[0126] in, Indicates knowledge K i Under the guidance of [the relevant authority], the model predicts the vehicle's position at time t.

[0127] S35 constructs the knowledge aggregation module.

[0128] More specifically, the knowledge aggregation module is composed of a fully connected neural network Ψ, which obtains the prediction results from the trajectory prediction module. As input, the predicted trajectory L of the vehicle, guided by all lane-changing knowledge, is obtained as follows:

[0129]

[0130] The knowledge aggregation module of this invention can aggregate the vehicle trajectory prediction results under different knowledge guidance to obtain the final trajectory prediction result.

[0131] S4 trains and adjusts the lane-changing trajectory prediction model for the autonomous vehicle using the model training dataset.

[0132] In some more specific implementations, during training, the parameters of the autonomous vehicle lane-changing trajectory prediction model are optimized and adjusted using the following root mean square error (RMSE) loss function:

[0133]

[0134] Where: N test T represents the total number of test set samples; T represents the number of prediction time-domain frames. These are the true values ​​of the horizontal and vertical coordinates, respectively; These are the predicted values ​​for the horizontal and vertical coordinates, respectively.

[0135] S5 uses a trained lane-change trajectory prediction model to predict the future trajectory of autonomous vehicles.

[0136] Example 1

[0137] The autonomous vehicle lane-changing trajectory prediction model of the present invention was constructed according to the above specific implementation method and experimentally tested. This model was built based on the PyTorch framework, with the adamW optimizer and a learning rate of 0.001. The experimental platform was a Windows environment, with an NVIDIA RTX8000 GPU, an Intel Xeon Gold 6250@3.9GHz CPU (8 cores, 16 processes), and 512GB of memory. During the experiment, the dataset was randomly divided into training, testing, and validation sets in a 7:2:1 ratio. The training batch size was 512, the number of iterations was 80 rounds, and there was no parallel processing.

[0138] Some parameters in the model are set as shown in Table 3:

[0139] Table 3 Model Parameter Settings

[0140]

[0141] We compared several baseline models applied to the HighD dataset and used the root mean square error (RMSE) of the models to represent their predictive performance.

[0142] Specific baseline models include (see references below):

[0143] (1) Long Short-Term Memory Networks (S-LSTM, CS-LSTM, and NLS-LSTM) are used for trajectory prediction.

[0144] (2) Trajectory prediction is performed using self-attention-based neural networks such as Dual-Transformer, STDAN, EA-Net, and FIHF.

[0145] (3) A series of graph neural networks and a self-attention-based neural network are used in conjunction with GCN-Transformer for trajectory prediction;

[0146] (4) A parallel graph neural network and a self-attention mechanism are used in conjunction with BAT for trajectory prediction.

[0147] The results of the comparative tests are shown in Table 4:

[0148] Table 4 Comparison of prediction performance of different models

[0149]

[0150] It can be seen that the RMSE values ​​of each model increase with the extension of the prediction time domain. This indicates that the longer the prediction time domain, the more possible movements the vehicle will exhibit, and the greater the final position deviation of the vehicle will be. The autonomous vehicle lane-changing trajectory prediction model of this invention has the lowest RMSE value across all prediction time domains, reducing it by 50%, 14.39%, 9.05%, and 14.07% compared to other algorithms. Its RMSE value in the 4-second prediction time domain is less than 0.4m, lower than the RMSE values ​​of many algorithms in the 3-second prediction time domain. This shows that compared to other methods, the autonomous vehicle lane-changing trajectory prediction model of this invention can more effectively acquire environmental information around the target vehicle for accurate prediction, performs better in predicting the overall movement trend of the vehicle, and generates a lower predicted trajectory deviation.

[0151] To more intuitively understand the effectiveness of the cutting trajectory prediction, see attached... Figure 4-7 The diagram further illustrates the predicted trajectories of the autonomous vehicle lane-changing trajectory prediction model of the present invention under different driving scenarios. The solid blue, red, yellow, and purple lines represent the historical trajectories of the vehicle and surrounding vehicles; the dashed blue, red, yellow, and purple lines represent the actual trajectories of the vehicle and surrounding vehicles in the next 4 seconds; and the green dashed line represents the predicted trajectory of the autonomous vehicle lane-changing trajectory prediction model of the present invention for the vehicle in the next 4 seconds.

[0152] in, Figure 4 The results show the predicted trajectory of the vehicle traveling from left to right and merging into the left lane. Figure 4 As can be seen, at this moment, the vehicle in front in the current lane and the vehicle in front in the target lane have the same longitudinal position, but the longitudinal speed of the vehicle in the target lane is faster than that of the vehicle in the current lane, and there are no vehicles traveling within 500m behind the vehicle in the target lane. Therefore, the vehicle chooses to change lanes to overtake in order to achieve better traffic performance: 4 seconds later, the vehicle has the same longitudinal position as the vehicle in front in the original lane.

[0153] Figure 5 The results show the trajectory prediction of a vehicle traveling from left to right and merging into the right lane. Figure 5 As can be seen, although the vehicle in front in the current lane is traveling slightly faster than the vehicle in front in the target lane, the vehicle behind in the target lane adjacent to the current vehicle is a heavy vehicle traveling at a slower speed. Therefore, the current vehicle will choose to accelerate and change lanes to avoid driving alongside the heavy vehicle for an extended period.

[0154] Figure 6 The results show the trajectory prediction of a vehicle traveling from right to left and merging into the left lane. Figure 6As can be seen, the vehicle in front in the target lane has left the current lane, and the distance between the current vehicle and the vehicle in front in the current lane is relatively close. Therefore, the current vehicle performs a lane change operation to choose the target lane with more space. Because the distance between the current vehicle and the vehicle behind in the target lane is relatively close, the current vehicle chooses a larger entry angle to change lanes, hoping to overtake the vehicle behind in the target lane.

[0155] Figure 7 The results show the trajectory predictions for a vehicle traveling from right to left and merging into the right lane. Figure 7 As can be seen, the vehicle in front in the target lane has left the current lane. Although there is a certain distance between the vehicle in front in the current lane and the vehicle in front, the traffic capacity is still not as good as that of the target lane. Moreover, the vehicle's speed is higher than that of the vehicle behind in the target lane, so there are sufficient conditions for changing lanes. Therefore, the vehicle decisively chooses to change lanes.

[0156] Based on the above prediction results, it can be seen that the autonomous vehicle lane-changing trajectory prediction model of the present invention can effectively obtain the potential relationship between the vehicle and surrounding vehicles, and make reasonable decisions based on the current scenario. The predicted trajectory has a high degree of overlap with the actual trajectory in different scenarios, showing good prediction performance. At the same time, there is no collision risk between the predicted trajectory of the vehicle and the actual driving trajectory of surrounding vehicles, indicating that the model can effectively obtain the interaction relationship between the target vehicle state and the surrounding environment, improving the model safety while improving the prediction accuracy.

[0157] The references in Table 4 are as follows:

[0158] [1]Alahi A, Goel K, Ramanathan V, et al.Social LSTM: Human Trajectory Prediction in Crowded Spaces[J].Proceedings of the IEEE / CVF conference oncomputer vision and pattern recognition.2016.

[0159] [2]Deo N,Trivedi M M.Convolutional social pooling for vehicle trajectory prediction[C].Proceedings of the IEEE conference on computer visionandpattern recognition workshops.2018:1468-1476.

[0160] [3]K.Messaoud,I.Yahiaoui,A.Verroust-Blondet,et al.Non-local SocialPooling for Vehicle Trajectory Prediction[C].2019IEEE Intelligent VehiclesSymposium(IV),2019:975-980.

[0161] [4]K.Gao,et al.Dual Transformer Based Prediction for Lane ChangeIntentions and Trajectories in Mixed Traffic Environment[J].IEEE Transactionson Intelligent Transportation Systems,2023,24(06):6203-6216

[0162] [5]X.Chen,H.Zhang,F.Zhao,et al.Intention-Aware Vehicle TrajectoryPrediction Based on Spatial-Temporal Dynamic AttentionNetwork for Internet ofVehicles[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(10):19471-19483.

[0163] [6]Y.Cai et al.Environment-AttentionNetwork for Vehicle TrajectoryPrediction[J].IEEE Transactions on Vehicular Technology.2021,70(11):11216-11227.

[0164] [7]Zuo Z, Wang X, Guo S, et al. Trajectory prediction network of autonomous vehicles with fusion of historical interactive features [J]. IEEE Transactions on Intelligent Vehicles, 2023.

[0165] [8] Han Tianli, Ma Chi, Hu Linzhi. A method for modeling vehicle lane-changing behavior and predicting trajectory based on GCN-Transformer [J]. Modeling and Simulation, 2024, 13(3):2754-2771.

[0166] Han TL,Ma C,Hu L ZA Vehicle Lane-Changing Behavior Modeling and Trajectory Prediction MethodBased on GCN-Transformer[J].Modeling andSimulation,2024,13(3):2754-2771.

[0167] [9] Liao H, Li Z, Shen H, et al. Bat: Behavior-aware human-like trajectory prediction for autonomous driving [C]. Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(9): 10332-10340.

[0168] The above embodiments are merely preferred embodiments of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A knowledge-data jointly driven method for predicting lane-changing trajectories of autonomous vehicles, characterized in that, It predicts the trajectory of an autonomous vehicle using a knowledge-data-driven lane-changing trajectory prediction model that has been trained. This model includes: an input module for inputting the current feature state of the vehicle and its nearest surrounding vehicles; and n driver lane-changing knowledge points. The constructed lane-change driving knowledge base, based on the input from the input module and each driver lane-change knowledge entry in the knowledge base, performs any lane-change operation. The system comprises n joint prediction modules for predicting the lane-changing trajectory of a self-driving vehicle under guidance, a knowledge aggregation module that summarizes the prediction results of the n joint prediction modules to obtain the final trajectory prediction result, and an output module that outputs the final trajectory prediction result obtained by the knowledge aggregation module. The joint prediction modules include a dynamic spatial module, a dynamic temporal module, and a trajectory prediction module connected in sequence. The dynamic spatial module extracts spatial interaction features between the self-driving vehicle and the nearest surrounding vehicles, and is constructed based on a multi-head graph attention network, including integrating the input with any lane-changing knowledge... An encoder layer for joint encoding and a decoder layer for decoding; the dynamic temporal module can extract the temporal interaction features between the vehicle and the nearest surrounding vehicles, and it is constructed based on a multi-head self-attention mechanism; the construction of the dynamic spatial module includes: Each vehicle is treated as an independent node v i The spatial interaction relationship between vehicles is edge e i The values ​​of the characteristic state quantities of vehicles recorded by each node constitute the traffic map. , where v i Let e ​​represent the i-th independent node. i Let represent the i-th edge, with a value of 0 or 1, indicating whether there is an interaction relationship between vehicles. Represents a set of independent nodes. Let n represent the set of edges, and n represent the total number of vehicles. The interaction relationship between the vehicle and its nearest surrounding vehicles is set as follows: the vehicle will pay attention to the state of all its nearest surrounding vehicles, the following vehicles will actively pay attention to the state of the vehicle in front, the vehicle in front will not actively pay attention to the state of the following vehicles, and there is a mutual attention state between the vehicles in front and between the following vehicles. Then, the edge E is obtained as shown in the following formula: ； Based on the traffic map Spatial interaction features are extracted through the encoder layer and the decoder layer, as follows: Where G1 represents the spatial interaction features updated by the encoder layer, and G2 represents the spatial interaction features parsed by the decoder layer; A represents the set of node features of all nodes; KGAN ENC Indicates encoder computation, GAN DEC This indicates that the decoder is calculating; The encoder calculation and the decoder calculation include: (1) Solve for the attention value between any i-th node and j-th node in the encoder layer and decoder layer respectively, as follows: Encoder layer: Decoder layer: in, Let be the attention value between the i-th node and the j-th node; and ... These are the weighting coefficients to be learned; The interaction characteristics between the i-th node and the j-th node under knowledge guidance are shown in the following formula: Among them, K s This represents the knowledge matrix corresponding to the s-th lane-changing knowledge, derived from the lane-changing knowledge K. s Weights are assigned to different characteristic state variables to form the state; (2) Solve for the attention score between the i-th node and the j-th node, as follows: In the formula: Let be the attention score between the i-th node and the j-th node; Let represent the set of all neighboring nodes of the i-th node; k represents any neighboring node of the i-th node. This represents an exponential function with base e; (3) Perform operations (1)-(2) on all nodes that have an edge relationship with the i-th node. After the operation is completed, perform node aggregation and update using the following formula: in, This represents the updated node characteristics; This represents the sigmoid activation function; W0 represents the weight coefficients to be learned; and H represents the number of attention heads in the spatial graph.

2. The method for predicting lane-changing trajectories of autonomous vehicles according to claim 1, characterized in that, The nearest surrounding vehicles include the vehicle in the current lane that is in the same lane as the vehicle and is the closest vehicle in front of the vehicle; the vehicle in the target lane that is in the adjacent lane of the vehicle's lane, with its front end behind the vehicle's rear end and the closest vehicle to the vehicle; and the vehicle in the target lane that is in the adjacent lane of the vehicle's lane, with its rear end in front of the vehicle's front end and the closest vehicle to the vehicle. The characteristic state quantities include the type, position, speed, acceleration, spatial distance and time distance between the vehicle and the nearest surrounding vehicles.

3. The method for predicting lane-changing trajectories of autonomous vehicles according to claim 1, characterized in that, The lane-changing driving knowledge base includes lane-changing knowledge containing drivers' lane-changing tendencies.

4. The method for predicting lane-changing trajectories of autonomous vehicles according to claim 1, characterized in that, The dynamic timing module comprises, in sequence: an input layer, a position encoding module, a multi-head self-attention layer, a first residual connection and normalization module, a fully connected feedforward neural network layer, a second residual connection and normalization module, and an output layer. The position encoding module adds positional encoding to the input, as follows: ； Where pos represents the position of a single node in the time series, and d model The dimension of the input vector. and The first term in the input vector The element and the first The position of each element.

5. The method for predicting the lane-changing trajectory of an autonomous vehicle according to claim 4, characterized in that, The output of the multi-head self-attention layer is: ， Where Q, K, and V represent the query vector Q, key vector K, and value vector V, respectively, Contact() represents the Contact function, and head i This represents the i-th attention head. , , , The parameter matrix that acts as the projection matrix, where R represents the dimensional space, h represents the total number of attention heads, and d v d represents the dimension of the value vector V. k This represents the dimension of the key vector K. ( ) represents the attention function as follows: ， Where T represents the transpose operation.

6. The method for predicting the lane-changing trajectory of an autonomous vehicle according to claim 4, characterized in that, in, The fully connected feedforward neural network layer includes two fully connected neural network layers and a linear activation layer in between; and / or, the first residual connection and normalization module and the second residual connection and normalization module both include a residual connection layer and a normalization layer.

7. The method for predicting lane-changing trajectories of autonomous vehicles according to claim 1, characterized in that, The trajectory prediction module is formed by two fully connected neural networks.

8. The method for predicting the lane-changing trajectory of an autonomous vehicle according to claim 1, characterized in that, The knowledge aggregation module is formed by a single fully connected neural network.

9. The method for predicting lane-changing trajectories of autonomous vehicles according to claim 1, characterized in that, The training includes optimizing and adjusting the lane change trajectory prediction model based on the root mean square error loss function.

Images