Knowledge data combined driving automatic driving vehicle lane changing track prediction method

Through the combined driving method of knowledge data, combined with multi-head graph attention network and self-attention mechanism, the driver's lane change knowledge is integrated, and the problem of missing spatial position relationship between vehicles in the existing technology is solved, and the prediction of trajectory of autonomous driving vehicles is improved. Accuracy and transparency.

CN118876996AActive Publication Date: 2024-11-01SHENZHEN AUTOMOTIVE RES INST BEIJING INST OF TECH (SHENZHEN RES INST OF NAT ENG LAB FOR ELECTRIC VEHICLES)

Patent Information

Application Number
CN202411215666.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-01
Estimated Expiration
2044-09-02

AI Technical Summary

Technical Problem

The existing trajectory prediction methods for autonomous driving vehicles lack consideration of spatial position relationships between vehicles, resulting in large errors in future trajectory prediction and a purely data-driven black box model with poor transparency and maintainability.

Method used

A method for predicting lane change trajectory of autonomous driving vehicles driven by knowledge data is proposed. By constructing a model driven by knowledge data, combining multi-head graph attention network and self-attention mechanism, it integrates driver lane change knowledge and extracts the space between vehicles. Interact features with time to generate more accurate trajectory predictions.

Benefits of technology

It significantly improves the prediction performance, transparency and maintainability of the model, reduces the error in future trajectory prediction, and the generated trajectory is more reliable and safe.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118876996A_ABST
    Figure CN118876996A_ABST
Patent Text Reader

Abstract

The invention discloses a method for predicting a lane changing track of an automatic driving vehicle jointly driven by knowledge data. The prediction method comprises the following steps: constructing a knowledge data combined driving automatic driving vehicle lane changing track prediction model; the model comprises a lane changing driving knowledge base composed of lane changing knowledge, a joint prediction module for predicting a lane changing track according to characteristic state quantities of a vehicle and surrounding vehicles and the lane changing knowledge, and a knowledge summarization module for summarizing prediction results to obtain a final prediction track, the joint prediction module comprises a dynamic space module formed based on a multi-head graph attention network, a dynamic time sequence module constructed based on a multi-head self-attention mechanism, and a trajectory prediction module. According to the method, knowledge about the driving process of people can be integrated into the deep learning model, and compared with a deep learning model only driven by data, the prediction accuracy, model transparency and maintainability are remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving. Background Art

[0002] In the autonomous driving system, vehicle trajectory prediction is an important auxiliary function. By predicting the future driving trajectory of the vehicle, the autonomous driving vehicle can more accurately judge the driving risks, make safe and effective interactive driving decisions in complex traffic scenarios such as urban roads, and improve traffic efficiency and traffic safety.

[0003] In recent years, with the rapid development of artificial intelligence and Internet of Vehicles technology, vehicle trajectory prediction methods based on deep learning have achieved good application results due to their powerful prediction performance. However, most of them are based on pure data-driven black box models and lack interpretability and transparency, which limits the continued development and industrialization of such methods.

[0004] For example, the existing patent application CN117141517 discloses a method for constructing a vehicle trajectory prediction model that combines data-driven and knowledge-guided methods. The method proposes a vehicle trajectory prediction model based on an intention-aware spatiotemporal attention network under an encoder-decoder framework, and introduces a traffic rule-assisted loss function in the loss function layer to improve the performance of the data-driven model; or the existing patent application CN118306403 discloses a vehicle trajectory prediction and behavior decision-making method and system that considers driving style, which extracts the vehicle state during vehicle driving by constructing a long short-term memory network based on Bayesian optimization and discrete cosine transform attention mechanism to predict the vehicle trajectory, and introduces the driver style to improve the prediction performance of the model; or the existing patent application CN117493424 discloses a vehicle trajectory prediction method that does not rely on map information, which inputs the processed data into different Transformer time encoders to encode the time information of the vehicle in each scene, and then uses a graph convolutional neural network combined with an attention mechanism to obtain the interaction relationship between vehicles, and finally uses multiple parallel linear residual layers, namely, a residual network decoder, to complete the trajectory prediction. The above existing prediction methods lack the spatial position relationship between vehicles and generally have large errors in predicting future trajectories, especially in long-term prediction tasks. In addition, they all adopt data-driven pure black box models with poor transparency. Summary of the invention

[0005] In view of the defects of the prior art, the purpose of the present invention is to propose a knowledge-data-co-driven lane change trajectory prediction method for autonomous driving vehicles. This method fully considers the changes in the spatial position relationship between vehicles and incorporates the knowledge about the human driving process into the deep learning model. Compared with the deep learning model that only uses data-driven, it significantly improves the prediction performance, transparency and maintainability of the model.

[0006] The technical solution of the present invention is as follows:

[0007] A method for predicting lane-changing trajectories of an autonomous driving vehicle driven by knowledge data, comprising: performing trajectory prediction by using a trained knowledge data-driven lane-changing trajectory prediction model for an autonomous driving vehicle, wherein the knowledge data-driven lane-changing trajectory prediction model, i.e., the lane-changing trajectory prediction model, comprises: an input module for inputting current characteristic state quantities of the vehicle and its nearest surrounding vehicles, a lane-changing trajectory prediction model composed of n driver lane-changing knowledge K n The constructed lane-changing driving knowledge base performs any lane-changing knowledge K according to the input of the input module and each lane-changing knowledge of the driver in the lane-changing driving knowledge base. i n joint prediction modules for predicting the lane change trajectory of the ego vehicle under the guidance of the present invention, a knowledge aggregation module for aggregating the prediction results of the n joint prediction modules and obtaining the final trajectory prediction result, and an output module for outputting the final trajectory prediction result obtained by the knowledge aggregation module; wherein the joint prediction module includes a dynamic space module, a dynamic time series module and a trajectory prediction module connected in sequence; wherein the dynamic space module can extract the spatial interaction features between the ego vehicle and the nearest surrounding vehicles, which is constructed based on a multi-head graph attention network, including combining the input and any lane change knowledge K i An encoder layer for joint encoding and a decoder layer for decoding; the dynamic timing module can extract the temporal interaction features between the vehicle and the nearest surrounding vehicles, which is constructed based on a multi-head self-attention mechanism.

[0008] According to some preferred embodiments of the present invention, the nearest surrounding vehicles include the vehicle in front of the current lane which is in the same lane as the vehicle and is at the closest distance in front of the vehicle, the vehicle in the target lane which is in an adjacent lane to the lane where the vehicle is located, with its front end located behind the rear end of the vehicle and is closest to the vehicle, and the vehicle in front of the target lane which is in an adjacent lane to the lane where the vehicle is located, with its rear end located in front of the front end of the vehicle and is closest to the vehicle; the characteristic state quantities include the type, position, speed, acceleration, spatial distance and time distance between the vehicle and the nearest surrounding vehicles.

[0009] According to some preferred embodiments of the present invention, the lane-changing driving knowledge base includes lane-changing knowledge including the driver's lane-changing tendency.

[0010] According to some preferred embodiments of the present invention, the construction of the dynamic space module includes:

[0011] Each vehicle is regarded as an independent node v i , the spatial interaction relationship between vehicles is edge e i , the characteristic state value of the vehicle recorded by the node is the node characteristic composition traffic graph G = {V, E, A}, where v i represents the ith independent node, e i represents the i-th edge, whose value is 0 or 1, indicating whether there is an interaction relationship between vehicles, V = {v1,v2,...,v n} represents a set of independent nodes, E = {e1, e2, ..., e n2} represents the edge set, n represents the total number of vehicles;

[0012] The interactive relationship between the self-vehicle and the nearest surrounding vehicles is set as follows: the self-vehicle will pay attention to the status of all the nearest surrounding vehicles, the rear vehicle will actively pay attention to the status of the front vehicle, the front vehicle will not actively pay attention to the status of the rear vehicle, and there is a mutual focus state between the front vehicle and the front vehicle and between the rear vehicles, then the edge E is as shown in the following formula:

[0013]

[0014] Based on the traffic graph G={V, E, A}, spatial interaction feature extraction is performed through the encoder layer and the decoder layer as follows:

[0015]

[0016] Among them, G1 represents the spatial interaction features after being updated by the encoder layer, G2 represents the spatial interaction features after being parsed by the decoder layer; A represents the set of node features of all nodes; KGANENC represents encoder calculation, and GANDEC represents decoder calculation;

[0017] Wherein, the encoder calculation and the decoder calculation include:

[0018] (1) Solve the attention value between any i-th node and j-th node in the encoder layer and decoder layer respectively, as follows:

[0019] Encoder layer:

[0020] Decoder layer:

[0021] Among them, eij is the attention value between the i-th node and the j-th node; and are the node features of the i-th node and the j-th node respectively; LeakyReLU represents the LeakyReLU activation function; || represents the concatenation operation of the vector; a, W, W k is the weight coefficient to be learned; it represents the interaction characteristics between the i-th node and the j-th node under the guidance of knowledge, as shown in the following formula:

[0022]

[0023] Among them, K s The knowledge matrix corresponding to the lane-changing knowledge K is represented by s The weights of different characteristic state quantities are assigned;

[0024] (2) Solve the attention score between the i-th node and the j-th node as follows:

[0025]

[0026] Where: α ij is the attention score between the i-th node and the j-th node; N i represents the set of all neighbor nodes of the ith node; k represents any neighbor node of the ith node, and exp(·) represents an exponential function with e as the base;

[0027] (3) Perform operations (1) to (2) on all nodes that have an edge relationship with the i-th node. After the operations are completed, the nodes are aggregated and updated using the following formula:

[0028]

[0029] in, represents the updated node feature; σ(·) represents the sigmoid activation function; W0 is the weight coefficient to be learned, and H is the number of spatial graph attention heads.

[0030] According to some preferred embodiments of the present invention, the dynamic timing module includes: an input layer, a position encoding module, a multi-head self-attention layer, a first residual connection and normalization module, a fully connected feedforward neural network layer, a second residual connection and normalization module and an output layer connected in sequence, wherein the position encoding module can add position encoding to the input as follows:

[0031]

[0032] Among them, pos represents the position of a single node in the time series, d modelis the dimension of the input vector, 2l and 2l+1 represent the positions of the 2lth element and 2l+1th element in the input vector.

[0033] According to some preferred embodiments of the present invention, the output of the multi-head self-attention layer is:

[0034] Multihead(Q,K,V)=Concat(head1,...,head h )W o

[0035] head i =Attention(QW i Q ,KW i K ,VW i V )

[0036] Among them, Q, K, and V represent the query vector Q, the key vector K, and the value vector V respectively, Contact() represents the Contact function, and head i represents the i-th attention head, is the parameter matrix that plays a projection role, where R represents the dimensional space, h represents the total number of attention heads, and d v Represents the dimension of the value vector V, d k represents the dimension of the key vector K, and Attention() represents the following attention function:

[0037]

[0038] Wherein, T represents the transpose operation.

[0039] According to some preferred embodiments of the present invention, the fully connected feedforward neural network layer includes two layers of fully connected neural networks and a linear activation layer in between.

[0040] According to some preferred embodiments of the present invention, the first residual connection and normalization module and the second residual connection and normalization module both include a residual connection layer and a normalization layer.

[0041] According to some preferred embodiments of the present invention, the trajectory prediction module is formed by a two-layer fully connected neural network.

[0042] According to some preferred embodiments of the present invention, the knowledge aggregation module is formed by a layer of fully connected neural network.

[0043] According to some preferred embodiments of the present invention, the training includes optimizing and adjusting the lane change trajectory prediction model according to a root mean square error loss function.

[0044] The present invention can integrate driving lane-changing knowledge including traffic rules and driving lane-changing tendencies into the network structure, realize the joint drive of knowledge and data, and improve the model transparency, maintainability and prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the interactive motion prediction model of the present invention.

[0046] Figure 2 Schematic diagram of the overall structure of the lane change trajectory prediction model driven by knowledge data in the present invention.

[0047] Figure 3 It is a structural schematic diagram of the dynamic timing module in the lane-changing trajectory prediction model of the present invention.

[0048] Figure 4 This is a diagram showing the trajectory prediction result of the vehicle traveling from left to right and merging left in the embodiment.

[0049] Figure 5 This is a diagram showing the trajectory prediction result of the vehicle traveling from left to right and merging right in the embodiment.

[0050] Figure 6 This is a diagram showing the trajectory prediction result of the vehicle traveling from right to left and merging left in the embodiment.

[0051] Figure 7 This is a diagram showing the trajectory prediction result of the vehicle traveling from right to left and merging right in the embodiment. DETAILED DESCRIPTION

[0052] The present invention is described in detail below in conjunction with the embodiments and drawings, but it should be understood that the embodiments and drawings are only used to exemplify the present invention and do not constitute any limitation on the protection scope of the present invention. All reasonable changes and combinations within the scope of the inventive concept of the present invention fall within the protection scope of the present invention.

[0053] According to the technical solution of the present invention, some specific implementation methods include the following steps:

[0054] S1 establishes an interactive motion prediction model containing the ego vehicle and its surrounding vehicles for lane change trajectory prediction

[0055] More specifically, as attached Figure 1 As shown, the interactive motion prediction model includes a simulated ego vehicle (i.e., ego vehicle) and three surrounding vehicles closest to it, i.e., nearest surrounding vehicles, wherein the future driving trajectory of the ego vehicle is determined by the current characteristic state quantities of the ego vehicle and the nearest surrounding vehicles.

[0056] Preferably, the nearest surrounding vehicles include the vehicle in the same lane as the vehicle and the vehicle in front of the current lane at the closest distance in front of the vehicle (such as Figure 1 The red car in the middle), the target lane adjacent to the lane where the ego vehicle is located, i.e., the target lane to which the ego vehicle can change lanes, the target lane rear vehicle whose front end is behind the rear of the ego vehicle and is closest to the ego vehicle (such as Figure 1 The vehicle in front of the target lane (shown as the purple vehicle in the middle), which is located in the adjacent lane of the lane where the vehicle is located, i.e., the target lane to which the vehicle can change lanes, and whose rear end is in front of the front of the vehicle and is closest to the vehicle (such as Figure 1 (shown in yellow).

[0057] Preferably, the characteristic state quantity, i.e., the characteristic quantity, includes the type, position, speed, acceleration, spatial distance and time distance between the vehicle and the nearest surrounding vehicles, more specifically, as shown in Table 1 below.

[0058] Table 1 Characteristic state quantities

[0059]

[0060]

[0061] The present invention fully considers that in a complex dynamic traffic environment, there are many factors that affect the trajectory of a vehicle's lane change. Not only should the driving conditions of the vehicle to be predicted, that is, the vehicle that changes lanes, be considered, but also the driving conditions of the vehicles around the vehicle to be predicted, that is, it is necessary to consider the characteristic information of the surrounding vehicles, the characteristic information of the interaction between the vehicle to be predicted and the surrounding vehicles, and between the surrounding vehicles, and comprehensively analyze the vehicle's lane change decision process. Based on this, the present invention establishes a motion prediction model that can understand the interactive behavior between vehicles. The motion prediction model can obtain the interactive relationship between the vehicle to be predicted and the surrounding vehicles when the vehicle is driving, and then generate a more reliable and safe future trajectory for the autonomous driving vehicle during the lane change process.

[0062] S2 establishes a model training data set based on the characteristic state quantities required by the interactive motion prediction model

[0063] More specifically, the data of the model training data set adopts the HighD natural vehicle trajectory data set released by the Institute of Automotive Engineering of RWTH Aachen University in Germany (reference: Krajewski R, Bock J, Kloeker L, et al. The HighD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems [C]. 2018 21st International Conference on Intelligent Transportation Systems (ITSC). 2018). In this data set, the sampling frequency of HighD trajectory data is 25 Hz, and the driving data of more than 110,000 vehicles in a 420m long highway section in 16.5 hours at six locations are collected. The original data is synchronously collected by the camera carried by the drone and directly output by video image processing technology.

[0064] When the above data set is used to train, verify and test the lane change trajectory prediction model of the autonomous driving vehicle driven by the knowledge data of the present invention, if the number of vehicles around the self-vehicle m is less than 3, in order to ensure the consistency of the characteristics of the surrounding vehicles, 3-m virtual vehicles can be introduced. In the process of setting the virtual vehicles, considering that the length of the collection section is 420m, if the virtual vehicle is located in front of the self-vehicle, the longitudinal distance between the virtual vehicle and the self-vehicle is set to 500m, the vehicle speed is set to 0m / s, and the acceleration is set to 0m / s 2 ; If the virtual car is behind the ego car, the longitudinal distance between the virtual car and the ego car is set to -500m, the speed is set to 0m / s, and the acceleration is set to 0m / s 2 .

[0065] S3 builds a lane-changing trajectory prediction model for autonomous vehicles driven by knowledge and data based on an interactive motion prediction model

[0066] For more details, please refer to the attached Figure 2 The knowledge data-jointly driven lane-changing trajectory prediction model for autonomous driving vehicles established by the present invention includes:

[0067] An input module for inputting the current state quantity of the vehicle and the nearest surrounding vehicles, composed of n driver lane change knowledge K n The lane-changing driving knowledge base is formed, and any lane-changing knowledge K is respectively processed according to the input of the input module and each lane-changing knowledge of the driver in the lane-changing driving knowledge base. iThe invention relates to n joint prediction modules for predicting lane change trajectories of the ego vehicle under the guidance of the present invention, a knowledge summarizing module for summarizing the prediction results of the n joint prediction modules and obtaining a final trajectory prediction result, and an output module for outputting the final trajectory prediction result obtained by the knowledge summarizing module; wherein the joint prediction module is established based on a graph network and a self-attention mechanism, and comprises a dynamic space module, a dynamic time series module and a trajectory prediction module connected in sequence, wherein the dynamic space module can extract the spatial interaction features between the ego vehicle and the nearest surrounding vehicles, including combining the input and any lane change knowledge K i The encoder module performs joint encoding and the decoder module performs decoding. The dynamic timing module can extract the time interaction features of the vehicle and the nearest surrounding vehicles.

[0068] Further, in some more specific implementations, S3 includes:

[0069] S31 constructs the lane-changing driving knowledge base

[0070] The present invention fully considers that in the driver's lane changing knowledge base, the driver's operations before and after lane changing and surrounding scenes should be collected to the maximum extent through expert prior knowledge, and the driver's behavior should be explained and sorted out in a reasonable way. More specifically, based on driving regulations and driver psychology, the present invention constructs a lane changing driving knowledge base containing 9 lane changing knowledge as shown in Table 2, wherein any lane changing knowledge is associated with one or more of the characteristic state quantities, as shown in Table 2.

[0071] Table 2. Lane-changing driving knowledge base and its associated characteristic state quantities

[0072]

[0073] According to the lane-changing trajectory prediction model of the present invention, when changing lanes, the driver will identify the associated characteristic state quantities in the driving scene under the guidance of the lane-changing knowledge and focus on them.

[0074] S32 constructs the dynamic space module

[0075] The dynamic space module of the present invention models the spatial relationship between vehicles into a directed graph structure, and uses a graph neural network to extract the spatial interaction features between vehicles.

[0076] In the directed graph, each vehicle is an independent node vi, which constitutes a node set V, V = {v1, v2, ..., v n}, the spatial interaction relationship between vehicles is edge e i , whose constituent edge set E, Each node records the value of the characteristic state quantity of the vehicle, namely the node feature, which together constitutes the node feature set A. Therefore, the spatial relationship between different vehicles can be abstracted into a simple and efficient non-Euclidean traffic graph G = {V, E, A}, where v i represents the ith independent node, e i represents the i-th edge, whose value is 0 or 1, indicating whether there is an interaction relationship between vehicles, and n represents the total number of vehicles.

[0077] Furthermore, the ego vehicle, the vehicle in front of the current lane, the vehicle in front of the target lane, and the vehicle behind the target lane are each regarded as a node, and it is assumed that the ego vehicle will pay attention to the status of all nearby vehicles, the vehicle behind will actively pay attention to the status of the vehicle in front, and the vehicle in front will not actively pay attention to the status of the vehicle behind. There is a mutual focus between the vehicle in front and the vehicle in front, and between the vehicle behind and the vehicle behind. Then the edge E of the traffic graph is as shown in formula (2):

[0078]

[0079] According to the above traffic map, the present invention constructs a dynamic spatial module containing an encoder layer and a decoder layer based on a graph attention network (GAT), wherein the aggregation update process of the encoder layer and the decoder layer is as follows:

[0080] (1) Solve the attention value between any i-th node and j-th node in the encoder layer and decoder layer respectively, as follows:

[0081] Encoder layer:

[0082] Decoder layer:

[0083] Where: e ij is the attention value between the i-th node and the j-th node; and are the node features of the i-th node and the j-th node respectively; LeakyReLU represents the LeakyReLU activation function; || represents the concatenation operation of the vector; is the weight coefficient to be learned; k ij It represents the interaction characteristics between the i-th node and the j-th node under the guidance of knowledge, as shown in the following formula:

[0084]

[0085] Among them, K s Represents the knowledge matrix corresponding to the s-th lane-changing knowledge.

[0086] The knowledge matrix is ​​formed by assigning weights to different characteristic state quantities according to the lane-changing knowledge. For example, in a specific implementation, the lane-changing knowledge K i The corresponding knowledge matrix is: In all characteristic state quantities, set the lane-changing knowledge K i The corresponding weight value of the associated characteristic state quantity is 5, and the weight value of other characteristic state quantities is 1. The weight matrix of all characteristic state quantities after setting is obtained, and normalized to obtain K i knowledge matrix.

[0087] (2) Solve the attention score between the i-th node and the j-th node as follows:

[0088]

[0089] Where: α ij is the attention score between the i-th node and the j-th node; N i represents the set of all neighbor nodes of the ith node (excluding the ith node itself); k represents any neighbor node of the ith node, and exp(·) represents an exponential function with e as the base.

[0090] (3) Perform operations (1) to (2) on all nodes that have an edge relationship with the i-th node. After the operations are completed, the nodes are aggregated and updated using the following formula:

[0091]

[0092] Where: represents the updated node feature; σ(·) represents the sigmoid activation function; W0 is the weight coefficient to be learned;

[0093] Preferably, the present invention can introduce a multi-head mechanism through GAT, and apply the graph attention mechanism to multiple subspaces in parallel, so that each head learns a different representation, thereby improving the model's ability to capture different attention features. Then, step (3) performs the aggregation update of the nodes through the following formula:

[0094]

[0095] Where H is the number of spatial graph attention heads.

[0096] The encoder structure constructed according to the above formulas (3), (6), and (8) is KGAN ENC According to the above formulas (4), (6), (8), the decoder structure is GAN DEC , then, the extraction process of vehicle space interaction features can be expressed as:

[0097]

[0098] Among them, G1 represents the spatial interaction features after being updated by the encoder layer, and G2 represents the spatial interaction features after being parsed by the decoder layer.

[0099] S33 constructs the dynamic timing module

[0100] For details, please refer to the attached Figure 3 The present invention constructs the dynamic timing module based on the self-attention mechanism, which can capture the motion relationship between the vehicle and the nearest surrounding vehicles, namely the temporal interaction characteristics, in a time series.

[0101] The dynamic timing module includes: an input layer, a position encoding module, a multi-head self-attention layer, a first residual connection and normalization module, a fully connected feedforward neural network layer, a second residual connection and normalization module and an output layer connected in sequence.

[0102] in:

[0103] The input of the input layer is the output result of the dynamic space module constructed in S32. Compared with the original input, the output fuses the spatial interaction information between different features with the help of the dynamic space module.

[0104] The position encoding module can add position encoding to the input to ensure that the time order of information at different times that is simultaneously fed into the dynamic timing module is preserved. The specific position encoding adding method is shown in the following formula:

[0105]

[0106] Among them, pos represents the position of a single node in the time series, d model is the dimension of the input vector, 2l and 2l+1 represent the positions of the 2lth element and 2l+1th element in the input vector.

[0107] Compared with the general self-attention model, the multi-head self-attention layer can pay attention to the global information of the time series more efficiently. It linearly projects the query vector Q, key vector K, and value vector V input with different, learned projection parameter matrices, and then inputs them into the following attention function Attention(Q,K,V) calculation formula:

[0108]

[0109] Among them, d k represents the dimension of the key vector K, and T represents the matrix transpose operation.

[0110] After repeating the above process h times, the output values ​​of each head are concatenated to obtain the final multi-head self-attention output, as follows:

[0111] Multihead(Q,K,V)=Concat(head1,...,head h )W o (12)

[0112] head i =Attention(QW i Q ,KW i K ,VW i V ) (13)

[0113] in, is the parameter matrix that plays a projection role, where R represents the dimensional space, h represents the total number of attention heads, and d v Represents the dimension of the value vector V.

[0114] The fully connected feedforward neural network layer FFN(X) consists of two layers of fully connected neural networks and a linear activation layer in the middle. The linear activation layer uses the ReLU activation function, as shown in formula (14):

[0115]

[0116] Among them W1 FFN , is the weight matrix of the fully connected layer, is the bias vector of the fully connected layer.

[0117] The first residual connection and normalization module has the same structure as the second residual connection and normalization module, both of which include a residual connection layer and a normalization layer, wherein the residual connection layer is used to improve the gradient vanishing problem, as shown in formula (15):

[0118] out res =in res +FFN(in res )(15)

[0119] Among them, res ,out res Represent the input and output of the residual connection layer respectively.

[0120] The normalization layer is used to improve the gradient explosion problem, as shown in formula (16):

[0121]

[0122] Among them, μ, σ 2 Output out for the input residual connection layer res The mean and variance of each layer in , ε is an infinitesimal quantity, γ, β are the parameters to be learned, outnor They represent the output of the normalization layer respectively.

[0123] S34 builds the trajectory prediction module

[0124] More specifically, the trajectory prediction module is composed of a two-layer fully connected neural network Ω, which transforms the output results of the dynamic timing module As input, we obtain the lane-changing knowledge K i The prediction result of the vehicle trajectory under the guidance of L k ,as follows:

[0125]

[0126] in, Indicates that in knowledge K i Under the guidance of , the model predicts the vehicle position at time t.

[0127] S35 builds the knowledge aggregation module

[0128] More specifically, the knowledge aggregation module is composed of a fully connected neural network Ψ, which obtains the prediction result by the trajectory prediction module As input, the vehicle trajectory prediction L under the guidance of all lane change knowledge is obtained as follows:

[0129]

[0130] The above knowledge aggregation module of the present invention can aggregate the vehicle trajectory prediction results under the guidance of different knowledge to obtain the final trajectory prediction result.

[0131] S4: training and adjusting the lane change trajectory prediction model of the autonomous driving vehicle through the model training data set

[0132] In some more specific implementations, during training, the following root mean square error loss function (RMSE) is used to optimize and adjust the parameters of the lane change trajectory prediction model of the autonomous driving vehicle:

[0133]

[0134] Where: N test is the total number of test set samples; T is the number of predicted time domain frames; are the true values ​​of the horizontal and vertical coordinates respectively; are the predicted values ​​of the horizontal and vertical coordinates respectively.

[0135] S5 predicts the future trajectory of the autonomous vehicle through the trained autonomous vehicle lane change trajectory prediction model

[0136] Example 1

[0137] According to the above specific implementation methods, the automatic driving vehicle lane change trajectory prediction model of the present invention is constructed and experimentally tested. The model is built based on the PyTorch framework, the optimizer of the model is adamW, the learning rate is 0.001, the experimental platform is the Windows environment, the GPU model is NVIDIA RTX8000, the CPU model is Intel Xeon Gold 6250@3.9GHz (8 cores, 16 processes), and the memory is 512GB. During the experiment, the data set was randomly divided into a training set, a test set, and a validation set in a ratio of 7:2:1, the training batch was 512, the number of iterations was 80 rounds, and there was no parallel process.

[0138] Some parameters in the model are set as shown in Table 3:

[0139] Table 3 Model parameter settings

[0140]

[0141] A variety of baseline models applied to the HighD dataset are used for comparison, and the root mean square error (RMSE) of the model is used to reflect the prediction performance of the model.

[0142] The specific comparison baseline models include (specific references are shown below):

[0143] (1) Use long short-term memory networks S-LSTM, CS-LSTM, and NLS-LSTM for trajectory prediction;

[0144] (2) Using the self-attention mechanism-based neural network Dual-Transformer, STDAN, EA-Net, and FIHF for trajectory prediction;

[0145] (3) Using a series of graph neural networks and a neural network based on the self-attention mechanism combined with GCN-Transformer for trajectory prediction;

[0146] (4) A parallel graph neural network and self-attention mechanism are used in conjunction with BAT for trajectory prediction.

[0147] The comparative test results are shown in Table 4:

[0148] Table 4 Comparison of prediction performance of different models

[0149]

[0150] It can be seen that the RMSE value of each model increases with the extension of the prediction time domain, which indicates that the longer the prediction time domain, the more movement possibilities the vehicle will present, and the greater the final position deviation of the vehicle will be. The autonomous driving vehicle lane change trajectory prediction model of the present invention has the smallest RMSE value in all prediction time domains, which is 50%, 14.39%, 9.05%, and 14.07% lower than other algorithms. The RMSE value at the 4s prediction time domain value is less than 0.4m, which is lower than the RMSE value of many algorithms at the 3s prediction time domain. This shows that compared with other methods, the autonomous driving vehicle lane change trajectory prediction model of the present invention can more effectively obtain the environmental information around the target vehicle for accurate prediction, and has better results in predicting the overall movement trend of the vehicle, and the generated prediction trajectory has a lower deviation.

[0151] In order to more intuitively understand the effect of cutting-in trajectory prediction, Figure 4-7 The figure further shows the predicted trajectories of the lane-changing trajectory prediction model of the autonomous driving vehicle of the present invention in different driving scenarios. The blue, red, yellow, and purple solid lines represent the historical trajectories of the vehicle and surrounding vehicles, the blue, red, yellow, and purple dotted lines represent the actual trajectories of the vehicle and surrounding vehicles in the next 4 seconds, and the green dotted line represents the predicted trajectory of the vehicle in the next 4 seconds by the lane-changing trajectory prediction model of the autonomous driving vehicle of the present invention.

[0152] in, Figure 4 The trajectory prediction results of the vehicle moving from left to right and merging left are shown. Figure 4 It can be seen that at this time, the vehicle in front of the current lane and the vehicle in front of the target lane have the same longitudinal position, but the longitudinal speed of the vehicle in front of the target lane is faster than that of the vehicle in front of the current lane, and there is no vehicle within 500m behind the target lane. Therefore, the ego vehicle chooses to change lanes and overtake to achieve higher traffic performance: 4 seconds later, the ego vehicle and the vehicle in front of the original lane have the same longitudinal position.

[0153] Figure 5 The trajectory prediction results of the vehicle moving from left to right and merging right are shown. Figure 5 It can be seen that although the speed of the front vehicle in the current lane is slightly faster than that of the front vehicle in the target lane, the rear vehicle in the target lane adjacent to the ego vehicle is a heavy vehicle with a slower speed. Therefore, the ego vehicle will choose to accelerate and change lanes to avoid driving side by side with the heavy vehicle for a long time.

[0154] Figure 6 The trajectory prediction results of the vehicle driving from right to left and merging left are shown. Figure 6It can be seen that at this time, the front car in the target lane has left the current lane, and the distance between the ego vehicle and the front car in the current lane is relatively close, so the ego vehicle changes lanes to select the target lane with a wider passage space. Since the distance between the ego vehicle and the rear car in the target lane is relatively close, the ego vehicle chooses a larger entry angle to change lanes, hoping to overtake the rear car in the target lane.

[0155] Figure 7 The trajectory prediction results of the vehicle driving from right to left and merging right are shown. Figure 7 It can be seen that at this time, the front car in the target lane has left the current lane. Although the front car in the current lane is a certain distance away from the ego vehicle, the traffic capacity is still not as good as the target lane. At this time, the ego vehicle's speed is higher than the rear car in the target lane, and sufficient prerequisites for lane change have been met. Therefore, the ego vehicle decisively chooses to change lanes.

[0156] According to the above prediction results, it can be seen that the lane change trajectory prediction model of the autonomous driving vehicle of the present invention can effectively obtain the potential relationship between the vehicle and the surrounding vehicles, and make reasonable decisions according to the current scenario. The predicted trajectory has a high degree of overlap with the actual trajectory in different scenarios, showing good prediction performance. At the same time, there is no collision risk between the predicted result of the vehicle's trajectory and the actual driving trajectory of the surrounding vehicles, indicating that the model can effectively obtain the interactive relationship between the target vehicle state and the surrounding environment, thereby improving the prediction accuracy and the model safety.

[0157] The references in Table 4 are as follows:

[0158] [1]Alahi A, Goel K, Ramanathan V, et al.Social LSTM: Human Trajectory Prediction in Crowded Spaces[J].Proceedings of the IEEE / CVF conference oncomputer vision and pattern recognition.2016.

[0159] [2]Deo N,Trivedi M M.Convolutional social pooling for vehicle trajectory prediction[C].Proceedings of the IEEE conference on computer visionandpattern recognition workshops.2018:1468-1476.

[0160] [3]K.Messaoud,I.Yahiaoui,A.Verroust-Blondet,et al.Non-local SocialPooling for Vehicle Trajectory Prediction[C].2019IEEE Intelligent VehiclesSymposium(IV),2019:975-980.

[0161] [4]K.Gao,et al.Dual Transformer Based Prediction for Lane ChangeIntentions and Trajectories in Mixed Traffic Environment[J].IEEE Transactionson Intelligent Transportation Systems,2023,24(06):6203-6216

[0162] [5]X.Chen,H.Zhang,F.Zhao,et al.Intention-Aware Vehicle TrajectoryPrediction Based on Spatial-Temporal Dynamic AttentionNetwork for Internet ofVehicles[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(10):19471-19483.

[0163] [6]Y.Cai et al.Environment-AttentionNetwork for Vehicle TrajectoryPrediction[J].IEEE Transactions on Vehicular Technology.2021,70(11):11216-11227.

[0164] [7]Zuo Z, Wang X, Guo S, et al. Trajectory prediction network of autonomous vehicles with fusion of historical interactive features [J]. IEEE Transactions on Intelligent Vehicles, 2023.

[0165] [8] Han Tianli, Ma Chi, Hu Linzhi. Vehicle lane-changing behavior modeling and trajectory prediction method based on GCN-Transformer[J]. Modeling and Simulation, 2024, 13(3): 2754-2771.

[0166] Han TL,Ma C,Hu L ZA Vehicle Lane-Changing Behavior Modeling and Trajectory Prediction MethodBased on GCN-Transformer[J].Modeling andSimulation,2024,13(3):2754-2771.

[0167] [9] Liao H, Li Z, Shen H, et al. Bat: Behavior-aware human-like trajectory prediction for autonomous driving [C]. Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(9): 10332-10340.

[0168] The above embodiments are only preferred implementations of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A knowledge-data-co-driven lane-changing trajectory prediction method for an autonomous driving vehicle, characterized in that: The lane change trajectory prediction model of the autonomous driving vehicle driven by the trained knowledge data is used for trajectory prediction, wherein the lane change trajectory prediction model of the autonomous driving vehicle driven by the knowledge data, i.e., the lane change trajectory prediction model, comprises: an input module for inputting the current characteristic state quantity of the vehicle and its nearest surrounding vehicles, a lane change trajectory prediction model composed of n driver lane change knowledge K n The constructed lane-changing driving knowledge base performs any lane-changing knowledge K according to the input of the input module and each lane-changing knowledge of the driver in the lane-changing driving knowledge base. i n joint prediction modules for predicting the lane change trajectory of the ego vehicle under the guidance of the present invention, a knowledge aggregation module for aggregating the prediction results of the n joint prediction modules and obtaining the final trajectory prediction result, and an output module for outputting the final trajectory prediction result obtained by the knowledge aggregation module; wherein the joint prediction module includes a dynamic space module, a dynamic time series module and a trajectory prediction module connected in sequence; wherein the dynamic space module can extract the spatial interaction features between the ego vehicle and the nearest surrounding vehicles, which is constructed based on a multi-head graph attention network, including combining the input and any lane change knowledge K i An encoder layer for joint encoding and a decoder layer for decoding; the dynamic timing module can extract the temporal interaction features between the vehicle and the nearest surrounding vehicles, which is constructed based on a multi-head self-attention mechanism.

2. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 1, characterized in that: The nearest surrounding vehicles include the vehicle in front of the current lane that is in the same lane as the vehicle and is at the closest distance in front of the vehicle, the vehicle in the target lane that is in the adjacent lane of the vehicle's lane, with its front end behind the rear end of the vehicle and the closest distance to the vehicle, and the vehicle in front of the target lane that is in the adjacent lane of the vehicle's lane, with its rear end in front of the front end of the vehicle and the closest distance to the vehicle; the characteristic state quantities include the type, position, speed, acceleration, spatial distance and time distance between the vehicle and the nearest surrounding vehicles.

3. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 1, characterized in that: The lane-changing driving knowledge base includes lane-changing knowledge including the driver's lane-changing tendency.

4. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 1, characterized in that: The construction of the dynamic space module includes: Each vehicle is regarded as an independent node v i , the spatial interaction relationship between vehicles is edge e i , the characteristic state value of the vehicle recorded by the node is the node characteristic composition traffic graph G = {V, E, A}, where v i represents the ith independent node, e i represents the i-th edge, whose value is 0 or 1, indicating whether there is an interaction relationship between vehicles, V = {v1,v2,...,v n } represents a collection of independent nodes, represents the edge set, n represents the total number of vehicles; The interactive relationship between the self-vehicle and the nearest surrounding vehicles is set as follows: the self-vehicle will pay attention to the status of all the nearest surrounding vehicles, the rear vehicle will actively pay attention to the status of the front vehicle, the front vehicle will not actively pay attention to the status of the rear vehicle, and there is a mutual focus state between the front vehicle and the front vehicle and between the rear vehicles, then the edge E is as shown in the following formula: Based on the traffic graph G={V, E, A}, spatial interaction feature extraction is performed through the encoder layer and the decoder layer as follows: Among them, G1 represents the spatial interaction features after being updated by the encoder layer, G2 represents the spatial interaction features after being parsed by the decoder layer; A represents the set of node features of all nodes; KGAN ENC represents encoder calculation, GAN DEC represents the decoder calculation; Wherein, the encoder calculation and the decoder calculation include: (1) Solve the attention value between any i-th node and j-th node in the encoder layer and decoder layer respectively, as follows: Encoder layer: Decoder layer: Among them, e ij is the attention value between the i-th node and the j-th node; and are the node features of the i-th node and the j-th node respectively; LeakyReLU represents the LeakyReLU activation function; || represents the concatenation operation of the vector; is the weight coefficient to be learned; k ij It represents the interaction characteristics between the i-th node and the j-th node under the guidance of knowledge, as shown in the following formula: Among them, K s The knowledge matrix corresponding to the lane-changing knowledge K is represented by s The weights of different characteristic state quantities are assigned; (2) Solve the attention score between the i-th node and the j-th node as follows: Where: α ij is the attention score between the i-th node and the j-th node; N i represents the set of all neighbor nodes of the ith node; k represents any neighbor node of the ith node, and exp(·) represents an exponential function with e as the base; (3) Perform operations (1) to (2) on all nodes that have an edge relationship with the i-th node. After the operations are completed, the nodes are aggregated and updated using the following formula: in, represents the updated node feature; σ(·) represents the sigmoid activation function; W0 is the weight coefficient to be learned, and H is the number of spatial graph attention heads.

5. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 1, characterized in that: The dynamic time series module includes: an input layer, a position encoding module, a multi-head self-attention layer, a first residual connection and normalization module, a fully connected feedforward neural network layer, a second residual connection and normalization module and an output layer, wherein the position encoding module can add position encoding to the input as follows: Among them, pos represents the position of a single node in the time series, d model is the dimension of the input vector, 2l and 2l+1 represent the positions of the 2lth element and 2l+1th element in the input vector.

6. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 5, characterized in that: The output of the multi-head self-attention layer is: Multihead(Q,K,V)=Concat(head1,...,head h )W o Among them, Q, K, and V represent the query vector Q, the key vector K, and the value vector V respectively, Contact() represents the Contact function, and head i represents the i-th attention head, is the parameter matrix that plays a projection role, where R represents the dimensional space, h represents the total number of attention heads, and d v Represents the dimension of the value vector V, d k represents the dimension of the key vector K, and Attention() represents the following attention function: Wherein, T represents the transpose operation.

7. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 5, characterized in that: in, The fully connected feedforward neural network layer includes two layers of fully connected neural networks and a linear activation layer in between; and / or, the first residual connection and normalization module and the second residual connection and normalization module both include a residual connection layer and a normalization layer.

8. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 1, characterized in that: The trajectory prediction module is formed by a two-layer fully connected neural network.

9. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 1, characterized in that: The knowledge aggregation module is formed by a layer of fully connected neural network.

10. The method for predicting lane change trajectories of an autonomous driving vehicle according to claim 1, characterized in that: The training includes optimizing and adjusting the lane change trajectory prediction model according to a root mean square error loss function.

Citation Information

Patent Citations

  • Expressway automatic driving commercial vehicle trajectory prediction and decision-making method and system based on complex network theory

    CN116050245A

  • Reactive trajectory prediction method and system for autonomous vehicle

    CN116258242A

  • Intelligent vehicle lane changing decision-making system and method based on BEV visual angle and digital twinning

    CN116653953A

  • Vehicle track prediction model construction method combining data driving and knowledge guiding

    CN117141517A

  • Vehicle track prediction method based on intention perception space-time attention network

    CN117141518A

Cited By

  • Automatic driving track prediction method based on driving style

    CN119577968A

  • Automatic driving decision optimization system and method based on data driving

    CN119882431A

  • Vehicle lane changing track prediction method, device and system

    CN120828828A