Vehicle trajectory prediction method and system based on decoupled graph diffusion neural network

Abstract

The present invention discloses a vehicle trajectory prediction method and system based on a decoupled graph diffusion neural network, and relates to the technical field of vehicle trajectory prediction; the method comprises: obtaining driving process information in real-time vehicle data, and constructing a vehicle trajectory graph according to the driving process information; processing the vehicle trajectory graph based on a sparse high-dimensional representation method to obtain an initial graph; inputting the initial graph into a pre-trained trajectory prediction model to obtain the vehicle prediction trajectory output by the trajectory prediction model; wherein the trajectory prediction model is a decoupled graph diffusion neural network; the trajectory prediction model uses generalized graph diffusion and multi-head attention mechanisms to capture the global information of the initial graph, and uses a hierarchical decoupling method to represent the learning process. The present invention uses a decoupled graph diffusion neural network to improve the accuracy of trajectory prediction, can more accurately grasp the vehicle's driving path, and help reduce traffic congestion and improve urban traffic conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle trajectory prediction, and particularly to a vehicle trajectory prediction method and system based on a decoupled graph diffusion neural network. Background Art

[0002] With the acceleration of the urbanization process, the number of vehicles in the city is constantly increasing, road congestion and traffic accidents occur frequently, and the problem of path prediction of motor vehicles has become increasingly important; accurately predicting the driving path of motor vehicles helps to better plan and manage traffic, improve road utilization rate and reduce traffic congestion; and realizing path prediction is also an important prerequisite for autonomous driving technology, which can help autonomous vehicles better perceive the surrounding environment, make more accurate decisions, and improve the safety and reliability of autonomous driving.

[0003] Path prediction collects and analyzes vehicle driving data, and uses advanced algorithms and models to predict the possible driving trajectories of vehicles in the future. In recent years, the development of deep learning technology has provided new ideas and methods for vehicle path prediction; however, in related technologies, the application of graph neural networks still has the following problems:

[0004] (1) The data structure of the graph is discrete, consisting of nodes and edges; this increases the complexity of training and optimizing the graph generation model (such as the calculation of gradients), making it difficult for widely used optimization algorithms to be directly applied to backpropagation training.

[0005] (2) There may be some information features that cannot be effectively considered, and it relies on neighbor nodes to update the hidden state of the current node, with low efficiency. Summary of the Invention

[0006] To solve the problems existing in related technologies, the present invention provides a vehicle trajectory prediction method and system based on a decoupled graph diffusion neural network, which can more comprehensively obtain information features and improve the accuracy of vehicle trajectory prediction.

[0007] To achieve the above object, the present invention provides a vehicle trajectory prediction method based on a decoupled graph diffusion neural network, including:

[0008] Obtain the driving process information in the vehicle real-time data, and construct a vehicle trajectory graph according to the driving process information;

[0009] Process the vehicle trajectory graph based on a sparse high-dimensional representation method to obtain an initial graph;

[0010] Input the initial graph into a pre-trained trajectory prediction model, and obtain the vehicle prediction trajectory output by the trajectory prediction model;

[0011] Among them, the trajectory prediction model is a decoupled graph diffusion neural network; the trajectory prediction model uses generalized graph diffusion and multi-head attention mechanism to capture the global information of the initial graph, and adopts a hierarchical decoupling method to represent the learning process.

[0012] Optionally, processing the vehicle trajectory graph based on a sparse high-dimensional representation method includes:

[0013] Classifying the node data in the vehicle trajectory graph according to attributes;

[0014] Discretizing the classified node data and constructing one-hot vectors for each attribute;

[0015] Constructing a mapping relationship between each one-hot vector and the corresponding high-dimensional real-valued embedding vector.

[0016] Optionally, the driving process information includes the angle relative to the road, the distance from the adjacent left lane mark, the distance from the adjacent right lane mark, and the distance from the obstacle ahead; constructing a vehicle trajectory graph according to the driving process information includes:

[0017] Obtaining the historical driving trajectory of the vehicle according to the driving process information;

[0018] Uniformly sampling the historical driving trajectory of the vehicle to obtain historical trajectory nodes of the vehicle;

[0019] Constructing a vehicle trajectory graph based on the historical trajectory nodes of the vehicle.

[0020] Optionally, the edge information transfer process between two adjacent trajectory nodes v i to v j in the initial structure diagram is represented by A i,j The expression of A i,j is as follows:

[0021]

[0022] In the formula, E() represents signal energy; S() represents information entropy; x i is the feature vector of the i-th trajectory node v i ; x j is the feature vector of the j-th trajectory node v j ;

[0023] The signal energy of the trajectory node v i is calculated using the following formula:

[0024]

[0025] Wherein, M represents the number of trajectory node features; τ represents the trajectory time length of the historical backtracking window; n represents the sequence length of x i ;

[0026] The information entropy of the trajectory node v i is calculated using the following formula:

[0027]

[0028] Wherein, x i 's non-repeating sequence is represented as {s 0 ,..., s j}; p(s j ) represents the probability of the value s j ; s j is the state representation of x j ;

[0029]

[0030] Wherein, δ() represents the Dirac delta function.

[0031] Optionally, when the trajectory prediction model performs the generalized graph diffusion operation, it uses the following formula:

[0032]

[0033] Wherein, Q l represents the diffusion matrix of the l-th layer of the neural network layer; k represents the maximum diffusion step; θ l,k represents the weight coefficient; T l,k represents the column stochastic transition matrix.

[0034] Optionally, the trajectory prediction model represents the learning process using a hierarchical decoupling method, including:

[0035] Adopting a hierarchical decoupling representation learning strategy to capture the hierarchical features of each hidden layer; each hidden layer includes a parallel generalized graph diffusion layer and an attention layer; the update rule of the hidden layer uses the following formula:

[0036]

[0037] Wherein, H' l represents the node representation of the l-th layer; σ() represents the activation function; ζ() represents the multi-head attention; || represents concatenation; Q l represents the diffusion matrix of the l-th layer of the neural network layer; W l represents the weight matrix for hierarchical training; represents the weight matrix of the diffusion process; represents the weight matrix of the l-th layer node update process; H lDenote the potential diffusion representation; A t Denote the adjacency matrix of all nodes; ⊙ is the Hadamard product, representing element-wise multiplication; Denote the bias term vector of the l-th layer.

[0038] Optionally, the training method of the trajectory prediction model includes:

[0039] Input the training data into the trajectory prediction model;

[0040] The trajectory prediction model outputs the predicted value through forward propagation of the features of the training data via the decoupled graph diffusion neural network;

[0041] The trajectory prediction model obtains the loss value by calculating the cross-entropy function between the actual output probability and the expected output probability, and then updates the weight parameters using backpropagation;

[0042] The trajectory prediction model iteratively updates the weight parameters until the preset condition is satisfied between the predicted value and the true value.

[0043] Optionally, the trajectory prediction model includes: a generalized graph diffusion layer, a multi-head attention layer, and a graph neural network;

[0044] A hierarchical decoupling method is used between the generalized graph diffusion layer and the multi-head attention layer to capture the hierarchical features and relationships at different levels among the trajectory nodes in the initial graph; the generalized graph diffusion layer uses generalized graph diffusion to capture the global information of the initial graph; the multi-head attention layer takes the output of the generalized graph diffusion layer as input to capture the global correlation of the connected nodes and the global dependence of the unconnected nodes in the initial graph; the graph neural network takes the output of the multi-head attention layer as input to aggregate the features from the vicinity of the trajectory nodes.

[0045] Optionally, the method further includes:

[0046] Render the vehicle prediction trajectory and display it on the UI interface.

[0047] The present invention also provides a vehicle trajectory prediction system based on a decoupled graph diffusion neural network, including:

[0048] A trajectory graph acquisition module, configured to acquire the driving process information in the vehicle real-time data and construct a vehicle trajectory graph according to the driving process information;

[0049] An initial graph acquisition module, configured to process the vehicle trajectory graph based on a sparse high-dimensional representation method to obtain an initial graph;

[0050] A prediction module, configured to input the initial graph into a pre-trained trajectory prediction model to obtain the vehicle prediction trajectory output by the trajectory prediction model;

[0051] Among them, the trajectory prediction model is a decoupled graph diffusion neural network; the trajectory prediction model uses generalized graph diffusion and multi-head attention mechanism to capture the global information of the initial graph, and adopts a hierarchical decoupling method to represent the learning process.

[0052] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0053] The present invention provides a vehicle trajectory prediction method based on a decoupled graph diffusion neural network. By using a sparse high-dimensional representation method to process vehicle trajectory data (vehicle trajectory graph), it is possible to expand the feature space to a higher dimension on the premise of given relatively low-dimensional observed vehicle real-time data, so as to avoid the difficulty of encoding complex motions in a low-dimensional space; it gets rid of the constraint of expressing complex motion patterns in a low-dimensional space, and provides a solid theoretical basis and technical support for the accuracy and reliability of vehicle trajectory prediction.

[0054] The trajectory prediction model adopted by the present invention when performing trajectory prediction uses generalized graph diffusion and multi-head attention mechanism to capture the global information of the initial graph; through generalized graph diffusion, information can be diffused in a larger neighborhood, so that the relationships and global information in the graph can be better captured, which helps to have a deeper understanding of the structure of the graph.

[0055] In the related art, GNN obtains the final node information representation by continuously aggregating and updating node information; however, this may lead to a high degree of distortion of the node feature representation; the trajectory prediction model applied in the present invention adopts a hierarchical decoupled representation learning strategy, and each layer is constructed as a parallel combination of a generalized graph diffusion layer and an attention layer to capture the hierarchical features inside each trajectory node, which can effectively prevent the loss of unique local information in the learned representation, helps to aggregate global information, and avoids the omission of potential information. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] By describing the exemplary embodiments of the present invention in more detail in conjunction with the drawings, the above and other objects, features and advantages of the present invention will become more obvious. Among them, in the exemplary embodiments of the present invention, the same reference numerals generally represent the same components.

[0057] Figure 1 It is a schematic flowchart of the method for the vehicle trajectory prediction method based on the decoupled graph diffusion neural network shown in the embodiments of the present invention;

[0058] Figure 2 It is a schematic diagram of the logical structure of the vehicle trajectory prediction method shown in the embodiments of the present invention Figure 1 ;

[0059] Figure 3Schematic diagram of the logical structure of the vehicle trajectory prediction method shown in the embodiments of the present invention Figure 2 ;

[0060] Figure 4 Schematic diagram of the hierarchical decoupled representation learning method shown in the embodiments of the present invention;

[0061] Figure 5 Data representation diagram of representing vehicle trajectories using high-dimensional embedding vectors shown in the embodiments of the present invention;

[0062] Figure 6 Schematic diagram of constructing four heat vectors and a multi-resolution loss function vector shown in the embodiments of the present invention;

[0063] Figure 7 Schematic diagram of the decoupled graph diffusion neural network model shown in the embodiments of the present invention;

[0064] Figure 8 Diagram showing the prediction results of the trajectories of autonomous vehicles shown in the embodiments of the present invention;

[0065] Figure 9 Schematic diagram of the module structure of the vehicle trajectory prediction system based on the decoupled graph diffusion neural network shown in the embodiments of the present invention;

[0066] Figure 10 Schematic diagram of the structure of the electronic device shown in the embodiments of the present invention. Detailed implementation manners

[0067] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0068] Please refer to Figure 2 and Figure 3 , the present invention collects real-time data of the vehicle sensing system, extracts vehicle driving information, and creates a vehicle trajectory map; then performs sparse high-dimensional representation on the node data, establishes a mapping relationship with high-dimensional real-valued embedding vectors, and constructs an initial graph using it as node information; afterwards, the processed node and edge information are input into the prediction model, the optimal graph topology structure is learned through generalized graph diffusion, the connections between nodes are gradually established, the hierarchical decoupled method is used to represent the learning process, and the optimal objective function is obtained by minimizing the risk function, so that the model is optimized and learned, and finally the predicted trajectory is output.

[0069] The present invention provides a vehicle trajectory prediction method and system based on a decoupled graph diffusion neural network, which adopts a DGDNN model. This model is a decoupled graph diffusion neural network. From the perspective of information transmission, a graph structure is constructed by using signal energy as a quantization index and information entropy as the directional connectivity. The application of the decoupled graph diffusion neural network can improve the accuracy of trajectory prediction, enable a more accurate grasp of the vehicle's driving path, contribute to reducing traffic congestion, optimizing road utilization, thereby improving the urban traffic conditions and bringing a more convenient and efficient travel experience to citizens.

[0070] Please refer to Figure 1 , the above vehicle trajectory prediction method based on a decoupled graph diffusion neural network may include the following steps:

[0071] Step 101: Obtain the driving process information in the vehicle real-time data, and construct a vehicle trajectory graph according to the driving process information.

[0072] In application, the vehicle can be equipped with a data acquisition device to collect driving trajectory data in real time, that is, obtain real-time data; the real-time data may exemplarily include: images collected by a camera sensor, point clouds collected by a lidar sensor, point clouds collected by a millimeter-wave radar sensor, etc.; by setting the data acquisition device, the motion data and original road data of the vehicle can be obtained in real time.

[0073] In one embodiment, the driving process information includes the angle relative to the road, the distance from the adjacent left lane mark, the distance from the adjacent right lane mark, and the distance from the obstacle in front; wherein, the obstacles include: vehicles, pedestrians, cheval de frise, guardrails, etc.; the above constructing a vehicle trajectory graph according to the driving process information includes:

[0074] According to the driving process information, obtain the vehicle historical driving trajectory;

[0075] Perform uniform sampling on the vehicle historical driving trajectory to obtain vehicle historical trajectory nodes;

[0076] Based on the vehicle historical trajectory nodes, construct a vehicle trajectory graph.

[0077] In application, the vehicle trajectory segment can be uniformly sampled at a time interval t to obtain vehicle historical trajectory nodes v i , and four types of information including the angle of the vehicle relative to the road, the distance to the adjacent two lanes (left lane and right lane) marks, and the distance from the vehicle to the vehicle in front are obtained through data processing as the feature information of the trajectory nodes.

[0078] Edge feature information: Information quantity is used as a measurement criterion. Information quantity refers to the information brought by a node. Information entropy is the expectation of the possible information quantity before the result comes out, that is, considering all possible values of the random variable, that is, the expectation of the information quantity brought by all possible events. Information entropy is used as the quantification of connectivity between two trajectory points, signal energy is used as its strength, and the information quantity is used to measure the quantified information brought by a trajectory node. A weight matrix is associated with each edge to represent the information transfer relationship between different nodes. Thus, A i,j represents the edge information transfer relationship from v i to v j .

[0079] Since the energy of the signal reflects its strength during propagation, which may affect the information received at the receiving end, information entropy is used as the directional connectivity between two adjacent trajectory points, signal energy is used as its strength, and the link between nodes is quantified; its adjacency matrix can be represented as A

[0080] In the initial structure diagram, the edge information transfer process between two adjacent trajectory nodes v i to v j can be represented by the above A i,j ; The expression of A i,j is as follows:

[0081]

[0082] In the formula, E() represents the signal energy; S() represents the information entropy;

[0083] The signal energy of the trajectory node v i is calculated using the following formula:

[0084]

[0085] In the formula, M represents the number of trajectory point features; τ represents the trajectory time length of the historical backtracking window;

[0086] The information entropy of the trajectory node v i is calculated using the following formula:

[0087]

[0088] In the formula, the non-repeating sequence of x i is {s 0 ,..., s j}; p(s j ) represents the probability of the value s j ;

[0089]

[0090] Where δ() represents the Dirac delta function.

[0091] It should be noted that since the collected real-time data and data sets are being used to predict the future trend of the current vehicle's trajectory, it is necessary to convert the regression task of predicting the precise trajectory route into a time node classification task. Given a path trajectory over a period of time τ in the past, let the model learn from the historical backtracking window of length τ and predict their probability in the next period of time. The mapping relationship of the prediction work is expressed as follows:

[0092]

[0093] Where, f() represents the DGDNN model; E t represents an edge set; represents a node set; G t represents the graph structure at time t; C t+1 indicates the prediction result at time t+1.

[0094] Step 102: Process the vehicle trajectory graph based on a sparse high-dimensional representation method to obtain an initial graph.

[0095] In the application, sparse high-dimensional representation methods can be used to process node data, including constructing high-dimensional real-valued embedding vectors e att 's mapping relationship; by Display vehicle tracks, using (A t ) i,j represents edge information and constructs the initial graph. e 0:T indicates the vehicle trajectory.

[0096] The outliers and missing data that the data usually contains will affect the prediction; in the training phase, the presence of outliers and missing data introduces additional noise and uncertainty, which may affect the convergence of the learning process; in the evaluation phase, missing data will prevent the calculation of prediction errors, and outliers will lead to inaccurate evaluation of prediction accuracy. In order to mitigate the impact of outliers and missing data, the vehicle trajectory map can be preprocessed first; among which, data preprocessing can exemplarily include: data cleaning, feature selection and scaling, data set construction, encoding construction, data normalization, etc.

[0097] 1) Data cleaning: Clean the noise, outliers and inconsistencies in the data; specifically, remove the vehicle position information with unrealistic speed values; remove the vehicle position information with unchanged position; remove the vehicle position observations that are too close to the starting point; divide the long journey into shorter journeys with a maximum sequence length of 10 minutes.

[0098] 2) Feature Selection and Scaling: In the initial stage of data preprocessing, selecting relevant features and scaling them proportionally helps eliminate redundancy and maintain data consistency. Feature selection aims to identify and retain only the most relevant attributes while discarding irrelevant or redundant ones, ensuring that all features are within a consistent range, thus facilitating more efficient and effective data analysis.

[0099] 3) Dataset Partitioning: The dataset used for analysis is carefully partitioned into three distinct subsets: the training set, the validation set, and the test set. The training set is the largest portion of the data and is used for model construction and parameter estimation. The validation set, although smaller, plays a crucial role in fine-tuning the model and preventing overfitting. Finally, the test set serves as an independent evaluation tool, enabling researchers to assess the model's generalization ability and predictive power.

[0100] 4) Data Encoding: The process of data encoding is to systematically encode the collected and preprocessed data to make model calculations easier and more efficient. This stage involves mapping the original data to a set of predefined categories or numerical values so that the data can be processed by machines. The goal of data encoding is to convert the data into a format that is machine-readable and conducive to statistical analysis.

[0101] 5) Data Normalization: Data normalization is a crucial step in data preprocessing, aiming to ensure the consistency of the numerical ranges of all features. This technique rescales the data to a common range, between 0 and 1. By this normalization, problems of different scales and ranges between different features can be alleviated. By normalizing the data, it is ensured that the contributions of all features to the model are equal, and the results will not be distorted by some dominant features with large numerical values.

[0102] In one embodiment, processing the vehicle trajectory map based on the sparse high-dimensional representation method includes:

[0103] Classifying the node data in the vehicle trajectory map according to attributes;

[0104] Discretizing the classified node data and constructing one-hot vectors for each attribute;

[0105] Constructing a mapping relationship between each one-hot vector and the corresponding high-dimensional real-valued embedding vector.

[0106] Please participate in Figure 4 、 Figure 5 and Figure 6 , in the application, one-hot encoding can be constructed for the node data to achieve discretization, construct four-hot vectors, and establish a mapping relationship with the high-dimensional embedding vectors; for the observed values of each trajectory data, after classifying according to attributes and discretizing, construct a one-hot vector h att for each attribute, and the concatenation of the four one-hot vectors forms the vector ht ; For each h att and the high-dimensional real-valued embedding vector e att construct a mapping relationship, and use the embedding vector e t to represent the set of e att , where e t represents the high-dimensional embedding vector of x t .

[0107] Discretization of node data: The observed values of each trajectory data are discretized after being classified according to attributes. To represent the size relationship inside the data, ranks are used to replace the original data for processing. In this processing process, through feature transformation into a numerical input acceptable to machine learning algorithms, the feature space is effectively expanded to represent more comprehensive and accurate feature information.

[0108] Construct one-hot encoding: Construct a one-hot vector h att for each attribute, that is, by using an N-bit status register to encode N states, each state has its own independent register bit, and at any time, only one of them is valid.

[0109] Construct a four-hot vector: Connect four one-hot encodings in sequence to form a long vector composed of four one-hot vectors, and the connection of the four one-hot vectors forms the vector h t .

[0110] Establish a mapping relationship with the high-dimensional embedding vector: For each h att and the high-dimensional real-valued embedding vector e att construct a mapping relationship, and use the embedding vector e t to represent the set of e att , then e t represents the high-dimensional embedding vector of x t ; that is, a high-dimensional one-hot vector representing a real value is mapped to a low-dimensional space through a linear transformation, and the transformed low-dimensional vector representing a real value embeds the high-dimensional representation of the item into the low-dimensional space. The i-th column of the matrix for the linear transformation is the "embedding vector" of the i-th real value. For each h att and the high-dimensional real-valued embedding vector e att construct a mapping relationship, and effectively adjust the mapping by imposing this sparsity constraint, and overfitting can be avoided when expanding the original 4D observed values to a higher-dimensional space.

[0111] Step 103: Input the initial graph into the pre-trained trajectory prediction model to obtain the vehicle prediction trajectory output by the trajectory prediction model;

[0112] Among them, the trajectory prediction model is a decoupled graph diffusion neural network; the trajectory prediction model uses generalized graph diffusion and multi-head attention mechanism to capture the global information of the initial graph, and adopts a hierarchical decoupling method to represent the learning process.

[0113] In the application, the processed node and edge information can be input into the prediction model, and the optimal graph topology of the task can be learned through generalized graph diffusion combined with the multi-head attention mechanism, so as to gradually establish the connection between each node. The hierarchical decoupling method is used to represent the learning process, and the best model performance is obtained by minimizing the optimal objective function, and finally the predicted trajectory is output.

[0114] As Figure 7 shown, the figure is a vehicle trajectory prediction model established by using DGDNN. This model uses the diffusion and propagation method of DGDNN (Decoupled Graph Diffusion Neural Network) to perform hierarchical decoupling, and establishes a prediction model and learns the complex relationship between vehicle points to obtain the global and local correlations of the vehicle.

[0115] In one embodiment, the trajectory prediction model includes: a generalized graph diffusion layer, a multi-head attention layer, and a graph neural network;

[0116] A hierarchical decoupling method is used between the generalized graph diffusion layer and the multi-head attention layer to capture the hierarchical features and relationships at different levels between the trajectory nodes in the initial graph; the generalized graph diffusion layer uses generalized graph diffusion to capture the global information of the initial graph; the multi-head attention layer takes the output of the generalized graph diffusion layer as input to capture the global correlation of the connected nodes and the global dependence of the non-connected nodes in the initial graph; the graph neural network takes the output of the multi-head attention layer as input and is used to aggregate the features from the vicinity of the trajectory nodes.

[0117] In the application, the trajectory prediction model mainly includes a generalized graph diffusion layer, a multi-head attention layer, and a graph neural network; a hierarchical decoupling method is used between the graph diffusion and the multi-head attention, so that the model can gradually capture the features and relationships at different levels, and thus learn a more accurate and comprehensive graph topology. The generalized graph diffusion layer can capture the complex relationships and global information in the graph through global diffusion, learn the optimal edges of the task, and adaptively propagate different types of information. The multi-head attention layer introduces the multi-head attention mechanism after graph diffusion. The multi-head attention layer takes the output of the generalized graph diffusion layer as input; the purpose is to better capture the global correlation of the connected nodes and the global dependence of the non-connected nodes.

[0118] Graph neural networks can aggregate features from neighboring nodes to propagate information. Each neuron receives the output of the neurons in the previous layer, processes it through a weighted sum, and then passes it through a softmax activation function. This enables it to handle non-linear problems and establish a mapping relationship between inputs and outputs by learning appropriate weights and biases.

[0119] In the trajectory prediction model, each hidden layer consists of a generalized graph diffusion layer and an attention layer. The augmented graph, as a global view, better aggregates global features, can more effectively capture long-range relationships in the graph. At the same time, an attention mechanism is introduced, and the multi-head attention mechanism can extract global features at different stages for layer-by-layer update. The loss function is used to calculate the loss value, which is the similarity between the actual output probability and the expected output probability. The weight parameters are updated through backpropagation to reduce the loss between the true value and the predicted value for optimization until the optimal result is obtained. After learning, training, and optimization, the trajectory prediction model can be used for predicting driving behaviors.

[0120] The trajectory prediction model adopted in the present invention is a prediction model based on graph neural networks, and its specific structure is as follows:

[0121] Given a graph G, the aggregation and update process of neighbor node features can be represented as follows:

[0122]

[0123] Among them, N(v) = {u ∈ V|(u, v) ∈ E} represents the neighbor nodes of node v; represents the neighbor information aggregation function; represents the node update function. represents the node representation of neighbor node v at layer; represents the update function; represents the node representation of node v at layer.

[0124] Ordinary graph convolutional networks only aggregate features from neighboring nodes to propagate information, but this may ignore the potential incompatibility between the provided graph and the task objective, and some of the edges being irrelevant to the task will lead to performance degradation. To solve such problems, the present invention captures complex relationships and global information in the graph through global diffusion, and the learned task-optimal edges adaptively propagate different types of information. Specifically, the generalized aggregation and update process of nodes can be represented by the following formula:

[0125]

[0126] Among them, Conv2d 1×1 () represents a 2D convolutional layer with a 1x1 kernel; Denote the function represented by stacking all relationship nodes; S l,r Denote the diffusion matrix; Denote the weight matrix during the diffusion process; H l-1 Denote the latent diffusion of the l - 1 layer; H l Denote the latent diffusion representation.

[0127] Capture various complex relationships and global information in the graph through the generalized graph diffusion technology to deeply learn the optimal graph topology of the task. Each node can receive information from its neighbor nodes or non - neighbor nodes and update its own state, capturing and propagating information in each layer. The specific representation method of the diffusion matrix is as follows:

[0128]

[0129] In the formula, Q l Denote the diffusion matrix of the l - th neural network layer; K denotes the maximum diffusion step; k denotes the diffusion index at the k - th step; θ l,k Denote the weight coefficient; T l,k Denote the column - stochastic transition matrix. Among them, θ l,k As a trainable parameter; T l,k As a trainable matrix; k as a hyperparameter.

[0130] Furthermore, by defining the neighborhood radius to control the effectiveness of the generalized graph diffusion, which represents the set of all nodes at r steps away from the central node, redundant and ineffective propagation can be avoided. Define the neighborhood radius r l at the L - th layer with the following expression:

[0131]

[0132] In the formula, K is the maximum diffusion step, as a hyperparameter, representing the diffusion range; Θ l,k Denote the weight coefficient.

[0133] Capture global and local correlations further by combining the multi - head attention module and graph diffusion. Introduce the multi - head self - attention mechanism, aiming to better capture the global correlation of connected nodes and the global dependence of non - connected nodes. For a molecular graph with N nodes, the input of the attention mechanism f att consists of three components, respectively called query Q ∈ R k×dk , key K ∈ R N×dk and value V ∈ R N×dv , where Q, K, V are linear projections of the input sequence (for the first layer, it is e t, for other layers, it is the output sequence of the previous layer). Here, dk and dv represent dimensions. To extend the attention to multiple heads, linear projections of Q, K, and V are further utilized to generate M representation subspaces. At each layer, the input sequence is projected into a new space V, and the output of the attention block is the weighted sum in V, where the weights represent the relative contributions at each time step. The projection operators of Q, K, and V are learned during the training phase, and the calculations are performed in parallel.

[0134] The multi-head attention mechanism can learn different attentions between queries and keys, thus improving the model's expressive power. The formula for multi-head self-attention is:

[0135]

[0136] where, W O ∈R Mdh×do is the linear projection matrix for learning the subspace representation that shrinks the output of the head to the output dimension; W Q m ∈R dk×dk , W K m ∈R dk×dk and W V m ∈R dv×dv are the projection matrices in the m-th head of the query, key, and value respectively. represents the output of each head in the multi-head self-attention mechanism; where O m represents the specific output representation of each head in the multi-head self-attention mechanism; f Aut () represents the function used to calculate self-attention.

[0137] The multi-head self-attention mechanism is adopted to assist in efficient information exchange to capture the long-range interactions between points in the trajectory graph, as described by the following formula:

[0138]

[0139] In the formula, represents the node representation after being processed by the multi-head self-attention mechanism; LayerNorm() represents the layer normalization operation; represents the node representation in the -th layer that has not been processed by the multi-head self-attention; represents the element-wise addition operation of features; MultiHead() represents the multi-head self-attention mechanism.

[0140] Hierarchical decoupling divides complex systems into multiple levels, separates different focus points or change points into different levels or modules, so that each level or module can focus more on its own tasks, thereby reducing the complexity of the system. By means of hierarchical decoupling, the graph diffusion process is separated from tasks such as feature extraction and classification, so that each level can focus on its own tasks. The self-attention layer assigns weights to nodes based on the feature information of the current layer, guiding the model to focus on important nodes and relationships. This layer-by-layer in-depth learning method enables the model to gradually capture the features and relationships of different levels, thereby learning a more accurate and comprehensive graph topology. The layer update rule is defined as follows:

[0141]

[0142] Wherein, H' l represents the node representation of the lth layer; σ() is the activation function; ζ() represents multi-head attention; || represents concatenation; W l Represents a weight matrix that can be trained layer by layer.

[0143] The introduction of loss function can measure the difference between the model's predicted value and the true value. In order to avoid gradient dissipation, the present invention introduces the cross entropy function. After obtaining the loss value by calculating the similarity between the actual output probability and the expected output probability, the model updates each parameter through back propagation to reduce the loss between the true value and the predicted value, so that the predicted value generated by the model is closer to the true value. Here, the vector h t is a discrete representation of continuous text, and this discretization can be performed at different resolutions. Here, the cross entropy function is improved to a multi-resolution version of the cross entropy function, by calculating h t+l The cross entropy at different spatial resolutions can slightly improve the prediction effect of the model. The cross entropy function is defined as follows:

[0144]

[0145] Among them, h′ t+l It is h t+l is a coarse version of ; β is a scalar for the relative importance of the coarse resolution loss; CE denotes cross entropy. p' t+l indicates the prediction result at time t+l; h' t+l represents the four-hot vector of coarse resolution; e 0:T+l-1 represents the high-dimensional embedding vector of the trajectory from 0 to T+l-1; p() represents the probability distribution function; p t+l represents the predicted probability at time t+l; h t+l represents a four-hot vector; β represents a scalar for the relative importance of coarse resolution loss; represents the multi-resolution cross-entropy loss function; L represents the total number of information propagation layers.

[0146] Through backpropagation, each trainable parameter is updated to gradually reduce the prediction error. Repeat the above steps until the model performance is optimized by calculating the final objective function on the validation set to minimize the objective function. The discriminant function of the linear classifier is used as the final objective function as follows:

[0147]

[0148] where B represents the batch size, which is the number of samples in the dataset; represents the multi-resolution cross-entropy loss function; L represents the number of information propagation layers; α represents the weight coefficient controlling the neighborhood radius. g represents the objective function; t represents the time step; p t+1 represents the predicted probability at time t + 1; f() represents this decoupled graph diffusion neural network model; X t represents the input feature at time t; A represents the adjacency matrix; α represents the weight coefficient controlling the neighborhood radius; l represents the current layer number; L represents the total number of information propagation layers; r l represents the residual of the l-th layer; Θ l,k represents the weight coefficient in the diffusion process; k represents the diffusion index at the k-th step; K represents the maximum diffusion step.

[0149] Finally, the regression task of predicting the precise trajectory route can be converted into a time node classification task. By giving a path trajectory at the given past time τ, let the model learn from the historical backtracking window of length τ to obtain the final driving behavior output sequence, which is activated by the Softmax function and predicts their probabilities in the next time period. Then sample possible potential future trajectory points, which will be decoded using the potential decoder, that is, obtain the end point information generated by the network.

[0150] In the application, the training set and validation set data can be used to train the built prediction model, and the optimal objective function value is obtained by minimization to gradually improve the model performance.

[0151] In one embodiment, the training method of the trajectory prediction model includes:

[0152] Define the dataset: Input the processed training data into the prediction model.

[0153] Forward propagation: The prediction model outputs the prediction result through forward propagation of the required data features via the decoupled graph diffusion neural network.

[0154] Backpropagation: Using the residuals in the training data to correct the errors in the test data. The loss value is obtained by calculating the cross-entropy function between the actual output probability and the expected output probability, and then the parameters are updated through backpropagation to reduce the loss between the true value and the predicted value, making the predicted value generated by the model approach the true value.

[0155] Iteration: Repeatedly execute the above operations to update the parameters and gradually optimize the performance of the model.

[0156] Tuning: Use the validation dataset prepared during the data processing to estimate the generalization error during or after training. According to the validation results, update the hyperparameters of the prediction model to optimize the model performance.

[0157] When training the trajectory prediction model, first, collect the dataset related to the prediction problem and complete data cleaning and preprocessing. Divide the cleaned and preprocessed dataset into a training set, a validation set, and a test set. When training, use the training set to train the selected model. During the training process, the model will learn the relationship between the input data and the output target; by setting an appropriate loss function, the difference between the predicted value and the true value of the model can be measured; the exemplary loss function adopted can be the cross-entropy function, and the parameters of the model are iteratively updated to minimize the loss function. Second, perform model tuning, adjust the hyperparameters of the model (such as learning rate, regularization coefficient, batch size, etc.) according to the performance of the model on the validation set; try different model architectures or increase the complexity of the model to improve the prediction performance. Finally, perform model evaluation and tuning, and use the test set to evaluate the tuned model. The evaluation metrics include accuracy, precision, etc. After the model training reaches the expected optimal performance, the model is deployed and monitored. Deploy the trained model to the actual application and continuously monitor its performance; regularly introduce new data, update and retrain the model to maintain its prediction ability.

[0158] In one embodiment, the above vehicle trajectory prediction method further includes:

[0159] Render the vehicle prediction trajectory and display it on the UI interface.

[0160] In the application, the trajectory prediction model can also be visualized to display the prediction trajectory; according to the predicted probability distribution of each driving behavior category, prompt straight, left turn, right turn, acceleration, deceleration, and stop to help the driver more intuitively understand the prediction results of this model.

[0161] Such as Figure 8As shown in the figure, it is a display diagram of the prediction results of autonomous driving behavior. The vehicle trajectory prediction display interface shows the historical trajectory of the target vehicle at the current moment, the local interaction situation with the surrounding roads and vehicles, and provides the interaction information of nearby vehicles. "Target historical trajectory" refers to the driving trajectory record of the target vehicle of interest in the past period of time. "Predicted trajectory" represents the prediction result of the system for the future driving path of the target vehicle based on the analysis of the historical trajectory during the vehicle operation. "True value trajectory" represents the driving trajectory of the target vehicle actually observed during the training process. By comparing with the predicted trajectory, the system is trained and optimized, and the accuracy and reliability of the prediction system are evaluated. During use, this interface can display the real-time driving behavior of the vehicle to the driver and provide relevant route suggestions.

[0162] The vehicle trajectory prediction method provided by the present invention cleverly solves the problem that it is difficult to accurately model the inherent heterogeneity and multimodality characteristics of motion data under the premise of given relatively low-dimensional observations by constructing a sparse high-dimensional representation of vehicle trajectory data. It not only expands the feature space to a higher dimension to avoid the difficulty of encoding complex motions in a low-dimensional space, but also effectively adjusts the mapping by adopting a unique four-hot connection method, thus greatly reducing the possibility of overfitting. This high-dimensional expansion of the feature space provides an effective solution to the problem of limited trajectory prediction when the vehicle has insufficient data collection conditions, gets rid of the constraint of expressing complex motion patterns in a low-dimensional space, and provides a solid theoretical basis and technical support for the accuracy and reliability of vehicle trajectory prediction.

[0163] The present invention adopts the method of generalized graph diffusion to learn the optimal graph topology of the task, so as to better capture the relationships and global information in the graph. The process of generalized graph diffusion can be carried out on the trajectory scatter plot constructed by us. Through this method, the principle of generalized diffusion can be used to learn the optimal topological structure. The advantage of this method is that by diffusing information in a larger neighborhood, it can more effectively capture the long-range dependence relationships and global information in the graph. Long-range dependence relationships and global information are the key elements for better understanding and analyzing the structure of the vehicle trajectory graph. In the specific implementation process, through generalized graph diffusion, information can be diffused in a larger neighborhood, the relationships and global information in the graph can be better captured, and thus a deeper understanding of the graph structure can be achieved.

[0164] Existing GNNs obtain the final node information representation by continuously aggregating and updating node information. However, this may lead to a high degree of distortion of the node feature representation. The present invention adopts a hierarchical decoupled representation learning strategy, constructs each layer as a parallel composition of a generalized graph diffusion layer and an attention layer, captures the internal hierarchical features between each trajectory point, can prevent the loss of unique local information in the learned representation, helps to aggregate global information, and avoids the omission of potential information.

[0165] Corresponding to the foregoing embodiments of the application function implementation method, the present invention further provides a vehicle trajectory prediction system, device, medium, and corresponding embodiments based on a decoupled graph diffusion neural network.

[0166] Please refer to Figure 9 , Figure 9 which is a schematic diagram of the module structure of the above vehicle trajectory prediction system. The above vehicle trajectory prediction system includes:

[0167] A trajectory graph acquisition module 91, configured to acquire driving process information in vehicle real-time data and construct a vehicle trajectory graph according to the driving process information;

[0168] An initial graph acquisition module 92, configured to process the vehicle trajectory graph based on a sparse high-dimensional representation method to obtain an initial graph;

[0169] A prediction module 93, configured to input the initial graph into a pre-trained trajectory prediction model and obtain a vehicle prediction trajectory output by the trajectory prediction model;

[0170] Wherein, the trajectory prediction model is a decoupled graph diffusion neural network; the trajectory prediction model uses generalized graph diffusion and a multi-head attention mechanism to capture the global information of the initial graph, and adopts a hierarchical decoupling method to represent the learning process.

[0171] In one embodiment, the above vehicle trajectory prediction system further includes:

[0172] A data preprocessing module, configured to perform data preprocessing on the vehicle trajectory graph.

[0173] Wherein, data preprocessing may exemplarily include: data cleaning, feature selection and scaling, constructing a data set, constructing an encoding, data normalization, etc.

[0174] In one embodiment, the above vehicle trajectory prediction system further includes:

[0175] A visualization module, configured to render the vehicle prediction trajectory and display it on a UI interface.

[0176] Regarding the system in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0177] The present invention embodiments also provide an electronic device. Please refer to Figure 10 , the electronic device 1000 includes a memory 1010 and a processor 1020.

[0178] The processor 1020 can be a Central Processing Unit (CPU), or can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0179] The memory 1010 can include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, the ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device can be a read-write storage device. The permanent storage device can be a non-volatile storage device that does not lose the stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device. In some other embodiments, the permanent storage device can be a removable storage device (such as a floppy disk, optical drive). The system memory can be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. The system memory can store some or all of the instructions and data required by the processor during operation. In addition, the memory 1010 can include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks can also be used. In some embodiments, the memory 1010 can include a removable storage device that can be read and / or written, such as a compact disc (CD), read-only digital versatile disc (such as DVD-ROM, dual-layer DVD-ROM), read-only Blu-ray disc, ultra-density disc, flash memory card (such as SD card, min SD card, Micro-SD card, etc.), magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and instantaneous electronic signals transmitted wirelessly or by wire.

[0180] An executable code is stored on the memory 1010. When the executable code is processed by the processor 1020, it can cause the processor 1020 to execute some or all of the methods described above.

[0181] In addition, the method according to the present invention can also be implemented as a computer program or a computer program product, which includes computer program code instructions for performing some or all of the steps in the above method according to the present invention.

[0182] Alternatively, the present invention can also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium), on which executable code (or a computer program or computer instruction code) is stored. When the executable code (or the computer program or computer instruction code) is executed by a processor of an electronic device (or a server, etc.), the processor is caused to execute some or all of the steps of the above method according to the present invention.

[0183] The various embodiments of the present invention have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the disclosed embodiments.

Claims

1. A vehicle trajectory prediction method based on decoupled graph diffusion neural network, characterized in that: include: Acquire driving process information from real-time vehicle data, and construct a vehicle trajectory map based on the driving process information; The driving process information includes an angle relative to the road, a distance to an adjacent left lane mark, a distance to an adjacent right lane mark, and a distance to a front obstacle; Constructing a vehicle trajectory map according to the driving process information includes: According to the driving process information, obtaining the historical driving trajectory of the vehicle; Uniformly sampling the historical driving trajectory of the vehicle to obtain the historical trajectory nodes of the vehicle; Based on the vehicle historical trajectory nodes, construct a vehicle trajectory graph; Processing the vehicle trajectory graph based on a sparse high-dimensional representation method to obtain an initial graph; Inputting the initial graph into a pre-trained trajectory prediction model to obtain a vehicle prediction trajectory output by the trajectory prediction model; The trajectory prediction model is a decoupled graph diffusion neural network; the trajectory prediction model includes: a generalized graph diffusion layer, a multi-head attention layer and a graph neural network; A hierarchical decoupling method is used between the generalized graph diffusion layer and the multi-head attention layer to capture hierarchical features and relationships at different levels between the trajectory nodes in the initial graph; the generalized graph diffusion layer uses generalized graph diffusion to capture the global information of the initial graph; the multi-head attention layer uses the output of the generalized graph diffusion layer as input to capture the global correlation of connected nodes and the global dependency of non-connected nodes in the initial graph; the graph neural network uses the output of the multi-head attention layer as input to aggregate features from the vicinity of the trajectory nodes; The processing of the vehicle trajectory graph based on a sparse high-dimensional representation method includes: Classifying the node data in the vehicle trajectory graph according to attributes; Discretize the classified node data and construct the unique hot vector of each attribute; A mapping relationship between each of the one-hot vectors and the corresponding high-dimensional real-valued embedding vector is constructed.

2. The vehicle trajectory prediction method based on decoupled graph diffusion neural network according to claim 1 is characterized in that: Two adjacent trajectory nodes in the initial graph v i arrive v j The side information transmission process utilizes express, The expression is as follows: ， In the formula, represents the signal energy; represents information entropy; For the i Trajectory nodes v i The eigenvector of For the Trajectory nodes v j The eigenvector of Track Node v i The signal energy is calculated using the following formula: ， In the formula, Indicates the number of trajectory node features; Indicates the trajectory time length of the historical backtracking window; express The length of the sequence; Track Node v i The information entropy of is calculated using the following formula: ， In the formula, x i The non-repetitive sequence is represented by ; Representation value probability; for x i Status statement; ， In the formula, represents the Dirac delta function.

3. The vehicle trajectory prediction method based on decoupled graph diffusion neural network according to claim 1 is characterized in that: The trajectory prediction model uses the following formula when performing the generalized graph diffusion operation: ， In the formula, Indicates Diffusion matrix of neural network layers; represents the maximum diffusion step length; represents the weight coefficient; Represents a column random transformation matrix.

4. The vehicle trajectory prediction method based on decoupled graph diffusion neural network according to claim 1 is characterized in that: The trajectory prediction model adopts a hierarchical decoupling method to represent the learning process, including: A hierarchical decoupled representation learning strategy is used to capture the hierarchical features of each hidden layer; each hidden layer includes a parallel generalized graph diffusion layer and an attention layer; the update rule of the hidden layer adopts the following formula: ， In the formula, Indicates Node representation of the layer; represents the activation function; Indicates multiple attentions; Indicates series connection; Indicates Diffusion matrix of neural network layers; Represents the weight matrix of layer-wise training; The weight matrix representing the diffusion process; Indicates The weight matrix of the layer node update process; Indicates potential diffusion representation; Represents the adjacency matrix of all nodes; is the Hadamard product, which means the multiplication of corresponding elements; Indicates The bias vector for the layer.

5. The vehicle trajectory prediction method based on decoupled graph diffusion neural network according to claim 1 is characterized in that: The training method of the trajectory prediction model includes: Input the training data into the trajectory prediction model; The trajectory prediction model outputs a prediction value by forward propagating the features of the training data through a decoupled graph diffusion neural network; The trajectory prediction model obtains the loss value by calculating the cross entropy function between the actual output probability and the expected output probability, and then uses back propagation to update the weight parameters; The trajectory prediction model iteratively updates the weight parameters until the predicted value and the true value meet the preset conditions.

6. The vehicle trajectory prediction method based on decoupled graph diffusion neural network according to claim 1 is characterized in that: The method further comprises: The predicted vehicle trajectory is rendered and displayed on a UI interface.

7. A vehicle trajectory prediction system based on a decoupled graph diffusion neural network, applied to execute the vehicle trajectory prediction method based on a decoupled graph diffusion neural network according to any one of claims 1 to 6, characterized in that: include: A trajectory map acquisition module is used to acquire the driving process information in the real-time data of the vehicle and construct a vehicle trajectory map according to the driving process information; An initial graph acquisition module, used for processing the vehicle trajectory graph based on a sparse high-dimensional representation method to obtain an initial graph; A prediction module, used to input the initial graph into a pre-trained trajectory prediction model to obtain a vehicle prediction trajectory output by the trajectory prediction model; Among them, the trajectory prediction model is a decoupled graph diffusion neural network; the trajectory prediction model uses generalized graph diffusion and multi-head attention mechanism to capture the global information of the initial graph, and adopts a hierarchical decoupling method to represent the learning process.

Images