Vehicle driving track prediction system and method based on improved graph neural network model

Abstract

The invention belongs to the field of traffic control systems, and relates to a vehicle driving track prediction system and method based on an improved graph neural network model, and the system comprises an FPGCN model and a Social LSTM model. The FPGCN model comprises a graph convolution layer, a Dropout layer, a differential pooling layer, a feature fusion module and a full connection layer; obtaining node feature vectors from graph structure data through a graph convolution layer, a Dropout layer and a differentiable pooling layer, then dividing and conquering node features, generating global features through a dense attention mechanism and principal component analysis, and generating local features through virtual filling and matrix calculation; after global features and local features are obtained, a feature fusion module is used for fusion, then the features are input into a full connection layer, and the influence score of a historical trajectory diagram on a current state diagram is calculated; integrating the preprocessed driving data and the data of which the score is calculated by the FPGCN model, and inputting the integrated data into a Social LSTM model; according to the system, while the model expression ability is maintained, the parameter quantity is remarkably reduced, the calculation logic is optimized, and the prediction accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the field of traffic control systems and relates to intelligent driving; specifically, it relates to a vehicle driving trajectory prediction system and method based on an improved graph neural network model. Background Art

[0002] With the advancement of science and technology and the gradual improvement of transportation facilities, intelligent networked vehicles have begun to develop rapidly, and now intelligent driving has become an important part of it. Through intelligent driving, vehicles can not only collect information about surrounding vehicles for interaction, but also assist in driving, detect road conditions, and avoid danger. When using intelligent driving, the first thing to do is to ensure the safety of the driver, which is inseparable from the efficient prediction of the vehicle's driving trajectory.

[0003] In the early vehicle trajectory prediction technology, the method based on the physical motion level was widely used, which is to predict the trajectory by extending the current vehicle state forward. This prediction method is only effective in relatively short time intervals, but if the surrounding vehicle information and the vehicle state are not considered, it will have a negative impact on long-term prediction. The subsequent machine learning technology provides a variety of time series network models for trajectory prediction, such as Recurrent Neural Network (RNN) or Long Short-Term Memory (LSTM), which predicts the driving trajectory by directly processing time series data; however, in actual situations, RNN and LSTM have great problems in time dependence. If the time step is large, RNN will have gradient problems.

[0004] Trajectory prediction technology not only needs to consider spatiotemporal interactive information, but also needs to infer future road conditions through screening. Therefore, the interactive information processing model has become a key improvement part of trajectory prediction technology. In the previous development of trajectory prediction technology, many papers have proved that using complex graph structure data as model input and information interaction is more interpretable for trajectory prediction, and it is easier to calculate feature relationships to achieve future trajectory prediction than other types of data. The experimental results are also more accurate than single time series models; and Kawasaki et al. [Kawasaki A, Seki A. Multimodal trajectory predictions for urban environments using geometric relationships between a vehicle and lanes [C] / / 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020: 9203-9209.] also combined lane information with KF filtering for trajectory prediction. However, when processing graph structure data, it is necessary to find a suitable method to process complex graph structure data. How to handle these complex relationships is a challenge. When processing road vehicle data, conventional convolutional neural networks may collect redundant information, which affects the model load, increases computational overhead and may affect detection accuracy. Summary of the invention

[0005] In order to address the problems of high data complexity and low efficiency in current vehicle driving trajectory prediction technology, the present invention aims to provide a vehicle driving trajectory prediction system based on an improved graph neural network model, which can not only efficiently process complex graph structure data converted from vehicle information, but also improve the calculation strategy to be closer to some behaviors during vehicle driving; by combining the advantages of deep feature divide-and-conquer structure and attention mechanism, the processing speed and accuracy of driving information features are significantly improved, while the model calculation efficiency is increased, so that the vehicle can make correct judgments in time during driving to ensure the personal safety of the driver.

[0006] To achieve the above object, the present invention adopts the following technical solution: A vehicle driving trajectory prediction system based on an improved graph neural network model, wherein the prediction system is obtained by training the improved graph neural network model, wherein the improved graph neural network model includes an FPGCN model and a SocialLSTM model; wherein the data input to the FPGCN model are two constructed graph structured data, namely, a historical trajectory graph and a current state graph, the FPGCN model is used to calculate the impact score of the historical trajectory graph on the current state graph, the pre-processed driving data is integrated with the data whose score is calculated by the FPGCN model, and then input into the Social LSTM model, and the predicted trajectory is output after passing through the social pooling network; Among them, the FPGCN model includes a graph convolution layer, a batch normalization layer, a Dropout layer, a differentiable pooling layer, a feature fusion module, and a fully connected layer; the graph convolution layer has three layers, and the scales of the three graph convolution layers are: 64, 32, and 16. After the graph convolution operation, the graph structure data introduces nonlinear factors through the activation function, and then the input of each layer is standardized through the batch normalization layer. The constructed graph structure data is aggregated through the three-layer graph convolution layer, so that each node considers its own feature information while also considering the relationship with the neighboring nodes, and then inputs the Dropout layer and the differentiable pooling layer; the Dropout layer randomly selects some nodes and forces them to be 0; the differentiable pooling layer compares the processed node feature matrix with the neighboring nodes of the graph The cluster assignment matrix is ​​calculated by connecting the matrix, and the node features assigned to the same cluster are aggregated through the cluster assignment matrix to complete the pooling operation; the node feature vector is obtained through the graph convolution layer, the Dropout layer, and the differentiable pooling layer, and then the node features are divided and conquered, and the global features are generated through the dense attention mechanism and principal component analysis, and the local features are generated through virtual filling and matrix calculation; after obtaining the global features and local features, they are fused using the feature fusion module and then input into a fully connected layer, and the relationship score between the historical trajectory graph and the current state graph is obtained through the fully connected layer, and the Sigmod function is used for 0-1 numerical mapping; the data with the completed score is integrated with the pre-processed driving data into vehicle driving data with score labels and then input into the Social LSTM model.

[0007] As a preferred embodiment of the present invention, the steps for training the improved graph neural network model are: Step S1. Select the vehicle dataset NGSIM as the vehicle driving trajectory dataset; Step S2. Processing the original data set; Step S3. After constructing the spatiotemporal graph structure data, the historical trajectory graph and the current state graph are input into the graph convolution layer of the FPGCN model to generate a node feature vector; Step S4. The node features generated by graph structure data after graph convolution, Dropout, and differentiable pooling are divided and conquered, and global features are generated through dense attention mechanism and PCA; local features are generated through virtual filling and matrix calculation; Step S5. After obtaining the global features and local features, the features are fused using the feature fusion module and then input into the fully connected layer; after obtaining the output of the fully connected layer, the output is converted into a 0-1 distribution through the Sigmod function; Step S6. The scored data is integrated with the preprocessed driving data into vehicle driving data with score labels and then input into the Social LSTM model. After that, the predicted trajectory is output after the social pooling network. During the training process, the root mean square error loss function and Adam optimizer are used, and the graph neural network model is continuously optimized and improved through Optuna.

[0008] As a further preference of the present invention, step S2 specifically includes the following steps: Step S201. Perform wavelet denoising on the original data set, and then introduce the same classification strategy. The vehicle data with the same vehicle_ID and unchanged Lane_ID in the vehicle driving data within the set time threshold is marked as LK; the normal vehicle data with the same vehicle_ID but different Lane_ID is marked as LC, and the positive and negative values ​​of the Lane_ID difference before and after the trajectory are marked again, with positive values ​​as TR and negative values ​​as TL; Step S202: Perform outlier detection on the denoised and classified data set; Step S203. Introduce a Savitzky-Golay smoothing window to perform data segmentation on the normalized vehicle data; Step S204: Convert complex data into graph structure data.

[0009] As a further preferred embodiment of the present invention, in step S3, the three-layer graph convolution layer aggregates the constructed graph structure data through a graph convolutional neural network, so that each node contains its own feature information. For each vehicle, its feature vector is represented as follows in a fixed frame:

[0010] in, H (l) It is l The feature matrix of the layer, =U+I is the adjacency matrix with self-loops added, U is the adjacency matrix of graph structure data, I is the identity matrix, and the matrix size is U The same matrix, the main diagonal element value is 1, and the other elements are 0, yes The degree matrix of , express The i-th diagonal element of Represents the neighbor matrix after adding the self-loop No. i Line j The elements of the column, W (l) It is l The weight matrix of the layer, σ() is the RELU activation function.

[0011] As a further preferred embodiment of the present invention, in step S4, before performing the dense attention mechanism, the feature vector output by the graph convolution layer is reshaped to [B, N, F], where B is the batch size, B=1 when a single graph is processed, N is the number of nodes, and F is the feature dimension; Then calculate the average feature, the calculation formula is:

[0012] in, Indicates i The average eigenvector of the graph, Indicates i Figure j The feature vector of a node, Indicates i Figure j The mask value of the node; through After traversing and calculating, stacking is performed to obtain the average feature matrix A sum ; Then calculate the global transformation vector by calculating the feature mean of all nodes :

[0013] in is a learnable weight matrix, is the average feature matrix, Global is the vector after global transformation, tanh() is the hyperbolic tangent function; Then, through the dot product operation of the feature vector and the global transformation vector, activation and weighting operations are calculated to obtain D ij :

[0014] in, It is i Tudi j The attention coefficient of each node, Indicates i Figure j The feature vector of a node, is the corresponding graph in the global transformation vector i The vector of () is the sigmoid activation function, which converts the dot product result into the attention weight in the form of probability, and calculates the attention coefficient of all nodes through iteration; Finally, the node features are summed and used for PCA processing;

[0015] in, z i It is i The weighted aggregate feature vector of the graph, Indicates i Figure j The mask value of each node, It is i Tudi j The attention coefficient of each node, Indicates i Figure j The feature vector of each node; In principal component analysis, the number of principal components with the greatest influence is set to 3.

[0016] As a further preferred embodiment of the present invention, the specific method of obtaining the local features in step S4 is: When virtual filling is performed on the feature vector, the missing node number difference is filled, and the filled elements are set to 0. The virtual filling data dimension F 12 for:

[0017] Among them, F 12 is the virtual filling quantity, N 1 is the feature dimension of feature vector 1, N 2 is the feature dimension of feature vector 2, It means taking the maximum value of two dimensions. It means taking the minimum value of two dimensions; After virtual filling, the abstract feature matrix is ​​formed. The shape of the history trajectory graph is (B1, N1, F), and the shape of the current state graph is (B2, N2, F); where B1 is the batch size of the history trajectory graph, N1 is the number of nodes in the history trajectory graph, B2 is the batch size of the current state graph, N2 is the number of nodes in the current state graph, and F is the feature dimension embedded in each node; Then, the historical trajectory graph is multiplied by the transpose of the current state graph through matrix multiplication to obtain a feature relationship matrix with a shape of (N1, N2), and the obtained relationship matrix can be normalized.

[0018] The present invention provides a vehicle driving trajectory prediction method based on an improved graph neural network model, the method comprising the following steps: Step 1. Obtain historical trajectory data and current status data; Step 2. Preprocess the data and convert it into graph structure data; Step 3. Input the historical trajectory and current state graph structure data into the vehicle driving trajectory prediction system based on the improved graph neural network model described above, and use the FPGCN model to calculate the impact score of the historical trajectory graph on the current state graph; Step 4. The preprocessed driving data is integrated with the data after the FPGCN model calculates the score and then input into the SocialLSTM model, which outputs the predicted trajectory after the social pooling network.

[0019] Advantages and beneficial effects of the present invention:

[0020] (1) The improved graph neural network model provided by the present invention combines the core ideas of graph convolutional neural networks and long short-term memory networks with a variety of optimization strategies, such as dense attention mechanism, differentiable pooling, social pooling, etc. These structures can significantly reduce the number of parameters and optimize the calculation logic while maintaining the expressive power of the model. This innovation is very suitable for realizing the trajectory prediction of target vehicles in complex traffic flows.

[0021] (2) The prediction system provided by the present invention divides and conquers feature vectors of different depths. Each stage focuses on processing features of different granularities, outputs node features as global features and local features, and gradually extracts and integrates graph structured information through a divide-and-conquer strategy, highlighting the main features for secondary data integration. This branching structure greatly improves the model's ability to perceive graph structure features and enhances feature collection effects, especially in complex target road conditions, increasing attention to surrounding vehicle information, enabling it to adapt to driving data under different road conditions and retain important local features without losing global features.

[0022] (3) The present invention selects feature vectors with large variance through PCA and retains them, and reduces their dimensionality; this design greatly reduces the risk of overfitting when the model processes global features. At the same time, when processing node features, the important data of the generated distribution map is mapped through matrix multiplication, and the calculated local features are combined with the global features to achieve more detailed feature information supplementation.

[0023] (4) The present invention uses an improved attention mechanism (dense attention mechanism) based on graph data when extracting global features. The improved mechanism can dynamically adjust the feature graph and allocate different degrees of attention. This dynamic attention optimization strategy can enhance the context-aware interaction capability, enabling the model to improve its ability to process feature relationships of complex vehicle graph structure data, thereby optimizing model performance and processing efficiency of complex data, increasing prediction accuracy, and reducing model loss.

[0024] (5) The present invention adopts a differentiable pooling strategy, which automatically learns a more efficient pooling operation through the back-propagation algorithm. The dense attention mechanism is combined with the differentiable pooling strategy (Diffpool strategy) to enhance the accuracy of feature extraction and the ability to perceive contextual interactions. While maintaining a high feature expression capability, the flexibility of the model is improved, the number of parameters and the amount of calculation are greatly reduced (the computational complexity is reduced), and the model training efficiency and prediction speed are optimized.

[0025] (6) The present invention adds Batch Normalization and Dropout layers to the improved network. The adaptive hierarchical dimensionality reduction method enables the model to cluster complex graph structure features when collecting them, automatically reduce invalid redundant data and retain key information, thereby improving the computational efficiency and robustness of the model.

[0026] (7) The present invention adopts the Social LSTM model, through social pooling and dynamic interaction modeling, so that the system can more efficiently aggregate and share vehicle status information, thereby capturing the interaction rules between road data, and finally mapping and outputting the output coordinates, and completing the model evaluation by comparing indicators.

[0027] (8) The prediction system provided by the present invention significantly reduces the number of parameters and error values ​​in processing complex graph structure data while maintaining a high prediction rate. Experimental results show that the model performs well in the task of vehicle trajectory prediction. It can quickly calculate surrounding information and make accurate predictions in complex vehicle driving environments, providing a new solution for achieving efficient and reliable vehicle trajectory prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings: Figure 1 It is a data processing flow chart of the improved graph neural network model (Hybrid FPGCN model) provided by the present invention; Figure 2It is the RMSE index value of the Hybrid FPGCN model of the present invention and other models on NGSIM data. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solution and advantages of the present invention, the present application is described in detail below in conjunction with the accompanying drawings, but it is not intended to limit the protection scope of the present invention.

[0030] like Figure 1 As shown, this embodiment provides a vehicle driving trajectory prediction system based on an improved graph neural network model, wherein the prediction system is obtained by training an improved graph neural network model (Hybrid FPGCN model), wherein the improved graph neural network model includes an FPGCN model and a Social LSTM model; wherein the data input to the FPGCN model are two constructed graph structured data, namely, a historical trajectory graph and a current state graph (target graph), and the FPGCN model is used to calculate the impact score of the historical trajectory graph on the current state graph, and the pre-processed driving data is integrated with the data whose score is calculated by the FPGCN model and then input into the Social LSTM model, and the predicted trajectory is output after the social pooling network; Among them, the FPGCN model includes a graph convolution layer (GCN), a batch normalization layer, a Dropout layer, a differentiable pooling layer (diffPool), a feature fusion module, and a fully connected layer; the graph convolution layer has three layers, and the scale of the three graph convolution layers is: 64, 32, and 16. After the graph convolution operation, the graph structure data introduces nonlinear factors and alleviates the gradient disappearance problem through the activation function (such as the relu activation function), and then standardizes the input of each layer through the batch normalization layer, thereby accelerating the convergence of the model. The constructed graph structure data is aggregated through the three-layer graph convolution layer, so that each node considers its own feature information while also considering the relationship with the neighboring nodes, and then inputs the Dropout layer and the differentiable pooling layer; the Dropout layer randomly selects some nodes and forces them to be set to 0, and the selected nodes no longer participate in the forward and backward propagation of the training process, thereby reducing the risk of overfitting in training, and random selection can improve the generalization ability of the model; the differentiable pooling layer (diffPool) is The processed node feature matrix and the adjacency matrix of the graph are used to calculate the cluster allocation matrix. The node features assigned to the same cluster are aggregated through the cluster allocation matrix to complete the pooling operation. Because the cluster allocation matrix is ​​adaptive, the pooling strategy will also change with the change of input data. This technology can adaptively change the pooling strategy to help the model learn the hierarchical data of the graph; the node feature vector is obtained through the graph convolution layer, Dropout layer, and differentiable pooling layer, and then the node features are divided and conquered. The global features are generated through the improved attention mechanism (dense attention mechanism) based on graph data and principal component analysis, and the local features are generated through virtual filling and matrix calculation; after obtaining the global features and local features, they are fused using the feature fusion module and then input into a fully connected layer, and the relationship score between the historical trajectory graph and the current state graph is obtained through the fully connected layer, and the Sigmod function is used for 0-1 numerical mapping; the data with the completed score is integrated with the pre-processed driving data into vehicle driving data with score labels and then input into the Social LSTM model.

[0031] This embodiment trains the above-constructed model, and the specific method is as follows: Step S1. Select the vehicle dataset NGSIM-US-101 as the vehicle driving trajectory dataset. This dataset is the vehicle trajectory data collected by the Next Generation Simulation project initiated by the Federal Highway Administration of the United States. It focuses on the vehicle driving conditions on highways such as US-101 in Los Angeles, California. It is an open dataset specifically used to study vehicle driving trajectories. Compared with traditional vehicle trajectory datasets (such as HighD, Argoverse1.0, etc.), the NGSIM dataset pays more attention to the surrounding road information during vehicle driving, especially the information data of the vehicle and surrounding vehicles when the vehicle behavior changes.

[0032] Step S2. Process the original data set. In the data processing stage, a series of methods are used to convert it into a form that can be processed by the model. The specific processing methods are as follows: Step S201: Perform wavelet denoising on the original data set, and then introduce the same classification strategy to further improve the representativeness of the data set.

[0033] Specifically, within the set time threshold, the vehicle data with the same vehicle_ID and Lane_ID is marked as LK (Lane_Keep); the normal vehicle data with the same vehicle_ID but different Lane_ID is marked as LC (Lane_Keep), and the positive and negative values ​​of the Lane_ID difference before and after the trajectory are marked again, with positive values ​​as TR (Turn_Right) and negative values ​​as TL (Turn_Left), so as to classify various behavior data more finely. Through this same classification strategy, the trajectory data of different behaviors can be effectively distinguished, which facilitates the model to further learn the data characteristics of the behavior during training.

[0034] The original data obtained in this embodiment and the quantity of each data after classification are shown in Table 1; Table 1 The number of original data and the number of classified data Category Label quantity Raw data 1048，576 Lane_Keep 286，496 Turn_Right 373，219 Turn_Left 213，608 Step S202: Perform outlier detection on the denoised and classified data set; Specifically, the code reads the data and normalizes the numerical features to ensure that all values ​​are on the same scale (between 0 and 1).

[0035] Step S203. The Savitzky-Golay smoothing window is introduced to segment the normalized vehicle data, and a low-order polynomial is applied to fit the data points of each segment, and then the original data points are replaced with the values ​​of the low-order polynomial. This can reduce the impact of extreme data on the model calculation amount and the negative impact on model training, and retain the local features of the data to the greatest extent; finally, the processed data is indexed and sorted.

[0036] Specifically, the Savitzky-Golay smoothing process is as follows: Savitzky-Golay smoothing (SG smoothing for short) is based on the least squares method, which analyzes and retains useful information in the data and reduces the impact of extreme data on the model and experimental results. In the present invention, the smoothing window width is selected as 2m+1, which is the number of original data points in the window n (n=2m+1).

[0037] Assume that the original data points in the window can be fitted by a k-1 degree polynomial, that is ;

[0038] Where: i = (-m, -m+1, ... 0, 1, ... m-1, m), each of the n original data points in the window can obtain one of the above polynomials, and n such polynomials constitute a k-variable linear equation system, and k fitting parameters a need to be solved. j (j=0, 1, 2, ...k-1), the filter window width z is generally selected w should be greater than or at least equal to k, when z w = k, the substitution method can be used to solve the fitting results; if z w >k, the least squares method is used to obtain the result.

[0039] ; Simplified to: ; The least squares solution of A is: ; Then the filtered value of Y The calculation formula is as follows: ; in: , X is the matrix containing the data indices to be fitted with the polynomial, is the coefficient set matrix, determined by and only by the X matrix, The matrix is ​​a (2m+1)×(2m+1) order matrix, A is the polynomial coefficient vector, E represents the error vector, the length is 2m+1, reflecting the error in the smoothing process, Y represents the original input data, and the SG smooth fitting equation can be obtained according to the coefficient matrix. For the smoothed data, the present invention visualizes the gradient changes of the data and finds that the smoothed experimental data is obviously more suitable for trajectory prediction than the original data.

[0040] Step S204: Convert complex data into graph structure data; Before the classified and processed data is input into the model, the complex vehicle data needs to be converted into a graph structure to facilitate the graph convolutional neural network in the model to extract its features.

[0041] This embodiment plans to predict future trajectories through historical trajectories. It requires a graph convolutional neural network to capture the important features of historical trajectories when performing certain behaviors. Therefore, before convolution, the data needs to be converted into a graph structure for pairwise processing, which are referred to as I1 (historical trajectory graph) and I2 (current state graph) below, respectively, to build a spatiotemporal relationship graph for the vehicle to be predicted. Assume that the total number of frames in which a certain vehicle appears in the training set is F n The experiment hopes to predict F through the model n+1 The subsequent trajectory is verified by comparison.

[0042] Specifically, data is extracted from the NGSIM dataset, such as vehicle ID (Vehicle_ID), timestamp (Total_Frames, Frame_ID), coordinates (Local_X, Local_Y), instantaneous speed (v_Vel), instantaneous acceleration (v_Acc), etc. to construct graph structure data. Taking the vehicle with Vehicle_ID a as an example and Frame_ID i as an example, a node is constructed. f a :

[0043]

[0044] in, f a Indicates the node data of car a itself when Frame_ID is i. Represents the relative coordinates of car a at time i are the instantaneous speed and acceleration of car a at time i respectively; G a represents the driving data of vehicles appearing near car a at time i. Suppose there are S vehicles around car a at time i. n Car, with S1, S2.. S i ..S n To distinguish, Indicates vehicle S i The relative coordinates of S i The instantaneous speed and acceleration of vehicle i at time i, f a The node is the center node of the graph at this time. After that, the relative position of car a and the surrounding cars needs to be calculated through coordinates to construct the edge of the undirected graph. Taking the vehicle S1 near car a at time i as an example, the edge value d between the two car nodes is i for:

[0045] in, and is the current relative coordinate of the S1 car.

[0046] Step S3. After constructing the spatiotemporal graph structure data, the historical trajectory graph and the current state graph (target graph) are input into the graph convolution layer (GCN) of the FPGCN model to generate node feature vectors.

[0047] Traditional GCN relies only on graph convolution layers and is difficult to pay attention to global context information. It only focuses on node relationships but ignores global spatiotemporal feature relationships, especially in complex road conditions (such as rush hour). To solve this problem, after obtaining the node feature vector, this embodiment divides and conquers the node features (the node feature vector is processed in two ways) to obtain global features and local features.

[0048] In addition, due to the characteristics of GCN, convolution operations exceeding three layers will cause gradient problems. Therefore, this application adopts a three-layer graph convolutional network with layer sizes of 64, 32, and 16. The three-layer graph convolutional network can greatly retain the results in the process of collecting features.

[0049] Specifically, the constructed graph structure data is aggregated through a graph convolutional neural network (scale 64, 32, 16), so that each node contains its own feature information. For each vehicle (node), its feature vector in a fixed frame can be expressed as:

[0050] in, H (l) It is l The feature matrix of the layer, =U+I is the adjacency matrix with self-loops added, U is the adjacency matrix of graph structure data, I is the identity matrix (the matrix size is the same as U The same matrix, where the main diagonal elements are 1 and the other elements are 0), yes The degree matrix of , express The i-th diagonal element of Represents the neighbor matrix after adding the self-loop No. i Line j The elements of the column, W (l) It is l The weight matrix of the layer, σ() is the RELU activation function.

[0051] Furthermore, due to the complexity of graph structure data, the model load will inevitably increase in the experiment, affecting the computing efficiency. Therefore, the present invention adds a Dropout layer (the drop probability is set to 0.3 in the experiment) and a differentiable pooling strategy (differentiable pooling layer) after the graph convolution layer before the node features are divided and conquered. The Dropout layer prevents the risk of overfitting and enhances the generalization ability of the model. The differentiable pooling strategy is used to dynamically adjust the pooling method with high flexibility to help the model learn the hierarchical data of the graph, further reduce the complexity of the graph while retaining key information.

[0052] Step S4. The node features generated by graph structure data after graph convolution, Dropout, and differentiable pooling are divided and conquered, and global features are generated through dense attention mechanism and PCA; local features are generated through virtual filling and matrix calculation; Specifically, the global features are obtained as follows: After obtaining the node features, in order to enhance the connection between local features, make the context interactive, increase the efficiency of the model in processing complex data, and enhance the ability to perceive context, this embodiment improves the attention mechanism to a dense attention mechanism (DAM) to make it more adaptable to complex graph feature data. It considers the pairwise relationship between all factors in the input sequence when calculating the attention score, thereby generating a dense attention score matrix, which helps the model make more accurate predictions and significantly improves the performance of the model when processing complex data. The specific steps are: (1) Increase data dimensions; For the feature vector (shape [N, F]) output by the graph convolution layer, it is necessary to adjust the shape to [B, N, F] (B is the batch size, B=1 when processing a single graph, N is the number of nodes, and F is the feature dimension) by increasing the batch dimension. This operation can meet the needs of implementing complex data and calculate the attention score without changing the essential content of the data.

[0053] (2) Calculate the average feature: The dense attention mechanism considers the characteristics of itself and neighboring nodes during calculation, and calculates the average characteristics through the adjustable mask matrix Q (a matrix composed of 0, 1, the dimension is the same as the node feature dimension, indicating whether the corresponding node is valid):

[0054] in, Indicates i The average eigenvector of the graph, Indicates i Figure j The feature vector of a node, Indicates i Figure jThe mask value of the node; through After traversing and calculating, the average feature matrix can be obtained by stacking A sum .

[0055] (3) Global transformation vector: The global transformation vector can provide contextual information for the dense attention mechanism, so that each node feature will interact with the global transformation vector when calculating the attention coefficient. It is calculated by calculating the feature mean of all nodes:

[0056] in is a learnable weight matrix, is the average feature matrix, Global It is a vector after global transformation, tanh() is the hyperbolic tangent function, which introduces nonlinear activation to enable the model to learn complex data.

[0057] (4) Calculate the attention coefficient: The attention coefficient enables the model to adaptively assign different weights to each node of dense data, so as to better capture the feature information of dense data. The coefficient is calculated by the dot product operation of the feature vector and the global transformation vector, activation and weighting operations. D ij :

[0058] in, It is i Tudi j The attention coefficient of each node, Indicates i Figure j The feature vector of a node, is the corresponding graph in the global transformation vector i The vector of () is the sigmoid activation function, which converts the dot product result into attention weight in the form of probability. The attention coefficient of all nodes can be calculated through iteration.

[0059] (5) Weighted summation: The purpose of node feature weighting is to assign different weights to features of different importance to highlight important node features; the purpose of combining node features is to integrate all node information and unify them for subsequent PCA processing:

[0060] in, zi It is i The weighted aggregate feature vector of the graph, Indicates i Figure j The mask value of each node, It is i Tudi j The attention coefficient of each node, Indicates i Figure j The feature vector generated by the graph convolution layer has stronger feature expression after being processed by the improved attention mechanism (dense attention mechanism), thereby improving the model's ability to process complex graph data.

[0061] After obtaining the data after global feature processing, principal component analysis (PCA) is used to centralize, decompose, select principal components, and transform the clustered feature points to obtain the feature projection calculated by the global feature, completing the global feature processing steps in the divide-and-conquer processing structure.

[0062] It should be noted that in this embodiment, the existing PCA is directly used. The key lies in adjusting the parameters and setting the number of principal components with the greatest influence to 3 to reduce the complexity of the model and the risk of overfitting.

[0063] Specifically, the local features are obtained as follows: (1) After obtaining the node features, the node and edge lengths are determined. However, in order to avoid the influence of the experimental results due to the difference in the data structure or dimension of the two features in the later stage of the experiment, it is necessary to perform virtual filling processing on the feature vector and fill the missing node number difference. In the experiment, the filled elements are set to 0, so that the data dimension can be kept consistent. Assuming that the two feature dimensions are (N1, N2), the virtual filled data dimension F 12 for:

[0064] Among them, F 12 is the virtual filling quantity, N 1 is the feature dimension of feature vector 1, N 2 is the feature dimension of feature vector 2, It means taking the maximum value of two dimensions. Indicates taking the minimum value of two dimensions.

[0065] (2) Abstract the feature matrix. The two features input to the model of this embodiment are the historical trajectory graph feature and the target graph feature. Taking the historical trajectory graph as an example, its shape and size are (B1, N1, F), where B1 is the batch size of the historical trajectory graph (indicates the number of graphs in batch processing), N1 is the number of nodes in the historical trajectory graph, and F is the feature dimension embedded in each node; similarly, the shape of the target graph is (B2, N2, F), B2 is the batch size of the target graph (current state graph), and N2 is the number of nodes in the target graph (current state graph). Then, the abstracted graph is batch processed.

[0066] (3) Convert the abstract features processed in step 2 from sparse format to dense format; Specifically, the historical trajectory graph is multiplied by the transpose of the target graph through matrix multiplication to obtain a feature relationship matrix with a shape of (N1, N2), and the obtained relationship matrix is ​​normalized; at this time, the mapped value shows the density of the feature relationship of the node data.

[0067] Step S5. After obtaining the global features and local features, the features are fused using a feature fusion module and then input into a fully connected layer; Specifically, the global features after PCA mapping and the local features after matrix transformation are fused and then input into a fully connected layer; the function of the fully connected layer is to map the fused features.

[0068] Assume that the feature vector is f, the weight matrix of the fully connected layer is W and the bias term is b, then the output of the fully connected layer can be expressed as:

[0069] Among them: the output of the fully connected layer is Z, which is the corresponding score; W is N out ×C in The weight matrix, N out is the number of output features, C in is the number of input features.

[0070] After obtaining the output of the fully connected layer, this embodiment usually converts the output into a 0-1 distribution through a Sigmod function. The formula of the Sigmod function is:

[0071] in: z is the input of the function, e is the base of natural logarithms (approximately equal to 2.71828), is the output of the function, and its value range is (0, 1). The closer the value is to 0, the lower the relationship between the historical trajectory graph and the target graph, and the closer it is to 1, the closer the relationship is. When facing similar road conditions, the probability of making driving behaviors with the same trajectory as the history is greater.

[0072] Step S6. The scored data is integrated with the pre-processed driving data into vehicle driving data with score labels and then input into the Social LSTM model. After that, the predicted trajectory is output after the social pooling network.

[0073] It should be noted that in this embodiment, the differentiable pooling layer, the Social LSTM model and the social pooling network can all adopt the existing network structure. The key to this application is to combine it with the divide-and-conquer structure. Therefore, this application will not introduce its specific network structure and data processing process in detail.

[0074] In the training process, this embodiment uses the root mean square error (RMSE) loss function and Adam optimizer, and optimizes the hyperparameters such as learning rate, batch size, and number of training rounds through Optuna to improve the performance and accuracy of the model. In the testing phase, the processed NGSIM vehicle data is integrated by generating relationship scores through the FPGCN model to complete the input preparation for trajectory prediction of the Social LSTM model; for example, to i The trajectory of the vehicle v in the frame is predicted, and the historical trajectory map (frame 1 to frame 2) is trained by the model. i -1 frame as the historical trajectory map) and the target trajectory map ( i The system processes the vehicle driving data accurately and efficiently, and provides strong technical support for efficient and intelligent driving.

[0075] Most existing trajectory predictions use attention mechanisms and Transformers for encoding and decoding in feature collection, and only use global features. Therefore, the root mean square error (RMSE) of the predicted value will be a little higher. The prediction system provided in this embodiment obtains global features and local features through a divide-and-conquer structure. The local features are fused with the global features and added to the feature calculation. The global features are used as the main part, and the local features are used as information completion after calculation. This is equivalent to considering road vehicle data more comprehensively than previous methods to achieve better prediction results. In order to prove that the model provided in this embodiment is more suitable for vehicle trajectory prediction and has better prediction results, this application compares it with other existing models. The specific results are as follows: Figure 2 shown.

[0076] pass Figure 2It can be seen that the Hybrid FPGCN model provided by the present invention performs better than other prediction models in terms of root mean square error (RMSE) index. The average value of the RMSE index in the first 30 frames (3s) is 0.31, 0.31, 0.25, 1.74, 0.27, 0.16, 0.69, and 0.19 lower than that of CS-LSTM, M-LSTM, TS-GAN, BRDP, WSIP, HMNET, S-GCN, and MFP models, and the RMSE values ​​in the time periods of 10, 20, and 30 frames are lower than those of the above models. This shows that the present invention has better capabilities in graph structure data processing and trajectory prediction. Example

[0077] This embodiment provides a vehicle driving trajectory prediction method based on an improved graph neural network model, the method comprising the following steps: Step 1. Obtain historical trajectory data and current status data; Step 2. Preprocess the data and convert it into graph structure data; Step 3. Input the historical trajectory and current state graph structure data into the vehicle driving trajectory prediction system based on the improved graph neural network model described in Example 1, and use the FPGCN model to calculate the impact score of the historical trajectory graph on the current state graph; Step 4. The preprocessed driving data is integrated with the data whose scores are calculated by the FPGCN model and then input into the SocialLSTM model, which outputs the predicted trajectory after passing through the social pooling network.

[0078] The present invention also provides an electronic device, comprising: one or more processors and a memory; wherein the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the vehicle driving trajectory prediction method based on the improved graph neural network model described in Example 2.

[0079] The present invention also provides a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processor, the vehicle driving trajectory prediction method based on the improved graph neural network model described in Example 2 is implemented.

[0080] Those skilled in the art will appreciate that all or part of the functions of the various methods / modules in the above embodiments may be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program may be stored in a computer-readable storage medium, which may include: a read-only memory, a random access memory, a disk, an optical disk, a hard disk, etc. The program is executed by a computer to implement the above functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above functions can be implemented.

[0081] In addition, when all or part of the functions in the above-mentioned embodiments are implemented by means of a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash drive or a mobile hard disk, and saved to the memory of a local device by downloading or copying, or the system of the local device is updated. When the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0082] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art of the present invention, according to the idea of ​​the present invention, some simple deductions, deformations or substitutions can be made. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A vehicle driving trajectory prediction system based on an improved graph neural network model, wherein the prediction system is obtained by training the improved graph neural network model, and is characterized in that: The improved graph neural network model includes an FPGCN model and a Social LSTM model; wherein the data input to the FPGCN model are two constructed graph structured data, namely, a historical trajectory graph and a current state graph; the FPGCN model is used to calculate the impact score of the historical trajectory graph on the current state graph; the pre-processed driving data is integrated with the data whose score is calculated by the FPGCN model and then input into the Social LSTM model, and the predicted trajectory is output after the social pooling network; Among them, the FPGCN model includes a graph convolution layer, a batch normalization layer, a Dropout layer, a differentiable pooling layer, a feature fusion module, and a fully connected layer; the graph convolution layer has three layers, and the scales of the three graph convolution layers are: 64, 32, and 16. After the graph convolution operation, the graph structure data introduces nonlinear factors through the activation function, and then the input of each layer is standardized through the batch normalization layer. The constructed graph structure data is aggregated through the three-layer graph convolution layer, so that each node considers its own feature information while also considering the relationship with the neighboring nodes, and then inputs the Dropout layer and the differentiable pooling layer; the Dropout layer randomly selects some nodes and forces them to be 0; the differentiable pooling layer compares the processed node feature matrix with the neighboring nodes of the graph The cluster assignment matrix is ​​calculated by connecting the matrix, and the node features assigned to the same cluster are aggregated through the cluster assignment matrix to complete the pooling operation; the node feature vector is obtained through the graph convolution layer, the Dropout layer, and the differentiable pooling layer, and then the node features are divided and conquered, and the global features are generated through the dense attention mechanism and principal component analysis, and the local features are generated through virtual filling and matrix calculation; after obtaining the global features and local features, they are fused using the feature fusion module and then input into a fully connected layer, and the relationship score between the historical trajectory graph and the current state graph is obtained through the fully connected layer, and the Sigmod function is used for 0-1 numerical mapping; the data with the completed score is integrated with the pre-processed driving data into vehicle driving data with score labels and then input into the Social LSTM model.

2. A vehicle driving trajectory prediction system based on an improved graph neural network model according to claim 1, characterized in that: The steps for training the improved graph neural network model are: Step S1. Select the vehicle dataset NGSIM as the vehicle driving trajectory dataset; Step S2. Processing the original data set; Step S3. After constructing the spatiotemporal graph structure data, the historical trajectory graph and the current state graph are input into the graph convolution layer of the FPGCN model to generate a node feature vector; Step S4. The node features generated by graph structure data after graph convolution, Dropout, and differentiable pooling are divided and conquered, and global features are generated through dense attention mechanism and PCA; local features are generated through virtual filling and matrix calculation; Step S5. After obtaining the global features and local features, the features are fused using the feature fusion module and then input into the fully connected layer; after obtaining the output of the fully connected layer, the output is converted into a 0-1 distribution through the Sigmod function; Step S6. The scored data is integrated with the pre-processed driving data into vehicle driving data with score labels, and then input into the Social LSTM model, and then outputs the predicted trajectory after passing through the social pooling network; During the training process, the root mean square error loss function and Adam optimizer are used, and the graph neural network model is continuously optimized and improved through Optuna.

3. A vehicle driving trajectory prediction system based on an improved graph neural network model according to claim 2, characterized in that: Step S2 specifically includes the following steps: Step S201. Perform wavelet denoising on the original data set, and then introduce the same classification strategy. The vehicle data with the same vehicle_ID and unchanged Lane_ID in the vehicle driving data within the set time threshold is marked as LK; the normal vehicle data with the same vehicle_ID but different Lane_ID is marked as LC, and the positive and negative values ​​of the Lane_ID difference before and after the trajectory are marked again, with positive values ​​as TR and negative values ​​as TL; Step S202: Perform outlier detection on the denoised and classified data set; Step S203. Introduce a Savitzky-Golay smoothing window to perform data segmentation on the normalized vehicle data; Step S204: Convert complex data into graph structure data.

4. A vehicle driving trajectory prediction system based on an improved graph neural network model according to claim 2, characterized in that: In step S3, the graph convolution layer described in the third layer aggregates the constructed graph structure data through the graph convolution neural network, so that each node contains its own feature information. For each vehicle, its feature vector is represented as follows in a fixed frame: ； in, H (l) It is l The feature matrix of the layer, =U+I is the adjacency matrix with self-loops added, U is the adjacency matrix of graph structure data, I is the identity matrix, and the matrix size is U The same matrix, where the main diagonal element value is 1 and the other elements are 0, yes The degree matrix of , express The i-th diagonal element of Represents the neighbor matrix after adding the self-loop No. i Line j The elements of the column, W (l) It is l The weight matrix of the layer, σ() is the RELU activation function.

5. The vehicle driving trajectory prediction system based on the improved graph neural network model according to claim 2 is characterized in that: Step S4: Before performing the dense attention mechanism, the feature vector output by the graph convolution layer is reshaped to [B, N, F], where B is the batch size, B=1 when processing a single graph, N is the number of nodes, and F is the feature dimension. Then calculate the average feature, the calculation formula is: ； in, Indicates i The average eigenvector of the graph, Indicates i Figure j The feature vector of a node, Indicates i Figure j The mask value of the node; through After traversing and calculating, stacking is performed to obtain the average feature matrix A sum ; Then calculate the global transformation vector by calculating the feature mean of all nodes : ； in is a learnable weight matrix, is the average feature matrix, Global is the vector after global transformation, tanh() is the hyperbolic tangent function; Then, through the dot product operation of the feature vector and the global transformation vector, activation and weighting operations are calculated to obtain D ij : ； in, It is i Tudi j The attention coefficient of each node, Indicates i Figure j The feature vector of a node, is the corresponding graph in the global transformation vector i The vector of () is the sigmoid activation function, which converts the dot product result into the attention weight in the form of probability, and calculates the attention coefficient of all nodes through iteration; Finally, the node features are summed and used for PCA processing; ； in, z i It is i The weighted aggregate feature vector of the graph, Indicates i Figure j The mask value of each node, It is i Tudi j The attention coefficient of each node, Indicates i Figure j The feature vector of each node; In principal component analysis, the number of principal components with the greatest influence is set to 3.

6. A vehicle driving trajectory prediction system based on an improved graph neural network model according to claim 2, characterized in that: The specific method of obtaining the local features in step S4 is: When virtual filling is performed on the feature vector, the missing node number difference is filled, and the filled elements are set to 0. The virtual filling data dimension F 12 for: ; Among them, F 12 is the virtual filling quantity, N 1 is the feature dimension of feature vector 1, N 2 is the feature dimension of feature vector 2, It means taking the maximum value of two dimensions. It means taking the minimum value of two dimensions; After virtual filling, the abstract feature matrix is ​​formed. The shape of the history trajectory graph is (B1, N1, F), and the shape of the current state graph is (B2, N2, F); where B1 is the batch size of the history trajectory graph, N1 is the number of nodes in the history trajectory graph, B2 is the batch size of the current state graph, N2 is the number of nodes in the current state graph, and F is the feature dimension embedded in each node; Then, the historical trajectory graph is multiplied by the transpose of the current state graph through matrix multiplication to obtain a feature relationship matrix with a shape of (N1, N2), and the obtained relationship matrix can be normalized.

7. A vehicle trajectory prediction method based on an improved graph neural network model, characterized in that: The method comprises the following steps: Step 1. Obtain historical trajectory data and current status data; Step 2. Preprocess the data and convert it into graph structure data; Step 3. Input the historical trajectory and current state graph structure data into the vehicle driving trajectory prediction system based on the improved graph neural network model according to any one of claims 1 to 6, and use the FPGCN model to calculate the impact score of the historical trajectory graph on the current state graph; Step 4. The preprocessed driving data is integrated with the data after the FPGCN model calculates the score and then input into the Social LSTM model, which outputs the predicted trajectory after the social pooling network.

Images