A traffic flow prediction method based on an improved GCN-attention algorithm

Abstract

The present invention relates to a traffic flow prediction method based on an improved GCN-attention algorithm, which establishes a GSTA model. The GSTA model includes a number of ST-Blocks and an output module. Each ST-Block includes a GCN sub-module, a time sub-module, and a gated fusion sub-module. Historical traffic flow data is input into one of the ST-Blocks. The GCN sub-module is used to extract spatial features; the time sub-module is used to extract temporal features; the gated fusion sub-module is used to fuse the spatial features and the temporal features to obtain spatio-temporal features. The spatio-temporal features and the initial historical traffic flow data are fused through a residual link as the input of the next ST-Block, and the process returns to re-extract spatial features until the spatio-temporal features of all ST-Blocks are obtained. The spatio-temporal features of all ST-Blocks are input into the output module, and after linear transformation, the predicted traffic flow data is obtained. Compared with the prior art, the present invention has the advantages of more accurate prediction results and the like.

Description

Technical Field

[0001] The present invention relates to the field of traffic flow prediction, and in particular to a traffic flow prediction method based on an improved GCN-attention algorithm. Background Art

[0002] As an important part of the intelligent transportation system, accurate spatio-temporal traffic prediction has attracted many researchers and has also been applied to many real-world scenarios. Due to its excellent performance in the field of time series analysis, RNN (Recurrent Neural Network) and its variants, such as GRU (Gated Recurrent Unit) and LSTM (Long Short-Term Memory Network), have become the mainstream tools for analyzing the temporal correlation of traffic flow data. In addition to temporal correlation, another key factor affecting the accuracy of traffic flow prediction is spatial correlation. To analyze the spatial correlation of traffic flow data, researchers transform the road network into a graph and use CNN (Convolutional Neural Network) to analyze the spatial correlation between different regions.

[0003] Recently, researchers have begun to consider traffic flow prediction problems from the perspective of graph modeling. After a road network graph is given, researchers use GCN (Graph Convolutional Network) to extract spatial features. Some researchers have combined GCN with RNN to capture spatio-temporal features and achieved amazing results. However, most of the current GCN methods used are traditional GCNs, and traditional GCNs have two disadvantages. The first disadvantage is that as the depth of the GCN increases, it will lead to the over-smoothing problem. This is because after the aggregation radius of the nodes reaches a threshold, the initial state of the nodes will be ignored, resulting in the states of all nodes becoming the same. Another disadvantage is that traditional GCNs only select the output of the last convolutional layer as the result and ignore the outputs of other convolutional layers. In extreme cases, there will no longer be spatial dependence, and at this time, aggregating the information of neighboring nodes can only add useless information to the target node. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a traffic flow prediction method based on an improved GCN-attention algorithm.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A traffic flow prediction method based on an improved GCN-attention algorithm, comprising the following steps:

[0007] S1. Input the historical traffic flow data into the GSTA model. The GSTA model includes several ST-Blocks and an output module. Each ST-Block includes a GCN sub-module, a time sub-module, and a gated fusion sub-module. The historical traffic flow data is input into one of the ST-Blocks;

[0008] S2. Extract spatial features through the GCN sub-module;

[0009] S3. Extract temporal features through the time sub-module;

[0010] S4. Use the gated fusion sub-module to fuse the spatial features and temporal features to obtain spatio-temporal features;

[0011] S5. Fuse the spatio-temporal features and the initial historical traffic flow data through a residual link as the input of the next ST-Block, and return to execute step S2 until the spatio-temporal features of all ST-Blocks are obtained;

[0012] S6. Input the spatio-temporal features of all ST-Blocks into the output module, and obtain the predicted traffic flow data after linear transformation;

[0013] Among them, the steps for the GCN sub-module to extract spatial features are as follows:

[0014] S21. Establish the adjacency matrix of the historical traffic flow data;

[0015] S22. Perform information diffusion and information aggregation on the historical traffic flow data and its adjacency matrix in sequence, and link the outputs of all graph convolutional layers to obtain the spatial features of the urban road network.

[0016] Furthermore, the expressions for information diffusion and information aggregation are as follows:

[0017] Information diffusion:

[0018]

[0019]

[0020]

[0021] Among them, θ 1 、θ 2 、b 1 、b 2 are learnable parameters, H l is the historical traffic flow data, m is the current number of convolutional layers, is the output of the current convolutional layer, A is the adjacency matrix, and σ represents the sigmoid function;

[0022] Information aggregation:

[0023]

[0024] Among them, S is the spatial feature, W and c are learnable parameters, and M is the total number of convolutional layers.

[0025] Furthermore, the adjacency matrix is an adaptive adjacency matrix and is updated by the gradient descent method.

[0026] Furthermore, the steps for the time sub-module to extract temporal features are as follows:

[0027] S31. Obtain the time step information corresponding to the historical traffic flow data;

[0028] S32. Perform embedding on the time step to obtain a time vector;

[0029] S33. Calculate the attention vector according to the time vector and the historical traffic flow data;

[0030] S34. Calculate the temporal feature according to the attention vector.

[0031] Furthermore, the calculation expression of the attention vector is as follows:

[0032]

[0033]

[0034]

[0035] Among them, Q, K, and V all represent an attention vector, k represents the k-th head of attention, vi represents the urban road network node, tj represents the j-th time step, and Relu is the activation function, are all learnable parameter matrices.

[0036] Furthermore, the specific calculation expression of step S34 is as follows:

[0037] d = D / K

[0038]

[0039]

[0040]

[0041]

[0042] Among them, is the temporal representation vector, is the temporal feature, is the correlation parameter, is the attention score, D represents the hidden dimension of the input matrix, and K represents the number of attentions in the multi-head attention. are learnable parameters.

[0043] Furthermore, the time step information is the total time of a day divided by the frequency of the signals collected by the traffic flow sensors.

[0044] Furthermore, the calculation expression for the fusion in step S4 is as follows:

[0045]

[0046]

[0047] Among them, represents the spatial feature, represents the temporal feature, g represents the mixing ratio, and b g , W g are learnable parameters.

[0048] Furthermore, the output module performs the following steps:

[0049] S61. Input the input of the output module into the Relu function layer and the CNN layer in sequence for dimension conversion;

[0050] S62. Input the result after dimension conversion into the auxiliary ST-Block to output the result. The time step information in the auxiliary ST-Block is the time step information of the traffic flow data to be predicted;

[0051] S63. Input the result output in S62 into the linear layer to obtain the predicted traffic flow data.

[0052] Furthermore, the calculation expression for the input of the output module in step S6 is as follows:

[0053]

[0054] Among them, X out is the input of the output module, and H k is the output of the k-th ST-Block.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] 1. Compared with the traditional GCN method, the present invention links the outputs of all graph convolutional layers through information diffusion and information aggregation, takes into account all outputs, makes the prediction result more accurate, and sets multiple ST-Blocks to process the data with residual links and obtains multiple feature outputs by means of iterative calculation, solving the problem of over-smoothing of the output result in the prior art and making the prediction result more precise.

[0057] 2. Compared with the existing RNN method, the timing module of the present invention uses the embedding technology to extract time vectors, requiring fewer parameters and having higher efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic flowchart of the present invention.

[0059] Figure 2 It is a complete model framework diagram of GSTA related to the present invention.

[0060] Figure 3 It is a schematic framework diagram of the GNN module in the present invention.

[0061] Figure 4 It is a schematic framework diagram of the time sub-module in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives the detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0063] This embodiment provides a traffic flow prediction method based on an improved GCN-attention algorithm, which is mainly applied to predicting the traffic flow data of the next period by obtaining the historical traffic flow data within a period of time to play a role in guiding traffic. The process is as Figure 1 shown, and specifically includes the following steps:

[0064] Step S1: First, establish a GSTA model. The GSTA model includes several ST-Blocks and an output module. Each ST-Block includes a GCN sub-module, a temporal module (Temporal Module), and a gated fusion sub-module (gatedfusion). After adjusting the dimension of the historical traffic flow data through a linear layer (Linear), it is input into one of the ST-Blocks. The schematic diagram of the overall model is as Figure 2 shown.

[0065] Step S2: Extract spatial features through the GCN sub-module. The main execution diagram of the GCN sub-module is as Figure 3 shown, and the specific steps are as follows:

[0066] Step S21: Establish the adjacency matrix of historical traffic flow data. The efficient establishment of the adjacency matrix is very important for the GCN to extract spatial features. For the traditional method, simply setting the elements in the adjacency matrix to 0 or 1 to represent the spatial relationship between two nodes is too simple. In this embodiment, a method for constructing an adaptive adjacency matrix is proposed as follows:

[0067]

[0068] where S 1 and S 2 are two vector matrices, both with dimensions N*C. N is the number of nodes in the road network graph to which the traffic flow data belongs, and C is a custom dimension, which is taken as 10 in this patent. Among them, S 1 and S 2 are two self-learning vectors. In each iteration, they will be updated according to gradient descent, and the adjacency matrix A is updated with each iteration according to S 1 and S 2 The SoftMax function is used to normalize the matrix.

[0069] Step S22: Perform information diffusion and information aggregation on the historical traffic flow data and its adjacency matrix in sequence, and link the outputs of all graph convolutional layers to obtain the spatial features of the urban road network.

[0070] The expressions for information diffusion and information aggregation are as follows:

[0071] Information diffusion:

[0072]

[0073]

[0074]

[0075] where θ 1 、θ 2 、b 1 、b 2 are learnable parameters, H l is the historical traffic flow data, m is the current convolutional layer number, is the output of the current convolutional layer, A is the adjacency matrix, and σ represents the sigmoid function.

[0076] Information aggregation:

[0077]

[0078] where S is the spatial feature, W and c are learnable parameters, and M is the total number of convolutional layers.

[0079] By aggregating information, the outputs of each graph convolutional layer can be linked, taking into account the outputs of each layer and more efficiently extracting spatial features.

[0080] Step S3: Extract temporal features through the temporal sub-module. The main execution graph of the temporal sub-module is as Figure 4 shown, and specifically includes the following steps:

[0081] Step S31: Obtain the time step information corresponding to the historical traffic flow data. According to different observation periods, a day is divided into n time steps. For example, if the sensor measures the average vehicle speed every 5 minutes, then a day is divided into 288 time steps.

[0082] Step S32: Pass the time steps through an embedding layer (Time bedding) to obtain a d-dimensional time vector TE tj .

[0083] Step S33: Calculate the attention vector based on the time vector and the historical traffic flow data. In this embodiment, a multi-head attention mechanism (Temporal Attention) is adopted. For each head of attention, three attention vectors need to be calculated first, and the calculation expressions are as follows:

[0084]

[0085]

[0086]

[0087] where Q, K, and V all represent an attention vector, k represents the k-th head of attention, vi represents the urban road network node, tj represents the j-th time step, Relu is the activation function, are all learnable parameter matrices.

[0088] Step S34: Calculate the temporal features according to the attention vector according to the following formula.

[0089] d = D / K

[0090]

[0091]

[0092] where represents the correlation between the traffic flow at time step tj and t, The attention score of the k-th head representing the traffic flow at time step t, indicating the importance of the traffic state at time step t to the traffic state at time step tj. D represents the hidden dimension of the input matrix and K represents the number of attentions in multi-head attention. After obtaining the attention scores, the temporal representation vector of the node at the target time step can be calculated as follows:

[0093]

[0094]

[0095] Among them, is the temporal representation vector, is the temporal feature, are learnable parameters, K is the number of heads of multi-head attention, which is set to 8 in this embodiment.

[0096] Step S4: Use the gated fusion sub-module to fuse the spatial feature and the temporal feature to obtain the spatio-temporal feature. The specific calculation expression is as follows:

[0097]

[0098]

[0099] Among them, represents the spatial feature, represents the temporal feature, g represents the mixing ratio, b g and W g are learnable parameters.

[0100] Step S5: To improve the stability of the model and enhance the model's ability to extract spatio-temporal features, in this embodiment, the ST-Block is stacked with K layers, preferably 4 in this embodiment. To solve the problem of over-smoothing of the results of traditional traffic flow prediction methods, this embodiment also introduces a residual link (Skip connection). By adding the output of each layer of the gated fusion sub-module to the input historical traffic flow data through the residual link, the input of the next layer of the ST-Block can be obtained, and then return to execute step S2 until the spatio-temporal features output by all ST-Blocks are obtained.

[0101] Step S6: Input the spatio-temporal features of all ST-Blocks into the output module. The expression is as follows:

[0102]

[0103] Among them, X out is the input of the output module, H k is the output of the k-th ST-Block, and the predicted traffic flow data is obtained after linear transformation, which specifically includes the following steps:

[0104] Step S61: Input the input of the output module into the Relu function layer and the CNN layer in sequence for dimensionality conversion, and convert the matrix of dimension R N ×T×D into a matrix of R N×T′×D , where T’ is the time step length of the predicted traffic flow data.

[0105] Step S62: Input the result after dimensionality conversion into the auxiliary ST-Block and output the result. The only difference between the auxiliary ST-Block and the previous ST-Block is that the time step information therein is the time step information corresponding to the predicted traffic flow data.

[0106] Step S63: Input the result output in S62 into the linear layer. After dimensionality conversion, the predicted traffic flow data is obtained.

[0107] In this embodiment, the input data is a matrix of R N×T×D , where D is a custom dimension, N is the number of all nodes in the urban road network graph, T is the input time length, and D generally takes 64.

[0108] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it realizes the traffic flow prediction based on the improved GCN-attention algorithm as mentioned in the embodiments of the present invention. Any combination of one or more computer-readable media can be adopted. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0109] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art in this technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art should fall within the protection scope determined by the claims.

Claims

1. A traffic flow prediction method based on an improved GCN-attention algorithm, characterized in that, it includes the following steps: S1. Input historical traffic flow data into the GSTA model. The GSTA model includes several ST-Blocks and an output module. Each ST-Block includes a GCN sub-module, a time sub-module, and a gated fusion sub-module. The historical traffic flow data is input into one of the ST-Blocks; S2. Extract spatial features through the GCN sub-module; S3. Extract temporal features through the time sub-module; S4. Use the gated fusion sub-module to fuse the spatial features and temporal features to obtain spatio-temporal features; S5. Fuse the spatio-temporal features and the initial historical traffic flow data through a residual link as the input of the next ST-Block, and return to execute step S2 until the spatio-temporal features of all ST-Blocks are obtained; S6. Input the spatio-temporal features of all ST-Blocks into the output module, and obtain the predicted traffic flow data after linear transformation; Among them, the steps for the GCN sub-module to extract spatial features are as follows: S21. Establish an adjacency matrix of historical traffic flow data; S22. Perform information diffusion and information aggregation on the historical traffic flow data and its adjacency matrix in sequence, and link the outputs of all graph convolutional layers to obtain the spatial features of the urban road network; The expressions for the information diffusion and information aggregation are as follows respectively: Information diffusion: Among them, θ 1 , θ 2 , b 1 , b 2 are learnable parameters, H l is historical traffic flow data, m is the current convolutional layer number, is the output of the current convolutional layer, A is the adjacency matrix, and σ represents the sigmoid function; Information aggregation: Among them, S is the spatial feature, W and c are learnable parameters, and M is the total number of convolutional layers; The steps for the time sub-module to extract temporal features are as follows: S31. Obtain the time step information corresponding to the historical traffic flow data; S32. Perform embedding on the time step to obtain a time vector; S33. Calculate the attention vector according to the time vector and the historical traffic flow data; S34. Calculate the temporal features according to the attention vector; The calculation expression for the attention vector is as follows: Among them, Q, K, and V all represent an attention vector, k represents the k-th head of attention, vi represents a node in the urban road network, tj represents the j-th time step, and Relu is an activation function. They are all learnable parameter matrices; The specific calculation expression in step S34 is as follows: d = D / K Among them, is the timing representation vector, is the timing feature, is the correlation parameter, is the attention score, D represents the hidden dimension of the input matrix, and K represents the number of attentions in the multi-head attention, is the learnable parameter.

2. A traffic flow prediction method based on an improved GCN-attention algorithm according to claim 1, characterized in that, the adjacency matrix is an adaptive adjacency matrix and is updated by the gradient descent method.

3. A traffic flow prediction method based on an improved GCN-attention algorithm according to claim 1, characterized in that, the time step information is the total time of a day divided by the frequency of the traffic flow sensor collecting signals.

4. A traffic flow prediction method based on an improved GCN-attention algorithm according to claim 1, characterized in that, the calculation expression for the fusion in step S4 is as follows: Among them, represents the spatial feature, represents the temporal feature, g represents the mixing ratio, b g , W g are learnable parameters.

5. A traffic flow prediction method based on an improved GCN-attention algorithm according to claim 1, characterized in that, the output module executes the following steps: S61. Input the input of the output module into the Relu function layer and the CNN layer in sequence for dimension conversion; S62. Input the result after dimension conversion into the secondary ST-Block, and output the result. The time step information in the secondary ST-Block is the time step information of the predicted traffic flow data; S63. Input the result output in S62 into the linear layer to obtain the predicted traffic flow data.

6. A traffic flow prediction method based on an improved GCN-attention algorithm according to claim 1, characterized in that the calculation expression of the input of the output module in step S6 is as follows: Among them, X out is the input of the output module, and H k is the output of the k-th ST-Block.

Images