Traffic flow prediction method based on graph attention mechanism and bidirectional gated recurrent unit

Abstract

The application discloses a traffic flow prediction method based on a graph attention mechanism and a bidirectional gated recurrent unit, and is realized through four steps of capturing spatial dependence features, capturing time dependence features, full connection operation and result prediction.The application is superior to other benchmark models in the extraction of space-time features and achieves good results in long-time traffic flow prediction.The traffic prediction results of the application model at different nodes in the real road network are good, the space-time dependence features of the traffic road network can be fully captured, the effectiveness of the application model in long-time traffic flow prediction is better reflected, and the real vehicle speed change trend can be reflected.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of traffic monitoring, and particularly relates to a traffic flow prediction method based on a graph attention mechanism and a bidirectional gated recurrent unit. BACKGROUND

[0002] With the acceleration of urbanization, a large number of people are rapidly gathering in cities, and the rapid increase in the number of private cars and the growth in demand for public transportation services have brought great pressure on the current transportation system. In order to build a smart city and solve complex traffic problems, an intelligent transportation system has emerged, which combines emerging technologies such as the Internet of Things, artificial intelligence and 5G, and monitors vehicles in real time, and through early intervention measures, intelligently allocates traffic networks to alleviate traffic problems. In recent years, graph neural networks have become a frontier technology in deep learning research. Zhao L et al. use graph convolutional networks (GCN) to perform Laplace transformation on the adjacency matrix of the traffic network to capture spatial dependencies.

[0003] However, GCN is based on spectral methods (Spectral) to perform eigenvalue decomposition on the symmetric adjacency matrix of an undirected graph, which is not suitable for the structure of a directed graph. Kong X et al. construct an adaptive adjacency matrix of 1 for the entire network, and use a gated time convolutional network to extract time series features, however, when constructing the adaptive adjacency matrix, they assign weights between a large number of redundant nodes, thereby generating a large amount of redundant information, which is not conducive to the extraction of spatial features.

[0004] Zheng H et al. use CNN to extract spatial features, considering spatial information as two-dimensional Euclidean data composed of latitude and longitude, and performing deep convolution in the form of an image. This method has the advantages of parallel computing, gradient stability and low memory requirements; however, when the data is represented by a matrix, the spatial topology of the traffic network may be lost, which shows the limitations of CNN for traffic flow prediction. Existing models ignore the spatial features of the underlying network and cannot reflect the real vehicle speed change trend. SUMMARY

[0005] The purpose of the present application is to provide a traffic flow prediction method based on a graph attention mechanism and a bidirectional gated recurrent unit, which outperforms other benchmark models in the extraction of spatio-temporal features and can reflect the real vehicle speed change trend.

[0006] In order to achieve the above-mentioned application purpose, the technical scheme adopted by the present application is as follows:

[0007] The traffic flow prediction method based on a graph attention mechanism and a bidirectional gated recurrent unit comprises a spatial model and a temporal model combined into a spatio-temporal block, and a residual mechanism is used to design a multi-layer spatio-temporal block stack, wherein the spatial model comprises the following steps:

[0008] 1) GAT is used to capture the spatial dependence features of the real road network, and SAdpGAT is used to capture the spatial dependence features of the potential road network;

[0009] 2) The spatial dependence features are input into the BiGRU to capture the time dependence features;

[0010] 3) The output features of the spatio-temporal block are subjected to full connection operation;

[0011] 4) The prediction results of the real road network and the potential road network are connected to obtain the final prediction result of the model.

[0012] Further, when extracting the spatial dependence features, an adaptive adjacency matrix is constructed using the adjacency matrix structure of the real road network, and Xavier initialization is performed, that is, A i,j [Xavier], the node is N*N; then the initial time sequence is input into the GAT network after linear transformation with the adaptive adjacency matrix as a condition; finally, the correlation between nodes is calculated by using an attention score function.

[0013] Further, the time features are extracted based on the bidirectional gate recurrent unit, which is composed of a forward hidden layer and a reverse hidden layer.

[0014] As a preferred, BatchNorm2d is used to normalize the output results of each spatio-temporal module, and then Relu activation function is used to accelerate the convergence of the model.

[0015] As a preferred, the output results of the spatio-temporal modules of the real road network and the potential road network are input into a full connection layer for dimension transformation, and then the prediction results of the two paths are weighted and summed to obtain the final output result of the model.

[0016] As a preferred, the calculation formula of the final output result of the model is:

[0017] Compared with the prior art, the beneficial effects of the present application are:

[0018] 1) For the benchmark model of spatio-temporal combination, STGCN extracts spatio-temporal information by combining graph convolution and one-dimensional time convolution; DCRNN inputs the spatial dimension processed graph convolution information into the recurrent neural network in the form of encoder-decoder to extract spatio-temporal dependence features; both of them ignore the spatial features of the potential road network, and the model can only extract fixed structure spatial information; therefore, the model of the present application is better than other benchmark models in the extraction of spatio-temporal features, and good results are achieved in long-time traffic flow prediction.

[0019] 2) The traffic prediction results of the model at different nodes in the real road network are good, the space-time dependence characteristics of the traffic road network can be fully captured, the effectiveness of the model in long-time traffic flow prediction is better reflected, the real vehicle speed change trend can be reflected, and the changing traffic conditions can be predicted. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 It is a structural schematic diagram of the SAdpGAT-BiGRU model of the application.

[0021] Figure 2 It is a structural schematic diagram of the BiGRU network.

[0022] Figure 3 It is a three-index column chart of the model and the benchmark model.

[0023] Figure 4 It is a real value and predicted value curve diagram of different nodes at different time steps. DETAILED DESCRIPTION

[0024] Embodiments of the application will be described in more detail below with reference to the accompanying drawings. Although embodiments of the application are shown in the drawings, it should be understood that the application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the application more thorough and complete, and to fully convey the scope of the application to those skilled in the art. Those skilled in the art can obtain alternative technical solutions from the following description without deviating from the spirit and protection scope of the application.

[0025] The traffic flow prediction method based on graph attention mechanism and bidirectional gated recurrent unit includes a space model and a time model combined into a space-time block, and a residual mechanism is used to design a multi-layer space-time block stack, as shown in Table 1: the space model includes the following steps:

[0026] 1) GAT is used to capture the spatial dependence characteristics of the real road network, and SAdpGAT is used to capture the spatial dependence characteristics of the potential road network;

[0027] 2) The spatial dependence characteristics are input into BiGRU to capture the time dependence characteristics;

[0028] 3) Full connection operation is performed on the output characteristics of the space-time block;

[0029] 4) The prediction results of the real road network and the potential road network are connected to obtain the final prediction result of the model.

[0030] The spatial dependence and the time dependence have potential interaction, and each module has real-time dynamics due to the uncertainty of traffic conditions. In extracting the spatial dependence feature, an adaptive adjacency matrix is constructed using the adjacency matrix structure of the real road network, and Xavier initialization is performed, that is, A i,j [Xavier], the node is N*N; then the initial time sequence is input into the GAT network after linear transformation with the adaptive adjacency matrix as a condition; and finally, the correlation between nodes is calculated by using an attention score function.

[0031] The graph attention network is a network based on the attention mechanism in the field of graph neural networks, which does not depend on the overall structure of the graph, but only focuses on the correlation between nodes. Therefore, the spatial dependence feature can be better extracted, and the generalization performance of the model can be improved. The implementation of GAT is as follows: the node feature is input, where N represents the number of nodes, F represents the feature of each node, and the node feature output by the model is The self-attention mechanism is used to calculate the attention coefficient of node i to node j, and the formula is as follows:

[0032]

[0033] wherein, indicates a learnable weight parameter, indicates a connection operation, W is in R F′×F indicates a shared weight matrix, a leaky rectified linear unit (Leaky ReLU) activation function is used to normalize the linear transformation, and the problem of gradient disappearance of the ReLU function when the independent variable is negative is solved.

[0034] In order to calculate the attention coefficient of only the node j directly adjacent to the node i, a masked softmax operation is used, and the calculation formula is as follows:

[0035]

[0036] wherein, indicates the domain node set of node i; the node feature is calculated according to the normalized attention coefficient, and the node feature is calculated according to the normalized attention coefficient, and the formula is as follows:

[0037] wherein, sigma indicates a nonlinear activation function, W indicates a shared weight matrix, indicates the new feature of the node.

[0038] In order to stabilize the learning process, the multi-head attention mechanism is used to calculate the point feature, and K represents the number of attention heads.

[0039] The model of the application uses the connection operation in the hidden layer, and the formula is as follows:

[0040] The model uses a connection operation in the hidden layer, the formula of which is as follows:

[0041] The mean operation is used in the output layer, the formula of which is as follows:

[0042] The Gated Recurrent Unit (GRU) is a simplified variant of the Long-short-term Memory (LSTM) and can achieve almost the same effect as the LSTM, and can greatly improve the training efficiency of the model and solve the problems of gradient disappearance and gradient explosion. In the GRU network, the reset gate helps to capture short-term dependency features; the update gate helps to capture long-term dependency features.

[0043] The time feature is extracted based on the bidirectional gated recurrent unit, which is composed of a forward hidden layer and a reverse hidden layer. The problem that the GRU can only forward propagate is solved, and the time-dependent features can be better extracted. The BiGRU network structure is as follows:

[0044]

[0045] wherein, represents the output of the forward hidden layer, represents the output of the reverse hidden layer.

[0046] As a preferred, the output result of each spatio-temporal module is normalized using BatchNorm2d, and then the Relu activation function is used to accelerate the convergence of the model, the formula of which is as follows:

[0047] O ST =Relu(BatchNorm2d(O t ))

[0048] As a preferred, the output results of the spatio-temporal modules of the real road network and the potential road network are input into the full connection layer for dimension transformation, and then the prediction results of the two paths are weighted and summed to obtain the final output result of the model.

[0049] As a preferred, the calculation formula of the final output result of the model is as follows:

[0050] wherein, W1, W2 are weight coefficients, represents the output result of the existing road network, represents the output result of the potential road network, which is specifically as shown in the following formula. Figure 2

[0051] Experimental example

[0052] ​Since the traffic network is a topological structure reflected on the non-Euclidean distance, according to the characteristics, the application constructs the adjacency matrix of the traffic network, and according to the actual distance between the nodes of the network, the correlation between the nodes of the adjacency matrix is calculated using the Gaussian kernel function, and the adjacency matrix is generated, and the formula is as follows:

[0053]

[0054] Wherein, sigma represents variance, k represents threshold value, generally set as 0.1.

[0055] The time series of traffic flow is standardized, and the specific formula is as follows:

[0056]

[0057] Wherein, mu is the mean, and sigma is the variance.

[0058] The model of the application adopts 4-layer space-time block stacking. GAT is introduced in the spatial dimension, multi-head attention mechanism is used, the first three layers adopt series multi-head attention, and the last layer adopts average multi-head attention. BiGRU is introduced in the time dimension. The specific parameter setting of the model is shown in Table 1, the Adam optimizer is used to optimize the model parameters, and the clip_grad_norm_ method is used for gradient clipping to prevent gradient explosion.

[0059]

[0060] The application makes a comparative experiment on the design of the stacking number of space-time blocks, and it is known from Table 2 that when predicting the traffic speed in the future 60 min, the prediction performance of the model is best when the residual block is stacked for 4 layers.

[0061]

[0062] Since the application adopts the multi-head attention mechanism, comparative experiments are made on the selection of K, and the most suitable number of attention heads is selected according to the spatial characteristics of the adjacency matrix. It is known from Table 3 that when the number of heads of the multi-head attention is 6, the prediction performance of the model is best.

[0063]

[0064] Evaluation index:

[0065] The loss function is used to evaluate the error between the predicted value of the model and the true value of the data set. The loss function selected by the present application is mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE). The formulas of the three loss functions are as follows:

[0066]

[0067]

[0068] wherein n represents the number of predicted samples, represents the predicted value of the model, Y i represents the true value of the data set.

[0069] The model compares the prediction performance of 15min, 30min and 60min with other benchmark models of the same data set, and the experimental results are shown in Table 4. It can be seen from the table that the model of the present application has the lowest index value of MAE, RMSE and MAPE, and the prediction accuracy is obviously improved. The percentage increase of the loss function index value of the prediction results of the model of the present application and other benchmark models in 60min is shown in Table 5. Since the traditional time series model (ARIMA, FC-LSTM) only considers the extraction of time dimension features and does not consider the spatial dimension features of the traffic network, the prediction performance of the model is not as good as that of the model combining time and space characteristics, therefore, the prediction performance of the model of the present application is obviously better than that of these time series models.

[0070] For the benchmark model combining space and time, STGCN extracts space-time information by combining graph convolution and one-dimensional time convolution; DCRNN inputs the graph convolution information processed in the spatial dimension into the recurrent neural network in the form of encoder-decoder to extract space-time dependent features; both of them ignore the spatial features of the potential road network, and the model can only extract fixed structure spatial information; therefore, the present application model is better than other benchmark models in extracting space-time features, and good results have been achieved in long-term traffic flow prediction.

[0071] In order to verify the usefulness of the two road networks of the model of the present application in capturing traffic flow information, it can be seen from the prediction results of the last three models in Table 4 that the prediction results of the real road network and the potential road network have decreased, but the effective combination of the traffic road network information extracted from the two can obtain better prediction results.

[0072]

[0073]

[0074] As can be seen from Figure 3 It can be seen that the prediction effect of the model of the present application is most obviously improved at 60 min of prediction with the increase of the prediction time step, which also proves the advantage of the model of the present application in long-time traffic flow prediction compared with other benchmark models.

[0075] In order to further study the prediction accuracy of the model of the present application at different steps, three nodes (0, 60 and 80) in the PEMS-BAY data set are randomly selected, and the curve fitting graphs of the real data and the predicted data of the three nodes on May 26, 2017 are drawn, as shown in FIG. 7.

[0076] As can be seen from Figure 4 It can be seen that the prediction value and the real value of the three nodes are slightly decreased in the speed curve fitting degree with the increase of the prediction time step, but the fluctuation amplitude of the curve is consistent as a whole, which shows that the traffic prediction results of the model of the present application at different nodes in the real road network are good, can fully capture the space-time dependence characteristics of the traffic road network, and more reflect the effectiveness of the model of the present application in long-time traffic flow prediction, can reflect the real vehicle speed change trend, and is helpful for predicting the changing traffic conditions.

[0077] The above merely describes specific embodiments of the present disclosure, but the protection scope of the embodiments of the present disclosure is not limited thereto, and any person skilled in the art can easily think of changes, replacements or combinations within the technical range disclosed by the embodiments of the present disclosure or under the idea disclosed by the embodiments of the present disclosure, which should be covered within the protection scope of the embodiments of the present disclosure.

Claims

1. A traffic flow prediction method based on graph attention mechanism and bidirectional gated recurrent unit, comprising a combination of a spatial model and a temporal model into a spatio-temporal block, a multi-layer spatio-temporal block stack is designed using a residual mechanism, characterized in that The space model comprises the following steps: 1) GAT captures the spatial dependence features of the real road network, and SAdpGAT captures the spatial dependence features of the potential road network. When extracting spatial dependence features, the adjacency matrix structure of the real road network is used to construct an adaptive adjacency matrix, which is initialized by Xavier, that is, A i,j [Xavier], the node is N*N; then the initial time sequence is input into the GAT network after linear transformation with the adaptive adjacency matrix as the condition; finally, the correlation between nodes is calculated by using the attention score function; 2) input the space-dependent features into the BiGRU to capture time-dependent features; 3) perform full connection operation on the output features of the space-time block, use BatchNorm2d to normalize the output results of each layer of the space-time module, and then use the Relu activation function to accelerate the convergence of the model; 4) connecting the prediction results of the real road network and the potential road network to obtain the final prediction result of the model, and the calculation formula of the final output result of the model is:

2. The traffic flow prediction method based on graph attention mechanism and bidirectional gated recurrent unit according to claim 1, characterized in Based on the bidirectional gate recurrent unit to extract time features, composed of forward hidden layer and reverse hidden layer.

3. The traffic flow prediction method based on graph attention mechanism and bidirectional gated recurrent unit according to claim 1, characterized in The output results of the space-time module of the real road network and the potential road network are input into the full connection layer for dimension transformation, and then the prediction results of the two paths are weighted and summed to obtain the final output result of the model.

Images