Traffic flow prediction method based on interactive dynamic graph convolution and probabilistic sparse attention

Abstract

The traffic flow prediction method based on interactive dynamic graph convolution and probabilistic sparse attention, the traffic flow prediction method, the IDG-PSAtt method, consists of an interactive dynamic graph convolution network IDGCN, a spatiotemporal convolution block ST-Conv Block, and a probabilistic sparse self-attention mechanism ProbSSAtt Block; step 1, IDGCN divides the traffic flow time series into intervals and interactively shares the captured dynamic spatiotemporal features; step 2, ST-Conv Block captures the complex temporal dependence of traffic flow at the same location and the dynamic spatial correlation of traffic flow at adjacent locations at the same time step; step 3, ProbSSAtt Block captures dynamic spatiotemporal features to improve the medium- and long-term prediction performance of the IDG-PSAtt method; step 4, a dynamic graph convolution network generated by the fusion of an adaptive adjacency matrix and a learnable adjacency matrix is ​​constructed to learn the hidden dynamic associations between road network nodes; step 5, traffic flow prediction is performed through the prediction layer.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a traffic flow prediction technology based on interactive dynamic graph convolution and probabilistic sparse attention. Background Art

[0002] With the rapid development of urbanization and the increasing complexity of road networks, accurate traffic flow forecasting has become an indispensable component of intelligent transportation systems (ITS). Traffic flow forecasting aims to predict future traffic flows based on a given historical traffic time series. Accurate traffic flow forecasting is crucial for transportation services such as route planning, alleviating traffic congestion, and managing complex transportation networks.

[0003] Traffic flow prediction has long been a research hotspot in the ITS field. However, due to the complexity, non-Euclidean nature, and dynamic nature of road networks, accurate and efficient traffic flow prediction remains a significant challenge. Traffic flow is time series data with complex temporal dependencies and dynamic spatial correlations, as well as unique periodicity, such as morning and evening rush hours. Accurate traffic flow prediction requires methods that can fully capture the long-term dependencies and dynamic spatiotemporal correlations between input and output.

[0004] First, the difficulty in capturing complex time dependencies makes long-term traffic flow prediction difficult. For example, when predicting the traffic conditions for the next 12 time steps based on the traffic flow of 12 historical time steps, the traffic flow prediction for the 9th to 12th time steps is usually more difficult than that for the 1st to 3rd time steps. Second, due to the spatial heterogeneity, dynamic correlation and uncertainty of the traffic network, there are hidden dynamic spatiotemporal characteristics in the traffic flow, which also poses challenges to accurate traffic flow prediction. Figure 1 As shown in Figure 2, due to the complexity of the transportation network, the dynamic spatiotemporal relationship of traffic flow is also extremely complex. Figure 1 Traffic conditions across the display space can influence each other and change dynamically. For example, a traffic accident in one area may affect traffic conditions on nearby roads. Furthermore, traffic flows in different directions on the same road may also vary. Figure 1 Represents the hidden dynamic spatial characteristics of traffic flow, such as spatial heterogeneity, dynamic correlation, and uncertainty. Spatial heterogeneity refers to the different traffic patterns and spatiotemporal characteristics of different regions, such as residential and industrial areas. Dynamic correlation refers to the time-varying associations between road network nodes, which can be learned from given historical traffic data and road network structure. Uncertainty refers to the impact of traffic accidents, holidays, inclement weather, and other factors on traffic flow.

[0005] Traditional machine learning methods rely heavily on feature engineering, resulting in significant limitations in traffic flow prediction. In contrast, deep learning-based methods can effectively and automatically capture features that characterize traffic flow characteristics and have been widely used in traffic flow prediction. For example, to capture the spatiotemporal correlations of traffic flow, STGCN employs a graph convolutional network (GCN) and a gated linear unit (GLU) to capture spatial features and temporal dependencies, respectively. Graph WaveNet uses adaptive graph convolutions and a temporal convolutional network (TCN) to capture spatiotemporal correlations. SLCNN uses dynamic graph convolutions and a one-dimensional CNN to explore the spatiotemporal characteristics between traffic flow nodes. STFGNN employs multiple GCNs and a one-dimensional CNN to simultaneously extract spatiotemporal correlations. These aforementioned approaches employ serial or parallel architectures to extract dynamic spatiotemporal features, but this can weaken the captured spatiotemporal correlations and even amplify unimportant information, resulting in poor traffic flow prediction performance. Therefore, ASTGCN employs spatial and temporal attention mechanisms to further improve prediction performance. ASTGNN employs dynamic graph convolutions to capture spatial features and learns the temporal dependencies of traffic flow through an attention mechanism. Furthermore, AGCRN combines adaptive GCN with gated recurrent units (GRUs), embedding spatial modules within temporal modules to simultaneously capture complex temporal dependencies and dynamic spatial features. MRA-BGCN introduces an attention mechanism within the embedded structure to further extract dynamic spatiotemporal features. While these methods have improved their ability to capture the dynamic spatiotemporal features of traffic flow, the interactive learning capabilities of the spatiotemporal modules used to extract dynamic spatiotemporal features are poor, affecting the traffic flow prediction method's perception of the periodicity and changing trends of time series and failing to fully capture the dynamic spatiotemporal features of traffic flow.

[0006] Furthermore, many current studies have also defined various adjacency matrices to represent the deep structure of traffic flow and capture hidden dynamic spatial features. For example, MTGNN uses GCN, one-dimensional CNN, and an adaptive adjacency matrix to learn hidden spatial features, while STSGCN combines multiple adjacency matrices and employs an embedded structure for traffic flow prediction. Adaptive adjacency matrices can explore hidden relationships between road network nodes to improve the method's learning of the spatial heterogeneity of traffic flow. However, once training stops, the adaptive adjacency matrix cannot learn the dynamic associations between graph nodes over time, resulting in the method's inability to fully utilize historical traffic flows. Therefore, these methods still cannot fully and effectively capture the hidden spatial features of traffic flow. Summary of the Invention

[0007] The purpose of the present invention is to better capture the dynamic spatiotemporal characteristics of traffic flow.

[0008] The present invention is a traffic flow prediction method based on interactive dynamic graph convolution and probabilistic sparse attention. The traffic flow prediction method, IDG-PSAtt, combines an interactive dynamic convolution structure with spatiotemporal convolution and a probabilistic sparse self-attention mechanism to fully capture the dynamic spatiotemporal characteristics of traffic flow time series. The interactive dynamic graph convolution uses interactive learning planning to divide traffic flow data into intervals, and then synchronously captures the spatiotemporal dependencies of the divided sequences and shares the spatiotemporal features learned between the sequences. The probabilistic sparse self-attention mechanism adjusts the attention coefficient so that a few key point products in the traffic flow provide the main attention to reduce computational complexity and improve the computational efficiency of the method. The interactive learning strategy and the probabilistic sparse self-attention mechanism enable the IDG-PSAtt method to effectively perform long-term predictions. A dynamic graph convolution network is constructed through a dynamic graph generation method to capture the hidden dynamic associations between traffic flow nodes, thereby capturing the dynamic spatial correlation of the traffic network. Finally, a gated fusion mechanism is used to adaptively fuse the dynamic spatiotemporal features extracted by the multi-head probabilistic sparse self-attention mechanism to reduce the propagation of errors in the prediction process and improve the prediction accuracy.

[0009] The present invention has the following advantages:

[0010] 1. We propose a traffic flow prediction method based on interactive dynamic graph convolution and probabilistic sparse attention, IDG-PSAtt. This method embeds dynamic graph convolution into an interactive learning structure and inherits the advantages of spatiotemporal convolution and probabilistic sparse self-attention mechanism. It learns spatiotemporal dependencies through the interactive learning structure and spatiotemporal convolutional network, and uses the probabilistic sparse self-attention mechanism to achieve effective long-range prediction.

[0011] 2. Construct a dynamic graph convolutional network to capture spatiotemporal features. The network is generated by fusing an adaptive adjacency matrix and a learnable adjacency matrix. The adaptive adjacency matrix can capture the heterogeneity of traffic flow time series, and the learnable adjacency matrix can learn the dynamic associations between road network nodes.

[0012] 3. Construct a spatiotemporal convolutional network and introduce a probabilistic sparse self-attention mechanism. The spatiotemporal convolutional network learns the hidden spatial features between different nodes at the same time and the complex temporal dependencies of the same node at different times. The probabilistic sparse self-attention mechanism combines dynamic spatiotemporal features and adjusts the attention coefficient to improve the computational efficiency of the method.

[0013] 4. A large number of comparative experiments were conducted on two sets of traffic data sets. The experiments showed that compared with the existing baseline methods, the method of the present invention achieved the best prediction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is the dynamic spatiotemporal correlation graph of traffic flow, Figure 2It is the structure diagram of the spatiotemporal graph convolutional network. Figure 3 This is the overall framework diagram of IDG-PSAtt. Figure 4 This is the IDGCN structure diagram, Figure 5 It is the DGCN structure diagram, Figure 6 This is the ST-Conv Block framework diagram, Figure 7 This is a comparison chart of MAE indicators on two sets of data sets. Figure 8 This is a comparison chart of MAPE indicators on two sets of data sets. Figure 9 This is a comparison chart of the RMSE indicators on the two sets of data. Implementation Method

[0015] The present invention is further described in detail below with reference to the embodiments.

[0016] Step 1: Method

[0017] The present invention proposes a traffic flow prediction method IDG-PSAtt based on interactive dynamic graph convolution and probabilistic sparse attention, which is characterized by the following construction steps:

[0018] Step 1.1: Prepare the raw traffic flow data by feeding it into the Start Conv layer. The Start Conv layer obtains the data in a high-dimensional space and captures the traffic flow dependencies at a deeper level.

[0019] Step 1.2: Input the data represented in the high-dimensional space into the interactive dynamic graph convolution (IDGCN). IDGCN uses interleaved sampling to recursively generate two subsequences of equal size. The subsequence length is halved. IDGCN interactively learns the two subsequences, and the two subsequences share their learned traffic flow features.

[0020] Step 1.3: Embed DGCN into the interactive learning structure. Based on the dynamic graph convolution DGCN, the high-dimensional features extracted by the Start Conv layer are processed by using the interactive learning strategy, and the spatiotemporal features are extracted using IDGCN. After the spatiotemporal features are extracted, two subsequences are output.

[0021] Step 1.4: The output subsequences pass through the concatenation fusion module. In the concatenation, all the output subsequences are reorganized in time index order, and the reorganized sequences are fed into the diffusion graph convolution.

[0022] Step 1.5: Input the dynamic spatial features output by Diffusion Graph Conv into the multi-head probabilistic sparse self-attention layer ProbSSAtt Block to extract dynamic spatiotemporal features;

[0023] Step 1.6: Input the dynamic spatiotemporal features output by ProbSSAtt Block into ST-Conv to extract temporal features;

[0024] Step 1.7: Input the spatiotemporal features output by ST-Conv into the multi-layer perceptron (MLP), and aggregate the extracted dynamic spatiotemporal features through the MLP.

[0025] Step 1.8: Construct a prediction layer, input the dynamic spatiotemporal features output by the MLP into the prediction layer, and perform traffic flow prediction through the prediction layer.

[0026] Step 2: Problem Definition

[0027] Step 2.1: Represent the traffic road network as a graph G = (V, E, A), where |V| = N is a set of nodes, each node represents an observation sensor in the road network; E is a set of edges between nodes, whose weights are represented by the distances between nodes; Represents the initial adjacency matrix generated by graph G. If v i ,v j ∈V and (v i ,v j )∈E, then A ij is 1, otherwise it is 0; the initial adjacency matrix A generated by the original traffic network is used as prior knowledge, and the historical time series To predict future traffic flow in represents the observed value of graph G at time t, C represents the number of feature channels, T′ represents the length of the given historical time series, and T represents the length of the predicted future traffic series. The mapping relationship of the traffic flow prediction problem can be expressed as follows:

[0028]

[0029] Where f represents the prediction function that can predict future traffic flow based on a given historical time series.

[0030] Step 2.2: Implement the interactive learning module through convolutional neural network (CNN) and graph convolutional network (GCN). Use interleaved sampling to process the original data to achieve multi-resolution analysis and expand the receptive field. The interactive learning module consists of three identical interactive dynamic graph convolutional networks (IDGCNs). The structure of IDGCN is as follows: Figure 4 As shown in the figure; in IDGCN, the two subsequences interactively learn their respective dynamic spatiotemporal features, and each subsequence preprocesses the features through a convolution to expand the receptive field; the two subsequences share parameter weights in DGCN and capture each other's dynamic spatiotemporal features.

[0031] Step 2.3, Represents the input of IDGCN. After interleaved sampling, X obtains two subsequences, which are odd sequences even sequence Conv1, Conv2, Conv3, and Conv4 in the interactive dynamic graph convolution represent 1D convolution operations; the output of the first interactive learning of IDGCN is X o ' dd and X e ' ven The final output sequence is obtained through further interactive learning The specific calculations in interactive dynamic graph convolution are as follows:

[0032] X even ,X odd =Split(X) (2)

[0033] X′ odd =tanh(DGCN(Conv1(X even )))⊙X odd (3)

[0034] X e ' ven =tanh(DGCN(Conv2(X odd )))⊙X even (4)

[0035] X odd_out =X′ odd +tanh(DGCN(Conv3(X′ even ))) (5)

[0036] X even_out =X′ even +tanh(DGCN(Conv4(X′ odd ))) (6)

[0037] represents the Hadamard product, tanh represents the activation function, and DGCN represents the dynamic graph convolutional network in IDGCN.

[0038] Step 2.4, dynamic graph convolution DGCN is mainly composed of diffusion graph convolution network and graph generation module, such as Figure 5 As shown in the figure, DGCN uses diffusion graph convolution and graph generation modules to better learn deep dynamic spatial features. DGCN transforms hidden features into and the predefined initial adjacency matrix Feed it as input to the diffusion graph convolutional network, which is fed into the graph generator and the multi-layer perceptron layer to generate a discrete matrix containing spatiotemporal information A′ is expressed as follows:

[0039] A′=SoftMax(MLP(GCN(H,A))) (7)

[0040] Among them, GCN represents the diffusion convolution and graph generator operation, and MLP represents the multi-layer perceptron.

[0041] Step 2.5: Use Gumbel reparameterization to ensure that the sampling process is differentiable during training:

[0042]

[0043] Where g~Gumbel(0,1) represents a random variable, τ is the softmax temperature parameter with a value of 0.5, and A learn Represents the adjacency matrix generated by the graph generator that can simulate the dynamic dependencies between nodes.

[0044] Step 2.6: Build an adaptive adjacency matrix As shown below:

[0045]

[0046] in represents the learnable parameters, A apt The initial value of is the adjacency matrix predefined based on the original graph data

[0047] Step 2.7, by using the adaptive fusion module to learn and A apt Fusion, and then the resulting dynamic adjacency matrix Feed it into the diffusion graph convolutional network to extract the hidden dynamic spatiotemporal correlation in the traffic road. The calculation of this fusion module is as follows:

[0048] A dyn =αA apt +(1-α)A learn (10)

[0049] Where α represents the learnable adaptive parameter factor.

[0050] Step 2.8: Diffusion graph convolution is used in the graph generator network, fusion graph convolution and serial fusion modules, and the input of diffusion graph convolution is uniformly defined as In the graph generator network, the diffuse graph convolution is defined as:

[0051]

[0052] Among them, k is the diffusion step size, K is the maximum number of diffusion steps, and W represents the parameter matrix; in the fusion graph convolution module, A dyn is the adjacency matrix of the input of the fused graph convolution, where the diffusion graph convolution is expressed as follows:

[0053]

[0054] The IDG-PSAtt method captures and corrects the features of the entire time series by recombining the dynamic spatiotemporal features extracted from the interactive learning structure in the tandem module in the order of time index and feeding them into the Diffusion Graph Conv.

[0055] Step 2.9: In the diffusion graph convolutional network, a predefined initial adjacency matrix is ​​used. and a dynamic adjacency matrix generated by an interactive learning structure For the initial adjacency matrix A, use the directed graph and use P respectively f =A / rowsum(A) and P b =A T / rowsum(A T ) represents the forward and backward transfer matrices of A; the diffusion graph convolution in the serial fusion module is represented as follows:

[0056]

[0057] The DGCN module can not only extract deep hidden spatial features by exploring the invisible dependencies between nodes in the traffic network, but also generate dynamic associations between data based on the input traffic flow time series simulation; in addition, by embedding DGCN into an interactive learning framework, it is possible to fully utilize the dynamic spatial information captured by GCN during the training process to enhance the capture of complex temporal dependencies of traffic flow.

[0058] Step 2.10, construct a spatiotemporal convolution module consisting of three convolution kernels, such as Figure 6 As shown in the figure, the three convolution kernels correspond to the three angles of time, space and time and space respectively to capture the spatiotemporal features extracted from the diffusion graph convolution module, and capture the influence of multiple node features on single node features in the topological graph structure of traffic flow; the time kernel captures the dependence of traffic flow at different times at the same location, and the space kernel captures the spatial correlation of traffic flow at adjacent locations at the same time step; each spatiotemporal convolution block takes the output of the previous spatiotemporal attention block as input The output can be calculated

[0059]

[0060]

[0061] in, is the time kernel of f×1, is the spatial kernel of 1×f, is an f×f spatiotemporal kernel, LeakyReLU(·) represents the Leaky rectified linear unit function, and * represents the convolution operation; the outputs of the three convolution kernels are connected and a 1×1 convolution is used. To compress features and limit the number of channels.

[0062] Step 2.11. The input of the self-attention mechanism is in the form of (Q, K, V), and the dot product is calculated as follows:

[0063]

[0064] in, and d represent the input query, key, value, and dimension respectively; the attention coefficient A(q i ,K,V) are as follows:

[0065]

[0066] Among them, q i , k i and v i are the i-th row in Q, K and V respectively; and k(q i ,k l ) using an asymmetric exponential kernel

[0067] Step 2.12, spatiotemporal convolution probability sparse self-attention mechanism STC-ProbSSAtt adopts M(q i ,K) represents the sparsity of the i-th query, and KL divergence is used to measure the sparsity of the query, as shown below:

[0068]

[0069] The first item is q on all keys i The logarithm and exponential of , the arithmetic mean of all keys in the second term, and the probabilistic sparse self-attention mechanism are calculated as follows:

[0070]

[0071] in, represents a sparse matrix of the same dimension as q, consisting only of the first u queries at the sparsity evaluated by M(q,K), u = c lnL QControlled by a constant sampling factor c; the complexity of the probabilistic sparse self-attention mechanism for each query is only O(lnL Q ), a multi-head probabilistic sparse self-attention mechanism is adopted, which can generate different sparse query key pairs to avoid severe information loss.

[0072] Step 2.13: By constructing a prediction layer, the dynamic spatiotemporal features output by the multi-layer perceptron (MLP) are input into the prediction layer, and traffic flow prediction is performed through the prediction layer.

[0073] Step 3: Experiment

[0074] Step 3.1. The prediction performance of the STC-ProbSSAtt method was verified on the public transportation datasets METR-LA and PEMS-BAY. METR-LA is composed of traffic speed statistics recorded by 207 sensors on Los Angeles County freeways over a period of four months; PEMS-BAY is composed of traffic speed information recorded by 325 sensors on traffic roads in the San Francisco Bay Area over a period of six months. Both METR-LA and PEMS-BAY record the detection location, detection date, data type, etc. In the experiment, the datasets were split into training set, test set, and validation set in a ratio of 7:2:1 in chronological order to predict 15-minute, 30-minute, and 60-minute traffic flow speeds. The details of the experimental datasets are shown in Table 1:

[0075] Table 1. Description of experimental datasets

[0076]

[0077] Step 3.2: Since some data are missing from the METR-LA dataset, linear interpolation is used to fill the missing values ​​in the experiment. Before the data is input into the prediction method, the data is normalized to the minimum and maximum values, limiting the data to [0, 1]. The normalization formula is:

[0078]

[0079] Among them, x i Represents the i-th original data, x min and x max Represent the minimum and maximum values ​​of the original data, respectively. Represents the normalized input data.

[0080] Step 3.3, Evaluation Metrics and Baseline Methods: Use mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) to evaluate the error between the actual traffic flow speed and the predicted results:

[0081] 1) Mean Absolute Error (MAE):

[0082]

[0083] 2) Root Mean Square Error (RMSE):

[0084]

[0085] 3) Mean Absolute Percentage Error (MAPE):

[0086]

[0087] Where N is the number of observations, y i and y i Represent the actual traffic speed and predicted traffic speed respectively; the smaller the predicted values ​​of MAE, RMSE and MAPE, the better the prediction effect of the IDG-PSAtt method.

[0088] The main comparisons are with deep learning methods and other baseline methods, including:

[0089] 1) Historical average method HA: The prediction is based on historical average traffic flow information;

[0090] 2) Vector Autoregression VAR: n variables in the same sample period can be treated as linear functions of their historical values;

[0091] 3) Support Vector Regression (SVR): This method uses a linear vector machine training method based on support vector regression and predicts traffic flow based on the input-output relationship.

[0092] 4) Feedforward neural network FNN: A feedforward neural network that combines two hidden layers and L2 regularization;

[0093] 5) Autoregressive Integrated Moving Average (ARIMA): Autoregressive Integrated Moving Average (ARIMA) with Kalman filter;

[0094] 6) Long Short-Term Memory Network (FC-LSTM): A recurrent neural network with fully connected LSTM hidden units.

[0095] 7) WaveNet: Convolutional neural network for predicting sequence data;

[0096] 8) Graph WaveNet: A combination of graph convolutional networks and dilated causal convolutional networks;

[0097] 9) Spatial-Temporal Graph Convolutional Network (STGCN): a spatial-temporal graph convolutional network that combines graph convolution and 1D convolution.

[0098] 10) Attention-based Spatiotemporal Graph Convolutional Network (ASTGCN): Integrates the spatiotemporal attention mechanism into the spatiotemporal graph convolutional network to simultaneously capture the dynamic spatiotemporal characteristics of traffic flow;

[0099] 11) Spatiotemporal Synchronized Graph Convolutional Network (STSGCN): Captures spatiotemporal features by stacking multiple local GCN layers in the temporal direction.

[0100] Step 3.4. Experimental results and analysis

[0101] The performance of the IDG-PSAtt method was compared with 11 common baseline methods for 15-minute, 30-minute, and 60-minute predictions; on both datasets, the IDG-PSAtt method achieved the best prediction results in terms of all evaluation metrics.

[0102] The experimental results in Table 2 show that the statistical methods HA, VAR, ARIMA, and traditional machine learning methods SVR and FC-LSTM perform poorly because these methods only consider temporal dependencies and ignore the dynamic spatial characteristics of traffic flow; GCN-based methods can process non-Euclidean traffic data and more effectively capture the hidden relationships between road network nodes. Therefore, spatiotemporal GCN methods represented by STGCN and STSGCN perform better; although STSGCN can capture spatiotemporal features simultaneously, it uses a simple sliding window to capture temporal correlations and does not pay attention to the capture of temporal dependencies, resulting in its overall poor performance; since the attention mechanism can capture the temporal dependencies of sequences, the method based on the attention mechanism ASTGCN also performs well; Graph WaveNet embeds GCN into TCN, making its performance better than ASTGCN, but Graph WaveNet does not combine the self-attention mechanism to further capture hidden spatiotemporal features.

[0103] Compared with the most advanced methods, the IDG-PSAtt method has significantly improved the prediction results of 15 minutes, 30 minutes and 60 minutes on METR-LA and PEMS-Bay. The reason is that the IDG-PSAtt method fully captures the dynamic spatiotemporal characteristics of traffic flow through interactive learning structure, dynamic graph convolutional network and spatiotemporal convolution, and uses the probabilistic sparse self-attention mechanism to achieve effective long-range prediction. In the 15-minute, 30-minute and 60-minute predictions on METR-LA, the IDG-PSAtt method improves the state-of-the-art methods by 3.8 in MAE, RMSE and MAPE respectively. %, 8.0%, 5.0% and 9.2%, 13.7%, 8.78% and 17.2%, 15.1%, 9.6%; in the 15-minute, 30-minute and 60-minute predictions on PEMS-Bay, the IDG-PSAtt method improved the state-of-the-art methods by 29.4%, 41.9%, 32.2% and 9.4%, 12.0%, 8.3% and 15.5%, 12.1%, 16.1% in terms of MAE, RMSE and MAPE, respectively; it verified that the proposed IDG-PSAtt method can fully capture the dynamic spatiotemporal characteristics of traffic flow and make accurate traffic flow predictions.

[0104] The IDG-PSAtt method combines an interactive learning strategy with spatiotemporal convolution and a probabilistic sparse self-attention mechanism to effectively and synchronously capture dynamic spatiotemporal correlations. Compared with the baseline method, the IDG-PSAtt method can better capture the dynamic spatiotemporal correlations in each time period of the traffic flow and achieve the best prediction results in 15-minute, 30-minute, and 60-minute predictions. It proves that the IDG-PSAtt method can explore invisible dynamic associations in road network nodes to capture hidden spatial correlations. As the prediction period increases, the prediction difficulty also increases. However, as shown in Table 2, the prediction effect of the IDG-PSAtt method in long-term predictions is still very good, further verifying the effectiveness of the interactive learning strategy of the IDG-PSAtt method.

[0105] Table 2. Performance comparison of different traffic flow prediction methods on the METR-LA and PEMS-BAY datasets

[0106]

[0107]

[0108] Step 3.5: Ablation experiment

[0109] To further study the performance of different modules of the proposed IDG-PSAtt method, seven variants of the IDG-PSAtt method were designed to verify the influence of each module on the IDG-PSAtt method. The seven variants were compared with the IDG-PSAtt method in terms of MAE, RMSE and MAPE averages on the METR-LA and PEMS-BAY datasets. The results of the ablation experiment are shown in Table 3.

[0110] Table 2. Performance comparison of different components on METR-LA and PEMS-BAY datasets

[0111]

[0112] The differences between these eight variant methods and the IDG-PSAtt method are:

[0113] 1) GCN w / o: Based on the IDG-PSAtt method, the GCN network is removed;

[0114] 2) DGCN w / o: Based on the IDG-PSAtt method, the DGCN network is removed;

[0115] 3) Conv w / o: Based on the IDG-PSAtt method, the one-dimensional convolution module is removed from the interactive learning structure;

[0116] 4) Apt Adj w / o: Based on the IDG-PSAtt method, the adaptive adjacency matrix in the DGCN network is removed;

[0117] 5) Learned Adj w / o: Based on the IDG-PSAtt method, the Graph Generator Structure is removed and the adaptive adjacency matrix is ​​retained;

[0118] 6) ProbSSAtt w / o: Based on the IDG-PSAtt method, the probabilistic sparse self-attention mechanism module is removed;

[0119] 7) ST-Conv Block w / o: Based on the IDG-PSAtt method, the spatiotemporal convolution module is removed.

[0120] First, GCN is crucial to the IDG-PSAtt method. Second, the proposed IDGCN module and ProbSSAtt module are crucial to improving the overall performance of the method. One-dimensional convolution is used to expand the receptive field and is a key component of the interactive learning structure. Ablation experiments show that one-dimensional convolution can significantly improve the performance of the method. In the IDG-PSAtt method, an ablation study is conducted on the two adjacency matrices defined in the DGCN module. Figure 7 As can be seen in Figure 2, the adaptive adjacency matrix is ​​crucial for the prediction of the method. The learnable adjacency matrix is ​​combined with the adaptive adjacency matrix to generate a dynamic adjacency matrix; the dynamic adjacency matrix allows graph convolution to better capture the hidden spatial correlations in traffic data, proving the effectiveness of the two core structures proposed, namely interactive learning and dynamic graph convolution.

[0121] The above results show that the IDG-PSAtt method achieves the best prediction results at different prediction time stages. The IDG-PSAtt method can accurately predict traffic congestion, capture the changing trends of traffic flow, and identify the start and end times of traffic peaks. This demonstrates the excellent prediction performance of the IDG-PSAtt method in traffic flow prediction tasks and its effectiveness in real-time traffic forecasting.

[0122] Although the present invention has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications and improvements may be made based on the present invention. Therefore, such modifications and improvements, which do not depart from the spirit of the present invention, are intended to be within the scope of protection claimed herein.

Claims

1. Traffic flow prediction method based on interactive dynamic graph convolution and probabilistic sparse attention, characterized by: The steps include: Step 1: Feed the original data to the start convolution layer Start Conv to obtain the data represented in high-dimensional space through Start Conv; Step 2: Input the data represented in the high-dimensional space into the interactive dynamic graph convolution (IDGCN). IDGCN uses interleaved sampling to recursively generate two subsequences of equal size for the input high-dimensional data. IDGCN interactively learns the two subsequences and shares the learned features of each. Step 3: DGCN is embedded into the interactive learning structure. Based on the dynamic graph convolution DGCN, the high-dimensional features extracted by the Start Conv layer are processed with an interactive learning strategy. After the IDGCN extracts the spatiotemporal features, two subsequences are output. Step 4: The output subsequences are fused in series, and all the output subsequences are reorganized in time index order and then fed into the Diffusion Graph Conv. Step 5: Input the dynamic spatial features output by Diffusion Graph Conv into the multi-head probabilistic sparse self-attention layer ProbSSAtt Block to extract dynamic spatiotemporal features; Step 6: Input the dynamic spatiotemporal features output by ProbSSAttBlock into ST-Conv to extract temporal features; Step 7: Input the spatiotemporal features output by ST-Conv into the multi-layer perceptron MLP; Step 8: Construct a prediction layer, input the dynamic spatiotemporal features output by the MLP into the prediction layer, and perform traffic flow prediction through the prediction layer; Step 9: Configure the model's hyperparameters, optimizer, loss function, and number of iterations. The steps for constructing the traffic flow prediction method described in step 2 are: Step 1: Prepare the raw traffic flow data by feeding it into the start convolution layer StartConv. StartConv obtains the data represented in a high-dimensional space and captures the traffic flow dependencies at a deeper level. Step 2: Input the data represented in the high-dimensional space into the interactive dynamic graph convolution (IDGCN). IDGCN uses interleaved sampling to recursively generate two subsequences of equal size. The subsequence length is halved. IDGCN interactively learns the two subsequences, and the two subsequences share their learned traffic flow features. Step 3: Embed DGCN into the interactive learning structure. Based on the dynamic graph convolution DGCN, the high-dimensional features extracted by the Start Conv layer are processed by using the interactive learning strategy, and the spatiotemporal features are extracted using IDGCN. After extracting the spatiotemporal features, two subsequences are output. Step 4: The output subsequences pass through the concatenation fusion module. In the concatenation, all the output subsequences are reorganized in time index order, and the reorganized sequences are fed into the diffusion graph convolution; Step 5: Input the dynamic spatial features output by Diffusion Graph Conv into the multi-head probabilistic sparse self-attention layer ProbSSAttBlock to extract dynamic spatiotemporal features; Step 6: Input the dynamic spatiotemporal features output by ProbSSAttBlock into ST-Conv and extract the temporal features through ST-Conv; Step 7: Input the spatiotemporal features output by ST-Conv into the multi-layer perceptron MLP, and aggregate the extracted dynamic spatiotemporal features through MLP; Step 8: Construct a prediction layer, input the dynamic spatiotemporal features output by the MLP into the prediction layer, and use the prediction layer to predict traffic flow; Step 9: Configure the model's hyperparameters, optimizer, loss function, and number of iterations. The specific steps of step 3 are: Step 3.1: Represent the traffic road network as a graph G = (V, E, A), where |V| = N is a set of nodes, each node represents an observation sensor in the road network; E is a set of edges between nodes, whose weights are represented by the distance between nodes; Represents the initial adjacency matrix generated by graph G. If v i ,v j ∈V and (v i ,v j )∈E, then A ij is 1, otherwise it is 0; the initial adjacency matrix A generated by the original traffic network is used as prior knowledge, and the historical time series To predict future traffic flow in represents the observed value of graph G at time t, C represents the number of feature channels, T′ represents the length of the given historical time series, and T represents the length of the predicted future traffic series. The mapping relationship of the traffic flow prediction problem can be expressed as follows: Where f represents the prediction function that can predict future traffic flow based on a given historical time series; Step 3.2: Implement an interactive learning module using a convolutional neural network (CNN) and a graph convolutional network (GCN). The original data is processed using an interleaved sampling method to achieve multi-resolution analysis and expand the receptive field. The interactive learning module consists of three identical interactive dynamic graph convolutional networks (IDGCNs). In the IDGCN, two subsequences interactively learn their own dynamic spatiotemporal features. Each subsequence preprocesses the features through a convolution to expand the receptive field. The two subsequences share parameter weights in the DGCN and capture each other's dynamic spatiotemporal features. Step 3.3, Represents the input of IDGCN. After interleaved sampling, X obtains two subsequences, which are odd sequences even sequence Conv1, Conv2, Conv3, and Conv4 in the interactive dynamic graph convolution represent 1D convolution operations; the output of the first interactive learning of IDGCN is X′ odd and X′ even The final output sequence is obtained through further interactive learning The specific calculations in interactive dynamic graph convolution are as follows: X even ,X odd =Split(X) (2) X′ odd =tanh(DGCN(Conv1(X even )))⊙X odd (3) X′ even =tanh(DGCN(Conv2(X odd )))⊙X even (4) X odd_out =X′ odd +tanh(DGCN(Conv3(X′ even ))) (5) X even_out =X′ even +tanh(DGCN(Conv4(X′ odd ))) (6) ⊙ represents the Hadamard product, tanh represents the activation function, and DGCN represents the dynamic graph convolutional network in IDGCN; The specific steps of step 4 are: Step 4.1, dynamic graph convolution DGCN is mainly composed of diffusion graph convolution network and graph generation module. DGCN uses diffusion graph convolution and graph generation module to better learn deep dynamic spatial features. DGCN converts hidden features into and the predefined initial adjacency matrix Feed it as input to the diffusion graph convolutional network, which is fed into the graph generator and the multi-layer perceptron layer to generate a discrete matrix containing spatiotemporal information A′ is expressed as follows: A′=SoftMax(MLP(GCN(H,A))) (7) Among them, GCN represents the diffusion convolution and graph generator operation, and MLP represents the multi-layer perceptron; Step 4.2: Use Gumbel reparameterization to ensure that the sampling process is differentiable during training: Where g~Gumbel(0,1) represents a random variable, τ is the softmax temperature parameter with a value of 0.5, and A learn Represents the adjacency matrix generated by the graph generator that can simulate the dynamic dependencies between nodes; Step 4.3: Build an adaptive adjacency matrix As shown below: in represents the learnable parameters, A apt The initial value of is the adjacency matrix predefined based on the original graph data Step 4.4: Use the adaptive fusion module to transform A learn and A apt Fusion, and then the resulting dynamic adjacency matrix Feed it into the diffusion graph convolutional network to extract the hidden dynamic spatiotemporal correlation in the traffic road. The calculation of this fusion module is as follows: A dyn =αA apt +(1-a)A learn (10) Where α represents the learnable adaptive parameter factor; Step 4.5: Diffusion graph convolution is used in the graph generator network, fusion graph convolution and serial fusion modules, and the input of diffusion graph convolution is uniformly defined as In the graph generator network, the diffuse graph convolution is defined as: Among them, k is the diffusion step size, K is the maximum number of diffusion steps, and W represents the parameter matrix; in the fusion graph convolution module, A dyn is the adjacency matrix of the input of the fused graph convolution, where the diffusion graph convolution is expressed as follows: The IDG-PSAtt model captures and corrects the entire time series features by recombining the dynamic spatiotemporal features extracted from the interactive learning structure in the tandem module in time index order and feeding them into the Diffusion Graph Conv. Step 4.6: In the diffusion graph convolutional network, the predefined initial adjacency matrix is ​​used and a dynamic adjacency matrix generated by an interactive learning structure For the initial adjacency matrix A, use the directed graph and use P respectively f =A / rowsum(A) and P b =A T / rowsum(A T ) represents the forward and backward transfer matrices of A; the diffusion graph convolution in the serial fusion module is represented as follows: The DGCN module not only extracts deep hidden spatial features by exploring the invisible dependencies between nodes in the traffic network, but also generates dynamic associations between data based on the input traffic flow time series simulation. In addition, by embedding DGCN into an interactive learning framework, it is possible to fully utilize the dynamic spatial information captured by GCN during the training process to enhance the capture of complex temporal dependencies of traffic flow. The specific steps of step 5 are: Step 5.

1. Construct a spatiotemporal convolution module consisting of three convolution kernels. The three convolution kernels correspond to the three angles of time, space, and time and space to capture the spatiotemporal features extracted from the diffusion graph convolution module, and capture the influence of multiple node features on single node features in the topological graph structure of traffic flow; the time kernel captures the dependency of traffic flow at different times at the same location, and the space kernel captures the spatial correlation of traffic flow at adjacent locations at the same time step; each spatiotemporal convolution block takes the output of the previous spatiotemporal attention block as input The output can be calculated in, is the time kernel of f×1, is the spatial kernel of 1×f, is an f×f spatiotemporal kernel, LeakyReLU(·) represents the Leaky rectified linear unit function, and * represents the convolution operation; the outputs of the three convolution kernels are connected and a 1×1 convolution is used. To compress features and limit the number of channels; The specific steps of step 6 are: Step 6.1: The input of the self-attention mechanism is in the form of (Q, K, V), and the dot product is calculated as follows: in, and d represent the input query, key, value, and dimension respectively; the attention coefficient A(q i ,K,V) are as follows: Among them, q i , k i and v i are the i-th row in Q, K and V respectively; and k(q i ,k l ) using an asymmetric exponential kernel Step 6.2, spatiotemporal convolution probability sparse self-attention mechanism STC-ProbSSAtt adopts M(q i ,K) represents the sparsity of the i-th query, and KL divergence is used to measure the sparsity of the query, as shown below: Among them, the first item is q on all keys i The logarithm and exponential of , the arithmetic mean of all keys in the second term, and the probabilistic sparse self-attention mechanism are calculated as follows: in, represents a sparse matrix of the same dimension as q, consisting only of the first u queries at the sparsity evaluated by M(q,K), u = c lnL Q Controlled by a constant sampling factor c; the complexity of the probabilistic sparse self-attention mechanism for each query is only O(lnL Q ), adopting a multi-head probabilistic sparse self-attention mechanism, which can generate different sparse query key pairs to avoid severe information loss; The specific steps of step 7 are: Step 7.1: By constructing a prediction layer, the dynamic spatiotemporal features output by the multi-layer perceptron (MLP) are input into the prediction layer, and traffic flow prediction is performed through the prediction layer.

Images