Traffic flow prediction method based on space-time causal attention network

Abstract

The invention provides a traffic flow prediction method based on a space-time causal attention network. The method comprises the following steps: coding all road nodes through node embedding; self-adaptively learning the relationship between road nodes according to the road traffic conditions at different moments; the relation between the nodes is combined with a graph attention network GAT through a multi-head time attention module to form a node adaptive learning graph attention network AGAT; the space-time causal attention network STCAN reconstructs the node adaptive learning graph attention network AGAT into a gate circulation unit GRU; the space-time causal attention network STCAN captures the space-time causal relationship of the traffic data in a fine-grained manner by using a two-way gate circulation unit GRU; a residual module is introduced to reduce network degradation caused by the deep network; the space-time causal attention network STCAN is combined with the multi-head time attention module to process time sequence data; and a multi-step flow prediction task is realized by performing linear change on the two-layer full-connection neural network pair.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic flow prediction, and in particular, to a traffic flow prediction method based on a spatiotemporal causal attention network. Background Art

[0002] Traffic flow prediction is an important component of intelligent transportation systems (ITS)[1]. It can effectively mine potential traffic patterns from spatiotemporal traffic data, help traffic managers perceive road traffic conditions in advance, manage and control traffic flow scheduling, select good travel routes for travelers, avoid possible road congestion in the future, and improve travel efficiency[2].

[0003] Prediction models based on spatiotemporal traffic data should consider the temporal correlation of historical time series and the spatial correlation between traffic roads. Early traffic flow prediction methods were usually regarded as multivariate sequence analysis tasks in the time dimension, such as using queuing theory [3] models, traffic behavior theory [4], machine learning methods [5], etc. to model traffic time series data. However, these methods only consider the correlation in the time dimension and ignore the correlation in the spatial dimension. Therefore, more and more researchers have turned to studying spatiotemporal prediction models based on graph neural networks (GNNs). [6] , and achieved remarkable results. However, these models still have some limitations.

[0004] The first limitation is that the dynamic correlation information between nodes on the graph is ignored. Using GNN to capture the spatial relationship of the transportation network can be considered as edge transformation and aggregation through node information in the transportation network. [6] .like Figure 1 As shown, most existing methods [7][8][9] Use a predefined static adjacency matrix to describe the spatial relationship between traffic road nodes

[10] However, the relationships between road nodes in a transportation network are dynamic and interactive, depending on various complex factors on the road, such as traffic flow, number of lanes, and population density. Therefore, the spatial relationship of roads cannot be simply modeled by static spatial connections between roads.

[0005] Secondly, traffic information on the traffic network has high nonlinear correlation and uncertainty. Examples include regular road maintenance, sudden accidents, etc. However, these models usually use recurrent neural networks (RNN)

[11] and its variants (Long Short-Term Memory (LSTM)

[12] Gated Recurrent Unit (GRU)

[13] ) model to model the local temporal correlation of traffic data. When processing long time series, the signal must traverse along the long cycle path of the network, which poses a challenge to effectively modeling the global temporal correlation of long time series data. In addition, the sequential execution process of RNN makes it unable to capture the causal correlation information of traffic events.

[14] To solve the above problems, some research methods based on convolutional neural networks (CNNs) stack convolutional layers into multiple layers to simulate global temporal correlation.

[15]

[16] However, as the dilation rate increases, local time-varying information may be lost.

[17] At the same time, as the depth of the neural network increases, the optimization difficulty of the model increases, and network degradation problems will inevitably occur.

[18] .

[0006] Spatiotemporal graph neural networks provide a feasible solution for traffic prediction. Some models, such as DCRNN [7] , combining diffuse convolution with a GRU-based encoder-decoder architecture to capture the directionality of traffic propagation.

[19] proposed a local spatiotemporal graph to represent the transmission of information in the spatiotemporal direction, and adopted the residual connected GCN module to model the spatiotemporal heterogeneity. However, they ignored the fact that the predefined graph does not fully reflect the reality of traffic connections. Although GWNET

[20] and MTGNN

[21] Improved their practice through adaptive graph learning, but the separated spatiotemporal components cannot capture synchronous correlations. GMAN based on spatiotemporal causal attention fusion

[22] It performs well in long-term forecasting but poorly in short-term forecasting.

[23] and DSTAGNN

[24] Achieved good prediction performance, but brought about large computational consumption and system overhead. These works have contributed to the great improvement of performance from multiple perspectives. However, the present invention observed that they were not combined in a cohesive way to address comprehensive features and their correlations, and rarely balanced method performance and complexity.

[0007] References:

[0008] [1]Guevara L,Auat Cheein F.The role of 5G technologies:Challenges insmart cities and intelligent transportation systems[J].Sustainability,2020,12(16):6469.

[0009] [2]Richter A, M O,Ebendt R,et al.Towards an integrated urbandevelopment considering novel intelligent transportation systems:UrbanDevelopment Considering Novel Transport[J].Technological Forecasting andSocial Change,2020,155:119970.

[0010] [3]Xu X,Liu J,Li H,et al.Analysis of subway station capacity with theuseof queueing theory[J].Transportation research part C:emergingtechnologies,2014,38:28-43.

[0011] [4]Cascetta E.Transportation systems engineering:theory and methods[M].Springer Science&Business Media,2013.

[0012] [5]Li C,Xu P.Application on traffic flow prediction of machinelearning inintelligent transportation[J].Neural Computing and Applications,2021,33(2):613-624.

[0013] [6]Wu Z,Pan S,Chen F,et al.A comprehensive survey on graphneuralnetworks[J].IEEE transactions on neural networks and learning systems,2020,32(1):4-24.

[0014] [7]Li Y,Yu R,Shahabi C,et al.Diffusion convolutional recurrentneuralnetwork:Data-driven traffic forecasting[J].arXiv preprint arXiv:1707.01926,2017.

[0015] [8]Guo S,Lin Y,Feng N,et al.Attention based spatial-temporalgraphconvolutional networks for traffic flow forecasting[C] / / Proceedings ofthe AAAIconference on artificial intelligence.2019,33(01):922-929.

[0016] [9]Bai L,Yao L,Li C,et al.Adaptive graph convolutional recurrentnetworkfor traffic forecasting[J].Advances in neural information processingsystems,2020,33:17804-17815.

[0017]

[10] Waikhom L,Patgiri R.Graph neural networks:Methods,applications,and opportunities[J].arXiv preprint arXiv:2108.10733,2021.

[0018]

[11] Sha S,Li J,Zhang K,et al.RNN-based subway passenger flowrollingprediction[J].IEEE Access,2020,8:15232-15240.

[0019]

[12] Karimzadeh M,Aebi R,de Souza A M,et al.Reinforcementlearning-designed LSTM for trajectory and traffic flow prediction[C] / / 2021IEEEwireless communications and networking conference(WCNC).IEEE,2021:1-6.

[0020]

[13] Sun P,Boukerche A,Tao Y.SSGRU:A novel hybrid stackedGRU-basedtraffic volume prediction approach in a road network[J].ComputerCommunications,2020,160:502-511.

[0021]

[14] Li T,Ni A,Zhang C,et al.Short-term traffic congestionpredictionwith Conv–BiLSTM considering spatio-temporal features[J].IETIntelligentTransport Systems,2020,14(14):1978-1986.

[0022]

[15] Zheng H,Lin F,Feng X,et al.A hybrid deep learning modelwithattention-based conv-LSTM networks for short-term traffic flow prediction[J].IEEE Transactions on Intelligent Transportation Systems,2020,22(11):6910-6920.

[0023]

[16] Ma D,Song X,Li P.Daily traffic flow forecasting through acontextualconvolutional recurrent neural network modeling inter-and intra-daytrafficpatterns[J].IEEE Transactions on Intelligent Transportation Systems,2020,22(5):2627-2636.

[0024]

[17] Li M,Zhu Z.Spatial-temporal fusion graph neural networks fortrafficflow forecasting[C] / / Proceedings of the AAAI conference on artificialintelligence.2021,35(5):4189-4196.

[0025]

[18] He K,Zhang X,Ren S,et al.Deep residual learning forimagerecognition[C] / / Proceedings of the IEEE conference on computer visionandpattern recognition.2016:770-778.

[0026]

[19] Song C,Lin Y,Guo S,et al.Spatial-temporal synchronousgraphconvolutional networks:A new framework for spatial-temporal networkdataforecasting[C] / / Proceedings of the AAAI conference on artificialintelligence.2020,34(01):914-921.

[0027]

[20] Wu Z,Pan S,Long G,et al.Graph wavenet for deep spatial-temporalgraph modeling[J].arXiv preprint arXiv:1906.00121,2019.

[0028]

[21] Wu Z,Pan S,Long G,et al.Connecting the dots:Multivariatetimeseries forecasting with graph neural networks[C] / / Proceedings of the 26thACMSIGKDD international conference on knowledge discovery&data mining.2020:753-763.

[0029]

[22] Zheng C,Fan X,Wang C,et al.Gman:A graph multi-attention networkfor traffic prediction[C] / / Proceedings of the AAAI conference on artificialintelligence.2020,34(01):1234-1241.

[0030]

[23] Choi J,Choi H,Hwang J,et al.Graph neural controlled differentialequations for traffic forecasting[C] / / Proceedings of the AAAI Conference onArtificial Intelligence.2022,36(6):6367-6374.

[0031]

[24] Lan S,Ma Y,Huang W,et al.Dstagnn:Dynamic spatial-temporal awaregraph neural network for traffic flow forecasting[C] / / Internationalconference on machine learning.PMLR,2022:11906-11917. Summary of the invention

[0032] According to the technical problems mentioned in the above background technology, a traffic flow prediction method based on spatiotemporal causal attention network is provided.

[0033] The technical means adopted by the present invention are as follows:

[0034] A traffic flow prediction method based on spatiotemporal causal attention network, representing traffic information by traffic speed, includes the following steps:

[0035] S1: Encode all road nodes through node embedding;

[0036] S2: Adaptively learn the relationship between road nodes according to the road traffic conditions at different times, and get rid of the constraints of the predefined adjacency matrix;

[0037] S3: The relationship between nodes is combined with the graph attention network GAT through the multi-head temporal attention module to form a node-adaptive learning graph attention network AGAT to dynamically model the spatial dependency of the traffic road network;

[0038] S4: The spatiotemporal causal attention network STCAN reconstructs the node adaptive learning graph attention network AGAT into a gated recurrent unit GRU to obtain local spatiotemporal dependencies;

[0039] S5: The spatiotemporal causal attention network STCAN uses a bidirectional gated recurrent unit GRU to capture the spatiotemporal causal relationship of traffic data in a fine-grained manner;

[0040] S6: Introduce residual module to reduce network degradation caused by deep network;

[0041] S7: The spatiotemporal causal attention network STCAN processes time series data by combining with the multi-head temporal attention module and captures the global spatiotemporal dependency information of traffic data;

[0042] S8: Implement multi-step traffic prediction tasks by linearly transforming a two-layer fully connected neural network pair;

[0043]

[0044] Among them, W 1 ∈R F″′×(T*2F″) and W 2 ∈R T×F″′ represents the weight matrix of the fully connected neural network, and b 2 Denotes the corresponding bias term, Y′∈R N×T Represents the final prediction result; H S represents the output of the temporal attention module, defined as

[0045] Furthermore, in S1, a graph G = (V, E) is defined to describe the road network topology, and a set V = {v 1 ,v 2 ,…,v N} represents all road nodes on the road network topology graph, where N represents the total number of road nodes; E represents the set of edge connection relationships between road nodes;

[0046] Construct the characteristic matrix of the traffic road network X = {X 1 ,X 2 ,…,X N}∈R N×T×F , where T represents the length of the historical time series, and F represents the number of node feature attributes; represents the traffic information observed by N road nodes in the road network at time t, and Then it means that node v i All eigenvectors at time t;

[0047] Considering the spatial correlation and temporal correlation of each road node in the road network, traffic prediction has a strong spatiotemporal correlation. The goal is to use the road network topology G and the traffic information feature matrix X to predict the traffic speed information at the next T time by learning the function f(·), which can be expressed as:

[0048] [X t+1 ,X t+2 ,…,X t+T ]=f(X t-T ,X t-(T-1) , …, X t ).

[0049] Furthermore, in S3, the global context information of the sequence is extracted through the multi-head temporal attention module; the multi-head temporal attention module includes: position embedding, a multi-head temporal attention layer and a residual network.

[0050] Furthermore, in S3, the hidden layer sequence {H 1 [i,:],H 2 [i,:],…,H T [i,:]} is input into the temporal attention module; considering that the calculation of the multi-head attention mechanism ignores the relative position of the sequence, position encoding is added to the feature data at each time step, that is:

[0051]

[0052] The position token e t Defined as:

[0053]

[0054] First, the embedded features Project it into a high-dimensional latent subspace to generate the Query, Key, and Value matrices; then T Normalized as a matrix product, we get the attention distribution at each time step; finally, we superimpose the attention matrix onto V to generate an implicit representation with temporal context:

[0055]

[0056] in, Both represent the projection matrix to be learned, represents the weight scaling factor, and the weight score is normalized using the softmax function; the multi-head time attention module enriches the information representation, and the multi-head time attention module acts independently on the sequence; the calculated data is connected and then linearly transformed to obtain the output H Attn :

[0057] MultiHead(Q,K,V)=Concat(head 1 ,head 2 ,…,head h )W O ,

[0058]

[0059] in, Both represent the projection matrix of linear transformation;

[0060] H Attn Passed to the residual network, each sublayer is a feed-forward network and layer normalization defines the representation of the output of the temporal attention module as Where D = h × d model , h represents the number of attention heads.

[0061] Furthermore, in S4, a bidirectional channel selection recurrent network is used to capture the temporal dependency of the flow time series; the gated unit of GRU is replaced by AGAT, that is, the spatially correlated time series data X″ is used. t As the input data of GRU at any time t, it captures the temporal dependency of traffic data:

[0062]

[0063] in, represents the output of the previous moment, [·] represents the concatenation operation on the feature dimension, represents the candidate hidden layer state, ⊙ represents element-wise multiplication, Represents the output at the current moment;

[0064] Use residual modules to connect the layers of the network to enhance the model's long-term capture ability:

[0065]

[0066] Among them, ω 1 and ω 2 Both represent one-dimensional convolution kernels, ε represents a nonlinear activation function, represents the convolution operation, represents the output of the remaining cascade; W represents the weight matrix; A represents the adjacency matrix; X represents the input information at the current moment; E At ∈R N×d represents the embedding dictionary encoding each traffic road node;

[0067] Until the Tth time step, the above operations are completed and the spatiotemporal dependency is obtained. The reverse operation is the same. Finally, the forward output and the reverse output are concatenated to get the output H∈R N×T×2F″ , the formula is as follows:

[0068]

[0069] Furthermore, in S5, the dynamic association information between road nodes at different times is learned in a fine-grained manner through an adaptive graph learning mechanism to generate a traffic subgraph at the corresponding time; the adaptive graph learning mechanism generates an adjacency matrix corresponding to the time at any time t

[0070]

[0071] Among them, E At ∈R N×d is the embedding dictionary for encoding each traffic road node, d represents the embedding dimension of the node, Indicates E At The transposed matrix of ; ReLU represents the nonlinear activation function, and softmax represents the normalization function;

[0072] At any time t, GAT calculates the attention coefficients of its neighboring nodes point by point based on the node-related information generated by the node adaptive learning mechanism; aggregates the spatial dependencies of the nodes on the graph at time t; and shows that the node v i Any adjacent node v j Attention coefficient The calculation process:

[0073]

[0074] Where a represents the attention mechanism calculation function, W∈R F×F′ Represents the weight matrix of all nodes in the graph; by i Normalize the attention coefficients of all adjacent nodes to obtain the attention coefficient of the attention layer of the graph

[0075]

[0076] in, Indicates v i All neighbor nodes of

[0077] Build a shared weight pool W p ∈R d×F×F′ , according to the node embedding dictionary E At Get the weight matrix W′=E for each node At ·W p ∈R F×F′ ; The calculation process of attention coefficient is:

[0078]

[0079] in, represents the weight matrix, k represents the connection operation, and LeakReLu represents the nonlinear activation function; self-attention is extended to the multi-head attention mechanism, that is, the above equation is called Group independent attention mechanisms and then stitch the outputs together:

[0080]

[0081] in, Indicates the qth th The weight coefficient calculated by the attention mechanism of the group at time t, represents the weight matrix of the corresponding group, Indicates that through node v i New feature representations obtained through the attention layer of the multi-head graph;

[0082] Through the weight pool W′ p ∈R d×QF′×F″ Change the output dimension of the node from Map to Get the final output result

[0083]

[0084] When all nodes in the graph have completed the above graph attention layer operation, the output feature is obtained The time corresponding to each node; this process is expressed as:

[0085]

[0086] Among them, σ(·) represents the computation function of the graph attention layer.

[0087] Compared with the prior art, the present invention has the following advantages:

[0088] 1) We construct a node-adaptive learning graph attention network (AGAT) through a data-driven approach, which can dynamically generate a weighted adjacency matrix of the graph based on the road traffic status information at different times. Then, we integrate it into GAT to dynamically capture the spatially relevant information of the traffic network.

[0089] 2) This paper proposes a new spatiotemporal component to capture local spatiotemporal correlations in traffic networks. This component replaces the GRU gated unit with A-GAT and is further constructed as a recurrent bidirectional network to capture local spatiotemporal correlations and causal correlations. In order to alleviate the degradation problem that may occur in deep networks, residual connectivity is also introduced.

[0090] 3) The present invention integrates a temporal attention module to improve the model's ability to identify spatial field of view changes and long-range temporal correlations. This indeed reduces the error of the overall prediction in complex scenes and enhances the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0092] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0093] Figure 2 This is a schematic diagram of the spatiotemporal causal attention network model based on the present invention.

[0094] Figure 3 Schematic diagram for comparing the model prediction value and the actual value on PEMS08 of the present invention; wherein (a) closely tracks the ground truth value within 15 minutes; (b) closely tracks the ground truth value within 30 minutes; (c) closely tracks the ground truth value within 60 minutes. DETAILED DESCRIPTION

[0095] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0096] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0097] like Figure 1 As shown, the present invention provides a traffic flow prediction method based on spatiotemporal causal attention network, and the traffic flow prediction uses the historical information of the road to predict the future traffic pattern. The traffic information here is an abstract concept: traffic flow, density, speed, etc. The present invention represents traffic information by traffic speed, and is characterized by comprising the following steps:

[0098] Step S1: Encode all road nodes by node embedding; In S1, define a graph G = (V, E) to describe the road network topology, and use a set V = {v 1 ,v 2 ,…,v N} represents all road nodes on the road network topology graph, where N represents the total number of road nodes; E represents the set of edge connection relationships between road nodes;

[0099] Construct the characteristic matrix of the traffic road network X = {X 1 ,X 2 ,…,X N}∈R N×T×F , where T represents the length of the historical time series, and F represents the number of node feature attributes; represents the traffic information observed by N road nodes in the road network at time t, and Then it means that node v 1 All eigenvectors at time t;

[0100] Considering the spatial correlation and temporal correlation of each road node in the road network, traffic prediction has a strong spatiotemporal correlation. The goal is to use the road network topology G and the traffic information feature matrix X to predict the traffic speed information at the next T time by learning the function f(·), which can be expressed as:

[0101] [X t+1 ,X t+2 ,…,X t+T ]=f(X t-T ,X t-(T-1) , …, X t ).

[0102] Step S2: Adaptively learn the relationship between road nodes according to the road traffic conditions at different times, and get rid of the constraints of the predefined adjacency matrix.

[0103] GRU is processed by gradually expanding along the time axis, which causes the current output to depend on the state of the previous moment, and therefore lacks the ability to capture long-term temporal dependencies. In order to improve the performance of extracting long-term temporal dependencies, the present invention deploys a multi-head temporal attention module along the time axis.

[0104] Step S3: The relationship between nodes is combined with the graph attention network GAT through a multi-head temporal attention module to form a node adaptive learning graph attention network AGAT to dynamically model the spatial dependency of the traffic road network; in S3, the global context information of the sequence is extracted through the multi-head temporal attention module; the multi-head temporal attention module includes: position embedding, a multi-head temporal attention layer and a residual network.

[0105] As a preferred embodiment, in the present application, in S3, the hidden layer sequence {H 1 [i,:], 2 [i,:],…, T [i,:] is input to the temporal attention module; considering that the calculation of the multi-head attention mechanism ignores the relative position of the sequence, position encoding is added to the feature data at each time step, that is:

[0106]

[0107] The position token e t Defined as:

[0108]

[0109] First, the embedded features Project it into a high-dimensional latent subspace to generate the Query, Key, and Value matrices; then TNormalized as a matrix product, we get the attention distribution at each time step; finally, we superimpose the attention matrix onto V to generate an implicit representation with temporal context:

[0110]

[0111] in, Both represent the projection matrix to be learned, represents the weight scaling factor, and the weight score is normalized using the softmax function; the multi-head time attention module enriches the information representation, and the multi-head time attention module acts independently on the sequence; the calculated data is connected and then linearly transformed to obtain the output H Attn :

[0112] MultiHead(Q,K,V)=Concat(head 1 ,head 2 ,…,head h )W O ,

[0113]

[0114] in, Both represent the projection matrix of linear transformation;

[0115] H Attn Passed to the residual network, each sublayer is a feed-forward network and layer normalization defines the representation of the output of the temporal attention module as Where D = h × d model , h represents the number of attention heads.

[0116] In the present application, as a preferred embodiment, in the time dimension, there is a correlation between traffic conditions in different time periods. Depending on the length of different time steps and the method of unfolding the time axis, the available correlations also vary significantly. Step S4: The spatiotemporal causal attention network STCAN reconstructs the node adaptive learning graph attention network AGAT into a gated recurrent unit GRU to obtain local spatiotemporal dependencies. In S4, a bidirectional channel selection recurrent network is used to capture the temporal dependency of the traffic time series; the gated unit of the GRU is replaced by AGAT, that is, the spatially correlated time series data X″ is used. t As the input data of GRU at any time t, it captures the temporal dependency of traffic data:

[0117]

[0118] in, represents the output of the previous moment, [·] represents the concatenation operation on the feature dimension, represents the candidate hidden layer state, ⊙ represents element-wise multiplication, Represents the output at the current moment;

[0119] Use residual modules to connect the layers of the network to enhance the model's long-term capture ability:

[0120]

[0121] Among them, ω 1 and ω 2 Both represent one-dimensional convolution kernels, ε represents a nonlinear activation function, represents the convolution operation, represents the output of the remaining cascade; W represents the weight matrix; A represents the adjacency matrix; X represents the input information at the current moment; E At ∈R n×d represents the embedding dictionary encoding each traffic road node;

[0122] Until the Tth time step, the above operations are completed and the spatiotemporal dependency is obtained. The reverse operation is the same. Finally, the forward output and the reverse output are concatenated to get the output H∈R N×t×2F″ , the formula is as follows:

[0123]

[0124] Preferably, in the spatial dimension, the traffic condition information of different road nodes has a strong dynamic correlation. However, the traditional graph neural network constructs a predefined adjacency matrix based on the connectivity or distance between graph nodes to perform graph convolution operations. Although the predefined adjacency matrix intuitively represents the positional relationship between nodes, it cannot dynamically reflect the dynamic spatial dependency between road nodes at different times. Therefore, Figure 1 As shown, the present invention adopts an adaptive graph learning mechanism to learn the dynamic correlation information between road nodes at different times in a fine-grained manner to generate a traffic subgraph at the corresponding time. Step S5: The spatiotemporal causal attention network STCAN uses a bidirectional gated recurrent unit GRU to capture the spatiotemporal causal relationship of traffic data in a fine-grained manner. In S5, the dynamic correlation information between road nodes at different times is learned in a fine-grained manner through an adaptive graph learning mechanism to generate a traffic subgraph at the corresponding time; the adaptive graph learning mechanism generates an adjacency matrix corresponding to any time t at that time.

[0125]

[0126] Among them, E At ∈R N×dis the embedding dictionary for encoding each traffic road node, d represents the embedding dimension of the node, Indicates E At The transposed matrix of ; ReLU represents the nonlinear activation function, and softmax represents the normalization function;

[0127] At any time t, GAT calculates the attention coefficients of its neighboring nodes point by point based on the node-related information generated by the node adaptive learning mechanism; aggregates the spatial dependencies of the nodes on the graph at time t; and shows that the node v i Any adjacent node v j Attention coefficient The calculation process:

[0128]

[0129] Where a represents the attention mechanism calculation function, W∈R F×F′ Represents the weight matrix of all nodes in the graph; by i Normalize the attention coefficients of all adjacent nodes to obtain the attention coefficient of the attention layer of the graph

[0130]

[0131] in, Indicates v i All neighbor nodes of

[0132] Build a shared weight pool W p ∈R d×F×F′ , according to the node embedding dictionary E At Get the weight matrix W′=E for each node At ·W p ∈R F×F′ ; The calculation process of attention coefficient is:

[0133]

[0134] in, represents the weight matrix, k represents the connection operation, and LeakReLu represents the nonlinear activation function; self-attention is extended to the multi-head attention mechanism, that is, the above equation is called Group independent attention mechanisms and then stitch the outputs together:

[0135]

[0136] in, Indicates the qth th The weight coefficient calculated by the attention mechanism of the group at time t, represents the weight matrix of the corresponding group, Indicates that through node v i New feature representations obtained through the attention layer of the multi-head graph;

[0137] Through the weight pool W′ p ∈R d×QF′×F″ Change the output dimension of the node from Map to Get the final output result

[0138]

[0139] When all nodes in the graph have completed the above graph attention layer operation, the output feature is obtained The time corresponding to each node; this process is expressed as:

[0140]

[0141] Among them, σ(·) represents the computation function of the graph attention layer.

[0142] Step S6: Introduce the residual module to reduce the network degradation caused by the deep network.

[0143] Step S7: The spatiotemporal causal attention network STCAN processes time series data by combining with the multi-head time attention module, while capturing the global spatiotemporal dependency information of traffic data;

[0144] Step S8: Implement the multi-step traffic prediction task by linearly changing the two-layer fully connected neural network pair;

[0145]

[0146] Among them, W 1 ∈R F″′×(T*2F″) and W 2 ∈R T×F″′ represents the weight matrix of the fully connected neural network, and b 2 Denotes the corresponding bias term, Y′∈R N×T Represents the final prediction result; H S represents the output of the temporal attention module, defined as

[0147] During the training process, the goal is to minimize the error between the actual traffic speed Y on the road and the predicted value Y'. In this article, the present invention selects the L1 loss function to optimize the neural network model through back propagation. The specific calculation process is as follows:

[0148]

[0149] Example 1

[0150] Experiments are conducted on four public transportation network datasets (PeMS04 and PeMS08) to verify the effectiveness of the proposed framework. All the above traffic flow data are obtained by the Caltrans Performance Measurement System (PeMS), which collects data every 5 minutes. Table 1 provides a summary of the key statistics of these four datasets.

[0151] Table 1 Statistics of the dataset

[0152]

[0153] To evaluate the proposed model, it is compared with 12 baseline models, including traditional statistical models, deep learning models, and graph neural network models. These baselines serve as benchmarks to show the effectiveness of the proposed approach. The following is a brief overview of the baseline models:

[0154] HA: Use the weighted average of historical traffic data as the prediction result for future time periods.

[0155] VAR: Vector Autoregression is a traditional time series model that captures the pairwise relationships between time series.

[0156] LSTM: Predicting traffic data using long short-term memory networks with fully connected layers.

[0157] DCRNN: This model uses bidirectional random walks on the graph to capture spatial correlations, while an encoder-decoder architecture with scheduled sampling is used to capture temporal correlations.

[0158] STGCN: By modeling on a graph, a full convolutional structure is used to capture the spatiotemporal dependencies of traffic networks, thereby improving training speed and reducing parameters.

[0159] GWN: Accurately capture hidden spatial dependencies through adaptive dependency matrix and node embedding learning, and effectively process long time series through dilated 1D convolution components.

[0160] ASTGCN: This model dynamically models the spatiotemporal attention mechanism in space and time.

[0161] STSGCN: This model uses local spatiotemporal subgraph modules to independently model local correlations.

[0162] AGCRN: This model captures the fine-grained spatiotemporal correlations of specific nodes in traffic sequences.

[0163] Z-GCNETs: This model adopts time-aware GCN design to capture the complex spatiotemporal dependencies in traffic data.

[0164] DDSTGCN: This model exploits the dynamic characteristics of traffic flow graph nodes and also captures the dynamic spatiotemporal characteristics of edges by converting the traffic flow graph into its dual hypergraph.

[0165] STCGAT: Adaptively generates spatial adjacency subgraphs through node embedding methods, fine-grainedly models the topology of dynamically generated graphs, and combines causal time-related components to jointly learn local and global spatiotemporal dependencies.

[0166] The present invention divides the data set into a training set, a validation set, and a test set in a ratio of 6:2:2. Then a window of size T+T is slid on the partitioned data set to generate time series data. Specifically, T and T are both set to 12, the first T time steps are the input data of the present invention, and the last T time steps are considered to be the actual label values ​​of the present invention. The past 12 consecutive time steps are used to predict the future 12 consecutive time steps.

[0167] STCAN was written in Pytorch 1.10.0 and used the same hyperparameters in both datasets. In GAT, they used a three-head attention mechanism. The size of the GRU hidden layer was set to 64. The head size of the multi-head temporal attention was set to 4, the batch size was set to 64, the learning rate size was set to 0.001, and the model was optimized using the Adam optimizer with a maximum number of iterations of 300. All experiments were conducted on a server using an NVIDIA GeForce RTX 4090 graphics card.

[0168] In order to measure the prediction performance of the model, the present invention uses the following three indicators on the label value Y i and the predicted value Y i ' to be evaluated.

[0169] Mean Absolute Error (MAE):

[0170]

[0171] Root Mean Square Error (RMSE):

[0172]

[0173] Mean Absolute Percent Error (MAPE):

[0174]

[0175] Among them, M represents the total number of samples. The lower the values ​​of the above three indicators, the higher the prediction accuracy of the model.

[0176] The proposed model is compared with the above 12 representative baseline methods on PeMS04 and PeMS08. As shown in Table 2, the average traffic flow prediction performance results for the next hour (15min, 30min and 60min) are shown.

[0177] Table 2 Performance comparison of STCAN and baseline models on PEMS04 and PEMS08 datasets

[0178]

[0179] It can be observed that: 1) The GCN-based method outperforms the HA, VAR traditional machine learning methods and the LSTM deep learning method, which illustrates the importance of explicitly modeling spatial correlations and the effectiveness of GCN in traffic flow prediction tasks; 2) The performance indicators of the improved GAT-based method on each dataset are significantly better than those of other advanced baseline models, achieving remarkable results; 3) Figure 3 This is the visualization prediction result of node 166 under different prediction durations. The horizontal axis is the time step, each time step represents 5 minutes, and the vertical axis is the traffic flow speed.

[0180] It can be seen that the prediction results at 30min, 60min and 120min are very close to the ground truth, proving that STCAN can more accurately capture the spatiotemporal correlation in the traffic flow sequence and achieve the best prediction effect. Although the prediction accuracy decreases with the prediction time step, it can still accurately predict the changing trend of traffic flow.

[0181] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A traffic flow prediction method based on spatiotemporal causal attention network, which represents traffic information by traffic speed, characterized in that: The following steps are involved: S1: Encode all road nodes through node embedding; S2: Adaptively learn the relationship between road nodes according to the road traffic conditions at different times, and get rid of the constraints of the predefined adjacency matrix; S3: The relationship between nodes is combined with the graph attention network GAT through the multi-head temporal attention module to form a node-adaptive learning graph attention network AGAT to dynamically model the spatial dependency of the traffic road network; S4: The spatiotemporal causal attention network STCAN reconstructs the node adaptive learning graph attention network AGAT into a gated recurrent unit GRU to obtain local spatiotemporal dependencies; S5: The spatiotemporal causal attention network STCAN uses a bidirectional gated recurrent unit GRU to capture the spatiotemporal causal relationship of traffic data in a fine-grained manner; S6: Introduce residual module to reduce network degradation caused by deep network; S7: The spatiotemporal causal attention network STCAN processes time series data by combining with the multi-head temporal attention module and captures the global spatiotemporal dependency information of traffic data; S8: Implement multi-step traffic prediction tasks by linearly transforming a two-layer fully connected neural network pair; Where W1∈R F″′×(T*2F″) and W2∈R T×F″′ represents the weight matrix of the fully connected neural network, b 11 and b2 represents the corresponding bias term, Y′∈R N×T Represents the final prediction result; H S represents the output of the temporal attention module, defined as 2. The traffic flow prediction method based on spatiotemporal causal attention network according to claim 1 is characterized in that: In S1, a graph G = (V, E) is defined to describe the road network topology, and a set V = {v1, v2, ..., v N } represents all road nodes on the road network topology graph, where N represents the total number of road nodes; E represents the set of edge connection relationships between road nodes; Construct the characteristic matrix X of the traffic road network = {X1,X2,…,X N }∈R N×T×F , where T represents the length of the historical time series, and F represents the number of node feature attributes; represents the traffic information observed by N road nodes in the road network at time t, and Then it means that node v i All eigenvectors at time t; Considering the spatial correlation and temporal correlation of each road node in the road network, traffic prediction has a strong spatiotemporal correlation. The goal is to use the road network topology G and the traffic information feature matrix X to predict the traffic speed information at the next T time by learning the function f(·), which can be expressed as: [X t+1 ,X t+2 ,…,X t+T ]=f(X t-T ,X t-(T-1) ,…,X t )。 3. The traffic flow prediction method based on spatiotemporal causal attention network according to claim 1 is characterized in that: In S3, extracting global context information of the sequence through the multi-head temporal attention module; The multi-head temporal attention module includes: position embedding, a multi-head temporal attention layer and a residual network.

4. A traffic flow prediction method based on spatiotemporal causal attention network according to claim 1 or 3, characterized in that: In S3, the hidden layer sequence of AGAT-GRU {H1[i,:],H2[i,:],…,H T [i,:]} is input into the temporal attention module; considering that the calculation of the multi-head attention mechanism ignores the relative position of the sequence, position encoding is added to the feature data at each time step, that is: The position token e t Defined as: First, the embedded features Project it into a high-dimensional latent subspace to generate the Query, Key, and Value matrices; then T Normalized as a matrix product, we get the attention distribution at each time step; finally, we superimpose the attention matrix onto V to generate an implicit representation with temporal context: in, Both represent the projection matrix to be learned, represents the weight scaling factor, and the weight score is normalized using the softmax function; the multi-head time attention module enriches the information representation, and the multi-head time attention module acts independently on the sequence; the calculated data is connected and then linearly transformed to obtain the output H Attn : MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O , in, Both represent the projection matrix of linear transformation; H Attn Passed to the residual network, each sublayer is a feed-forward network and layer normalization defines the representation of the output of the temporal attention module as Where D = h × d model , h represents the number of attention heads.

5. The traffic flow prediction method based on spatiotemporal causal attention network according to claim 1 is characterized in that: In S4, a bidirectional channel selection recurrent network is used to capture the temporal dependency of the flow time series; the gated unit of GRU is replaced by AGAT, that is, the spatially correlated time series data X″ is used. t As the input data of GRU at any time t, it captures the temporal dependency of traffic data: in, represents the output of the previous moment, [·] represents the concatenation operation on the feature dimension, represents the candidate hidden layer state, ⊙ represents element-wise multiplication, Represents the output at the current moment; Use residual modules to connect the layers of the network to enhance the model's long-term capture ability: Among them, ω1 and ω2 both represent one-dimensional convolution kernels, ε represents a nonlinear activation function, represents the convolution operation, represents the output of the remaining cascade; W represents the weight matrix; A represents the adjacency matrix; X represents the input information at the current moment; E At ∈R N×d represents the embedding dictionary encoding each traffic road node; Until the Tth time step, the above operations are completed and the spatiotemporal dependency is obtained. The reverse operation is the same. Finally, the forward output and the reverse output are concatenated to get the output H∈R N×T×2F″ , the formula is as follows:

6. The traffic flow prediction method based on spatiotemporal causal attention network according to claim 1 is characterized in that: In S5, the dynamic association information between road nodes at different times is learned in a fine-grained manner through an adaptive graph learning mechanism to generate a traffic subgraph at the corresponding time; The adaptive graph learning mechanism generates the adjacency matrix corresponding to any time t Among them, E At ∈R N×d is the embedding dictionary for encoding each traffic road node, d represents the embedding dimension of the node, Indicates E At The transposed matrix of ; ReLU represents the nonlinear activation function, and softmax represents the normalization function; At any time t, GAT calculates the attention coefficients of its neighboring nodes point by point based on the node-related information generated by the node adaptive learning mechanism; aggregates the spatial dependencies of the nodes on the graph at time t; and shows that the node v i Any adjacent node v j Attention coefficient The calculation process: Where a represents the attention mechanism calculation function, W∈R F×F′ Represents the weight matrix of all nodes in the graph; by i Normalize the attention coefficients of all adjacent nodes to obtain the attention coefficient of the attention layer of the graph in, Indicates v i All neighbor nodes of Build a shared weight pool W p ∈R d×F×F′ , according to the node embedding dictionary E At Get the weight matrix W′=E for each node At ·W p ∈R F×F′ ; The calculation process of attention coefficient is: in, represents the weight matrix, k represents the connection operation, and LeakReLu represents the nonlinear activation function; self-attention is extended to the multi-head attention mechanism, that is, the above equation is called Group independent attention mechanisms and then stitch the outputs together: in, Indicates the qth th The weight coefficient calculated by the attention mechanism of the group at time t, represents the weight matrix of the corresponding group, Indicates that through node v i New feature representations obtained through the attention layer of the multi-head graph; Through the weight pool W′ p ∈R d×QF′×F″ Change the output dimension of the node from Map to Get the final output When all nodes in the graph have completed the above graph attention layer operation, the output feature is obtained The time corresponding to each node; this process is expressed as: Among them, σ(·) represents the computation function of the graph attention layer.

Images