Traffic flow prediction method and device based on global-local space-time dependency interaction

Abstract

The invention discloses a traffic flow prediction method and device based on global-local space-time dependency interaction. The method comprises the following steps: acquiring historical traffic flow data; performing data embedding on the historical traffic flow data by using a plurality of embedding modes, and synthesizing the obtained plurality of embedding representations by using a learnable weight to obtain a comprehensive embedding representation of the historical traffic flow data; encoding the comprehensive embedded representation by using a preset local space self-attention mechanism, a similarity-based space self-attention mechanism and a global time self-attention mechanism to obtain a local space feature, a remote space similarity feature and a time feature; fusing the remote spatial similarity feature and the time feature to obtain a preliminary fusion feature, and fusing the preliminary fusion feature and the local spatial feature to obtain a final fusion feature; and the traffic flow is predicted according to the final fusion features, so that the traffic flow prediction accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation, and in particular relates to a traffic flow prediction method and device based on global-local spatiotemporal dependency interaction. Background Art

[0002] With the acceleration of urbanization, traffic management faces increasingly severe challenges. Traditional traffic management systems are struggling to respond to emergencies, alleviate congestion during peak hours, and improve traffic efficiency. In the context of smart city development, intelligent transportation systems (ITS) are considered a core technology for optimizing traffic management. By integrating information technology and real-time data analysis, ITS not only optimizes the allocation of traffic resources but also improves the commuting experience. Among the many functions of intelligent transportation systems, traffic flow forecasting is widely considered a key technology and has long been an important research topic in the field of traffic management and planning. Accurate traffic flow forecasting is not only crucial for improving traffic efficiency but also significantly optimizes the allocation of traffic resources.

[0003] The primary task of traffic flow forecasting is to predict future traffic flow trends based on historical observational data. Modern traffic flow forecasting leverages machine learning and deep learning models to effectively capture the dynamic characteristics of traffic flow and plays a crucial role in route planning, vehicle scheduling, and congestion relief.

[0004] A key challenge in traffic flow prediction lies in effectively capturing and modeling the complex dynamic spatiotemporal dependencies in traffic data. To address this issue, numerous deep learning models have been proposed and widely used. Early solutions included the use of convolutional neural networks (CNNs) to capture spatial dependencies in grid-based traffic data, while recurrent neural networks (RNNs) were used to learn dynamic temporal dependencies. Subsequently, graph neural networks (GNNs) gained widespread application due to their advantages in modeling the underlying graph structure of traffic data. Consequently, GNN-based methods have gained significant popularity in the field of traffic prediction. While these models have achieved some success, several issues remain, including difficulty in effectively capturing the significant dynamics of traffic flow and a failure to effectively balance long-term and short-term temporal dependencies when modeling temporal correlations. This results in insufficient capture of inter-node temporal dependencies, impacting the accuracy of traffic flow predictions. Furthermore, existing models have limitations in the interaction and fusion of spatiotemporal features, making it difficult to fully capture the deep interactions between temporal and spatial features, leading to inaccurate traffic flow predictions. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a traffic flow prediction method and device based on global-local spatiotemporal dependency interaction.

[0006] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] A traffic flow prediction method based on global-local spatiotemporal dependency interaction, comprising:

[0008] Obtain historical traffic flow data;

[0009] Embedding the historical traffic flow data using a plurality of embedding methods respectively, and synthesizing the obtained multiple embedding representations using learnable weights to obtain a comprehensive embedding representation of the historical traffic flow data;

[0010] The comprehensive embedding representation is encoded using a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism to obtain local spatial features, long-range spatial similarity features, and temporal features; wherein the similarity is the similarity of traffic flow states;

[0011] fusing the remote spatial similarity feature and the temporal feature to obtain a preliminary fused feature, and fusing the preliminary fused feature and the local spatial feature to obtain a final fused feature;

[0012] Traffic flow is predicted based on the final fusion features.

[0013] Optionally, the multiple embedding methods include: high-dimensional embedding, time period embedding and spatial embedding; wherein,

[0014] The high-dimensional embedding is used to map the historical traffic flow data to a high dimension through a fully connected layer to obtain a high-dimensional embedding representation;

[0015] The time period embedding includes: weekly period embedding, daily period embedding and time position coding;

[0016] The weekly period embedding is used to convert the time in the historical traffic flow data into a weekly index and then generate a high-dimensional weekly embedding representation;

[0017] The daily period embedding is used to convert the time in the historical traffic flow data into minute indexes and generate a high-dimensional daily embedding representation;

[0018] The time position coding is a time position coding introduced into the historical traffic flow data by using the position coding method in the Transformer model;

[0019] The spatial embedding is used to capture the spatial features of the historical traffic flow data through a spatial graph Laplace embedding method to obtain a high-dimensional spatial embedding representation.

[0020] Optionally, the multiple embedded representations obtained by synthesis to obtain a comprehensive embedded representation of the historical traffic flow data include:

[0021] X emb =a data X data +a spe X spe +a w X w +a d X d +a tpe X tpe ;

[0022] Among them, X data is the high-dimensional embedding representation, X spe is the spatial embedding representation, X w is the week embedding representation, X d is the embedding representation of the day, X tpe Encode the time position; a data 、a spe 、a w 、a d 、a tpe X data 、X spe 、X w 、X d 、X tpe The corresponding learnable weight, X emb is a comprehensive embedding representation of the historical traffic flow data.

[0023] Optionally, the method of encoding the comprehensive embedding representation using the local spatial self-attention mechanism to obtain the local spatial features includes:

[0024] At each time step, generating a first query vector, a first key vector, and a first value vector for the comprehensive embedding representation, thereby achieving feature modeling of the spatial dimension and the traffic flow dimension at the time step;

[0025] Calculating a global spatial attention weight based on the first query vector and the first key vector by a scaled dot product operation;

[0026] Generate a global spatial attention matrix based on the global spatial attention weight and the first value vector, so as to describe spatially close nodes using the global spatial attention matrix; the nodes are nodes in the traffic scene to which the historical traffic flow data belongs;

[0027] Calculate the local space matrix based on the distance information between nodes;

[0028] Generate the local spatial features according to the global spatial attention matrix, the local spatial matrix and the first value vector.

[0029] Optionally, encoding the comprehensive embedding representation using the similarity-based spatial self-attention mechanism to obtain the long-range spatial similarity feature includes:

[0030] The dynamic time warping (DTW) algorithm is used to determine the node set that is similar to the historical traffic flow data of each node as the similarity cluster of the node, and a spatial similarity matrix between the nodes is constructed based on the determination results.

[0031] For each node, the historical traffic flow data of each node in its similar cluster is used to determine the centroid historical traffic flow data of the similar cluster using the K-shape clustering algorithm;

[0032] constructing a historical similarity matrix for all nodes based on the comprehensive embedding and the historical traffic flow data of each centroid;

[0033] updating the first key vector according to the historical similarity matrix;

[0034] Calculating a long-range spatial similarity attention weight based on the first query vector and the updated first key vector by a scaled dot product operation;

[0035] The long-range spatial similarity feature is generated according to the long-range spatial similarity attention weight, the spatial similarity matrix and the first value vector.

[0036] Optionally, encoding the comprehensive embedding representation using the global temporal self-attention mechanism to obtain the temporal feature includes:

[0037] Based on the comprehensive embedding representation, a multi-head self-attention mechanism is used to obtain the temporal attention distribution;

[0038] Extracting global temporal features from the comprehensive embedding representation using a convolution kernel of size (12, 1);

[0039] The temporal feature is obtained according to the temporal attention distribution and the global temporal feature.

[0040] The present invention also provides a traffic flow prediction device based on global-local spatiotemporal dependency interaction, comprising:

[0041] Acquisition module, used to obtain historical traffic flow data;

[0042] an embedding module, configured to embed the historical traffic flow data using a plurality of embedding methods, and synthesize the obtained multiple embedding representations to obtain a comprehensive embedded representation of the historical traffic flow data;

[0043] an encoding module for encoding the comprehensive embedding representation using a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism to obtain local spatial features, long-range spatial similarity features, and temporal features; wherein the similarity is the similarity of traffic flow states;

[0044] A fusion module is used to fuse the long-range spatial similarity feature and the time feature to obtain a preliminary fusion feature, and to fuse the preliminary fusion feature and the local spatial feature to obtain a final fusion feature;

[0045] The prediction module is used to predict the traffic flow according to the final fusion features.

[0046] The traffic flow prediction method based on global-local spatiotemporal dependency interaction provided by the present invention utilizes multiple embedding methods to embed historical traffic flow data, and uses learnable weights to synthesize the obtained multiple embedding representations, so that the model can dynamically adjust the embedding weights of different information sources according to actual conditions. In addition, the present invention utilizes a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism to encode the comprehensive embedding representation, which can effectively capture the spatiotemporal dependency in traffic flow prediction, explicitly extract long-term information, and achieve an effective balance between long-term and short-term temporal dependencies, as well as comprehensively capture the deep interactive relationship between temporal and spatial features. In addition, the present invention fuses remote spatial similarity features and temporal features to obtain preliminary fused features, and fuses the preliminary fused features with the local spatial features to obtain final fused features. Through this hierarchical interactive fusion mechanism, the interactive relationship between features is fused and optimized layer by layer, accurately capturing spatiotemporal dependencies. Taking all the above advantages into consideration, the present invention effectively improves the accuracy of traffic flow prediction.

[0047] The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The spatial dependence, temporal dependence and long-range spatial dependence in urban transportation systems are shown;

[0049] Figure 2 A schematic diagram of a traditional feature fusion method is shown;

[0050] Figure 3 1 is a flow chart of a traffic flow prediction method based on global-local spatiotemporal dependency interaction provided by an embodiment of the present invention;

[0051] Figure 4A schematic diagram showing a data embedding layer in the present invention is shown;

[0052] Figure 5 Shown is a schematic diagram of the Global TSA module of the present invention;

[0053] FIG6( a ) shows a schematic diagram of the deep learning model GLFormer in the present invention;

[0054] FIG6( b ) shows a schematic diagram of how the spatiotemporal coding modules in the present invention are cascaded;

[0055] Figure 7 The prediction performance comparison of the GLFormer model of the present invention and several existing models at different prediction step sizes on the PEMS04 dataset is shown;

[0056] Figure 8 The prediction performance comparison of the GLFormer model of the present invention and several existing models at different prediction step sizes on the PEMS08 dataset is shown;

[0057] Figure 9 The prediction performance of the proposed GLFormer model and its three variants under different time settings on the PEMS08 dataset is demonstrated;

[0058] Figure 10 The prediction performance of the proposed GLFormer model and its three variants under different time settings on the PEMS07 dataset is demonstrated;

[0059] Figure 11 A Friedman test graph obtained by performing a Friedman test to verify the performance difference between the GLFormer model of the present invention and several existing models is shown;

[0060] Figure 12 The comparison results of the 15-minute traffic flow predicted by the GLFormer model of the present invention and the actual value at node 0 set in the experiment are shown;

[0061] Figure 13 The comparison results between the predicted values and actual values of 30-minute traffic flow of the GLFormer model at node 0 set in the experiment are shown;

[0062] Figure 14 The comparison between the predicted value and the actual value of 60-minute traffic flow of the GLFormer model at node 0 set in the experiment is shown;

[0063] Figure 15 The comparison results between the GLFormer model's prediction results and the actual values of the 60-minute traffic flow at node 1 set in the experiment are shown. DETAILED DESCRIPTION

[0064] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0065] Figure 1 It shows the spatial dependence, temporal dependence and long-range spatial dependence in urban transportation system. Figure 1 It can be seen that the spatial dependence in urban traffic systems exhibits significant dynamic characteristics, and this characteristic is affected by multiple factors such as travel patterns and emergencies. However, most existing methods rely on static adjacency matrices (such as matrices based on distance or topological relationships) to model spatial dependence, which limits the adaptability of the model to dynamic traffic patterns. In addition, the long-distance spatial dependence problem brought about by urban functional zoning further increases the complexity of modeling. Some distant locations may exhibit similar traffic patterns (such as Figure 1 Nodes A and C in (a) and (c), but existing locality-based models are difficult to effectively capture these long-range dependencies.

[0066] The time dependency in urban traffic systems also exhibits significant dynamic characteristics, which is affected by multiple factors such as weather and holidays, and changes over time. However, many existing models usually focus on the dependency modeling of a single time scale (such as long-term or short-term) in time dimension modeling, and fail to effectively balance long-term and short-term time dependencies. This limitation is particularly disadvantageous when dealing with the diversity and periodicity of traffic flow. In fact, the traffic data of road nodes are essentially time series, and short-term and long-term time dependencies need to be modeled simultaneously. Although some methods have proposed solutions, they still have shortcomings in balancing long-term and short-term time dependencies. Especially when the traffic patterns on holidays and weekdays are significantly different (such as Figure 1 As shown in (d) in the figure), modeling that only focuses on a single time scale cannot effectively capture the differences between different traffic patterns, thus affecting the capture of overall trends and the identification of global traffic patterns.

[0067] In traffic flow prediction, temporal and spatial features are highly correlated, but traditional feature fusion methods have limitations in dealing with complex temporal and spatial dependencies. Common fusion methods can generally be divided into the following two categories: one method processes temporal and spatial features sequentially, or vice versa (e.g. Figure 2 Although it performs well in simple scenarios, this method cannot simultaneously model the interaction between temporal and spatial features. Another method models temporal and spatial features separately and finally fuses them (e.g. Figure 2Although this method overcomes the limitation of the former module-by-module processing, the independent operation of each module leads to insufficient interaction between different features and lack of deep fusion, which may lead to the loss of key information.

[0068] To address the aforementioned issues and improve the accuracy of traffic flow prediction, an embodiment of the present invention provides a traffic flow prediction method based on global-local spatiotemporal dependency interaction. Its core goal is to use historical traffic flow data to predict future traffic conditions (such as flow rate and speed). This method formalizes the traffic flow prediction problem as traffic flow observation data based on T historical steps. A mapping function f (deep learning model) is learned to map the traffic conditions in the past T time steps to traffic flow predictions in the future T′ time steps. This process can be expressed as:

[0069] [X t-T+1 ,…,X t ; G]→f[X t+1 ,…,X t+T ′].

[0070] Where G is a directed graph constructed based on the traffic network G = (V, E, A), V = {v1,…,v n} is a set of N nodes, is the set of edges representing node connectivity, A is the adjacency matrix of the graph G, and N is the number of nodes in the graph. In practice, the nodes are nodes in the traffic scene to which the historical traffic flow data belongs. For example, they can be various environmental perception nodes (such as various sensors) in the traffic scene, but this is not limited to these.

[0071] Historical traffic flow data is represented by traffic flow tensor: Let X t ∈R N×C represents the traffic status of N nodes in the road network at time t, where C is the characteristic dimension of traffic flow. For example, when the data contains inflow and outflow, C = 2. The traffic flow data of all nodes in T time slices can be represented as a tensor X = (X1, X2, ..., X T )∈R T×N×C , R represents a real number.

[0072] like Figure 3 As shown, the traffic flow prediction method based on global-local spatiotemporal dependency interaction provided by the embodiment of the present invention includes the following steps:

[0073] S10. Obtain historical traffic flow data.

[0074] Here, obtaining historical traffic flow data is to obtain tensor X.

[0075] S20. Embed the historical traffic flow data using a variety of embedding methods, and synthesize the obtained multiple embedding representations using learnable weights to obtain a comprehensive embedding representation of the historical traffic flow data.

[0076] In the task of traffic flow prediction, due to the significant dynamic characteristics of traffic flow and the uneven distribution of sensors, it is difficult for the original data to directly reflect the global traffic pattern. To this end, this paper designs an improved data embedding layer. Figure 4 A schematic diagram of the data embedding layer in the present invention is shown. This data embedding layer comprehensively captures spatiotemporal features by integrating multiple embedding methods and introduces a dynamic weighting mechanism, which enables the model to dynamically adjust the contribution of different embeddings according to actual scenarios, thereby improving the ability to capture complex traffic patterns.

[0077] See also Figure 4 In the present invention, the data embedding layer includes multiple embedding methods including: high-dimensional embedding, time period embedding and space embedding.

[0078] Among them, high-dimensional embedding is used to map historical traffic flow data to high dimensions through a fully connected layer (FC) to obtain a high-dimensional embedding representation.

[0079] Specifically, let X∈R T×N Converted to a high-dimensional representation and mapped to X through a fully connected layer data ∈R T×N×d , where T represents the number of time slices, N represents the number of nodes, and d is the embedding dimension. The generation of high-dimensional representations provides a foundation for subsequent operations, enabling the model to process complex spatiotemporal information in a unified representation space.

[0080] Spatial embedding is used to capture the spatial features of historical traffic flow data through the spatial graph Laplacian embedding method to obtain a high-dimensional spatial embedding representation.

[0081] Specifically, the road network in reality can be regarded as an irregular graph structure, which is usually converted into an analyzable form using the Laplacian matrix to capture the characteristics and connection relationships of the road network. The normalized Laplacian matrix is defined as:

[0082]

[0083] Where A is the adjacency matrix, D is the degree matrix, and I is the identity matrix. By performing eigenvalue decomposition on Δ, we obtain the eigenvalue matrix Λ and the eigenvector matrix U.

[0084] Then, in order to ensure the effective extraction of information, the l smallest non-trivial eigenvectors from the 2nd to the l+1th columns of the eigenvector matrix U are selected and projected into the linear space to generate the spatial embedding representation X spe ∈R N×dThis spatial embedding method can retain the global topological structure information of the traffic network in Euclidean space, thereby providing effective spatial feature support for deep learning models.

[0085] Traffic flow exhibits significant temporal cyclical characteristics, such as morning and evening peaks and nighttime troughs. To capture this characteristic, this paper designs temporal period embedding, including weekly and daily period embedding, and temporal position encoding. These temporal embedding methods collectively provide sufficient information support for modeling the temporal characteristics of traffic data.

[0086] Specifically, the weekly embedding is used to convert the time t in the historical traffic flow data into the weekly index w(t) and then generate a high-dimensional weekly embedding representation X w ∈R T×d The daily period embedding is used to convert the time t in the historical traffic flow data into minute index d(t) and then generate a high-dimensional daily embedding representation X d ∈P T×d The time position code z is the time position code X introduced by the position coding method in the Transformer model for historical traffic flow data. tpe ∈R T×d , thereby further enhancing the model's ability to model time information.

[0087] In addition, considering that the spatiotemporal relationship of traffic flow in different scenarios has dynamic characteristics, the contribution of each embedding in the model may not be fixed. The present invention introduces a learnable dynamic weighting mechanism to dynamically adjust the contribution value of each type of embedding through learnable weights, thereby synthesizing multiple embedding representations based on the learnable weights. Therefore, in one implementation, the multiple embedding representations synthesized in step S20 to obtain a comprehensive embedding representation of historical traffic flow data can be:

[0088] X emb =a data X data +a spe X spe +a w X w +a d X d +a tpe X tpe ;

[0089] Among them, X data is a high-dimensional embedding representation, X spe is the spatial embedding representation, X w is the week embedding representation, X d is the day embedding representation, X tpe is the time position code; a data 、a spe、a w 、a d 、a tpe X data 、X spe 、X w 、X d 、X tpe The corresponding learnable weight, X emb A comprehensive embedding representation for historical traffic flow data.

[0090] Here, the dynamic weighting mechanism adopted by the present invention dynamically adjusts the weights of various types of information by assigning learnable parameters to each type of embedded information to better cope with the dynamic changes in traffic flow, so that the deep learning model can adaptively adjust the proportion of various types of embeddings according to the specific traffic flow pattern, thereby more accurately capturing the spatiotemporal characteristics of the data.

[0091] In addition, to ensure the stability of the deep learning model and avoid overfitting, the final comprehensive embedding representation undergoes layer normalization (Layernorm) and (Dropout) operations at the output, such as Figure 4 As shown in . Among them, layer normalization standardizes the scales of different embeddings to avoid the problem of numerical imbalance after embedding splicing; Dropout randomly discards some features to enhance the generalization ability of the model. This process is expressed as:

[0092] X=Layernorm(Dropout(X emb )).

[0093] Here, X serves as the input of the subsequent spatiotemporal encoding step S30, providing rich spatiotemporal representation capabilities for traffic flow prediction.

[0094] S30. Encode the comprehensive embedding representation using a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism to obtain local spatial features, long-range spatial similarity features, and temporal features; wherein the similarity is the similarity of traffic flow states.

[0095] Specifically, the present invention designs and utilizes a spatiotemporal coding layer within a deep learning network. Figure 6(a) shows a schematic diagram of the deep learning model GLFormer in the present invention, which also illustrates the structure of the spatiotemporal coding layer. By utilizing a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism within this spatiotemporal coding layer, spatiotemporal coding of the integrated embedding representation is achieved.

[0096] The local spatial self-attention mechanism is implemented by the Local SS module in the spatiotemporal coding layer. In the Local SS module, the local spatial self-attention mechanism is used to encode the comprehensive embedding representation to obtain local spatial features, including:

[0097] (1) At each time step t, the first query vector is generated by First key vector and the first value vector V t (S,F) , to achieve feature modeling of the spatial dimension and traffic flow dimension at this time step, so as to analyze the spatial position of nodes in the road network and their traffic flow characteristics:

[0098]

[0099] The symbols (S, F) represent the spatial dimension and traffic flow feature dimension respectively. is a learnable parameter, d′ is the dimension of the first query vector, the first key vector, and the first value vector, X T is the transpose of X.

[0100] (2) Calculate the global spatial attention weight based on the first query vector and the first key vector through a scaled dot product operation;

[0101] Specifically, the correlation between any two nodes is calculated by scaling the dot product operation to obtain the global spatial attention weight. The operation is as follows:

[0102]

[0103] (3) Generate a global spatial attention matrix based on the global spatial attention weight and the first value vector to describe spatially close nodes using the global spatial attention matrix:

[0104]

[0105] Where Softmax(·) is the Softmax function. SSA represents the global spatial attention matrix.

[0106] (4) Calculate the local space matrix M based on the distance information between nodes loc , calculated as follows:

[0107]

[0108] Where dis(i,j) represents the distance (or number of hops) between node i and node j, λ is a predefined threshold, ⊙ represents the Hadamard product, and M loc (i,j) is the local space matrix Mloc The (i, j)th element in is used to describe whether nodes i and j are close in spatial distance. loc The mutual influence of road traffic conditions within a certain range in the traffic network can be taken into account.

[0109] (5) According to the global spatial attention matrix Local space matrix M loc and the first value vector V d (S,F) , generate local spatial features;

[0110] Specifically, by introducing the adjacency space matrix M when calculating the adjacency space self-attention loc , to capture the mutual correlation in the adjacent space and finally generate local spatial features:

[0111]

[0112] Therefore, by limiting the scope of attention, we ensure that the adjacent space self-attention mechanism (local space self-attention mechanism) can capture the interactive relationship within a limited spatial range and reflect the mutual influence of traffic conditions between adjacent road sections, thereby solving the dynamic characteristics of spatial dependence and the problem of long-distance spatial dependence.

[0113] Considering that similar traffic flow states may exist between different roads, in order to effectively capture these similar traffic patterns, the present invention designs a Similarity SSA module in the spatiotemporal coding layer to implement a similarity-based spatial self-attention mechanism. This mechanism constructs a similarity space matrix through the dynamic time warping (DTW) algorithm and updates the key-value matrix in combination with the K-shape clustering algorithm to capture the short-term spatial correlation of adjacent nodes in similar time periods, thereby expanding the influence range of the current node.

[0114] Specifically, in the Similarity SSA module, the method of encoding the comprehensive embedding representation using the similarity-based spatial self-attention mechanism to obtain long-range spatial similarity features includes:

[0115] (a) Using the dynamic time warping (DTW) algorithm, we determine the set of nodes that are similar to the historical traffic flow data of each node as the similarity cluster of the node, and construct a spatial similarity matrix between the nodes based on the determination results;

[0116]

[0117] Among them, top-K(i) represents the node set composed of K nodes that are most similar to the historical traffic flow data of node i calculated by the DTW algorithm, and each node set is a similar cluster.sim (i,j) is the spatial similarity matrix M sim The (i, j)th element in is used to describe whether the historical traffic patterns of nodes i and j are similar.

[0118] (b) For each node, based on the historical traffic flow data of each node in its similar cluster, the K-shape clustering algorithm is used to determine the centroid historical traffic flow data of the similar cluster;

[0119] Specifically, the K-shape clustering algorithm is used to cluster the historical traffic flow data of each node in the similar cluster, and the centroid historical traffic flow data p of the similar cluster is determined according to the clustering results. i , the centroid historical traffic flow data p i It can be used to represent the traffic pattern of the similar cluster, which, as a time series of length T, can more clearly describe and analyze the characteristics of the traffic pattern.

[0120] (c) Based on the comprehensive embedding and the historical traffic flow data of each centroid, a historical similarity matrix of all nodes is constructed.

[0121] Specifically, first calculate the historical traffic flow data similar to the historical traffic flow data of node n t,n :

[0122]

[0123] Among them, X t-T:t represents the traffic flow data at time step t and the previous T time steps, and W1, W2, and W3 are learnable parameters.

[0124] Then, the r of all N nodes t,n Summarized as the historical similarity matrix R t .

[0125] (d) According to the historical similarity matrix R t For the first key vector To update:

[0126]

[0127] in, Represents the updated first key vector.

[0128] (e) According to the first query vector and the updated first key vector The long-range spatial similarity attention weights are calculated by scaling the dot product operation:

[0129]

[0130] Here, when calculating the query matrix With the new key-value matrix When multiplying the product of and to obtain the spatial correlation score of time slice t, remote nodes with similar traffic flow status to the current node can be considered at the same time, thereby more accurately modeling the spatial correlation between nodes.

[0131] (f) Attention weights based on long-range spatial similarity Spatial similarity matrix M sim and the first value vector V t (S ,F) , generate long-range spatial similarity features:

[0132]

[0133] Therefore, by encoding the comprehensive embedding representation using a similarity-based spatial self-attention mechanism, we can effectively identify pairs of remote nodes with similar traffic patterns.

[0134] In the present invention, the local spatial self-attention mechanism and the similarity-based spatial self-attention mechanism are respectively used to spatially encode the comprehensive embedding representation, which can simultaneously combine local spatial features and long-range spatial similarity features, thereby capturing the spatial correlation between road network nodes more comprehensively and accurately.

[0135] In traffic flow prediction, the periodic and trend dependencies in traffic flow often vary between different time slices and nodes. Unlike traditional attention mechanisms based on time slice similarity modeling, this paper designs a Global TSA module in the spatiotemporal encoding layer to implement a global temporal self-attention mechanism. This mechanism effectively balances long-term and short-term temporal dependencies by combining a multi-head self-attention mechanism with global temporal convolution. Global temporal convolution explicitly extracts long-term information, while the multi-head self-attention mechanism captures holistic temporal dependencies, aiming to comprehensively capture both long-term and short-term temporal dependencies between nodes.

[0136] Specifically, Figure 5 The figure shows a schematic diagram of the GlobalTSA module in the present invention. In the GlobalTSA module, the global temporal self-attention mechanism is used to encode the comprehensive embedding representation to obtain the temporal features, including:

[0137] (i) Based on the comprehensive embedding representation, the multi-head self-attention mechanism is used to obtain the temporal attention distribution.

[0138] Specifically, the traffic flow at each time step in the comprehensive embedding representation is first represented as a vector, thus forming an N×D matrix. Then, for each time step, its N×D matrix is represented as three vectors, namely the second query vector Second key vector and the second value vector V t (T,F) ( Figure 5 (abbreviated as Q, K, V), (T, F) represents the time-traffic flow dimension.

[0139] Then, using the second query vector Second key vector and the second value vector V t (T,F) Calculate the time attention distribution. When calculating the time attention distribution, the multi-head self-attention mechanism usually uses the dot product method, that is, the second query vector and the second key vector The dot product of is used as the similarity score and is represented by the score matrix. Then, through the normalization function ( Figure 5 SoftMax in , processes these scores to obtain attention distribution. Finally, use these attention distributions to calculate the second value vector V t (T,F) Matrix multiplication is used to perform weighted averaging to obtain the output vector of the current time step. These output vectors are summarized to form the temporal attention distribution obtained by the multi-head self-attention mechanism. The above process can be expressed as follows:

[0140]

[0141] Among them, TSA is It is the abbreviation of time attention distribution.

[0142] (ii) extracting global temporal features from the comprehensive embedding representation using a convolution kernel of size (12, 1);

[0143] To enhance the model's ability to extract long-term temporal features, this paper introduces a global temporal convolution operation. This operation applies a convolution kernel of size (12, 1) to the comprehensive embedding representation X, covering the entire time series and extracting global trend features from the overall temporal structure. The formula for the global convolution operation (GTConv) is defined as follows:

[0144] GTConv(X)=W conv *X;

[0145] Among them, W conv ∈R 12×1 is the convolution kernel, and * denotes the convolution operation. This operation can capture periodic and trend information over long time spans in the temporal dimension. GTConv(X) is the global temporal feature.

[0146] (iii) Temporal features are obtained based on the temporal attention distribution and the global temporal features.

[0147] Here, see Figure 5 , the present invention combines the temporal attention distribution with the global temporal feature by adding them together to obtain the temporal feature:

[0148]

[0149] In the present invention, by introducing the global temporal convolution operation, long-term temporal information can be explicitly extracted in the time series. At the same time, the self-attention mechanism captures the overall temporal dependency and effectively combines the temporal attention distribution with the global temporal features, which can comprehensively capture the long-term and short-term temporal dependencies between nodes, thereby improving the performance of the deep learning model in capturing the overall temporal dependency, and further enhancing the ability of long-term and short-term comprehensive temporal dependencies, significantly improving the prediction performance and robustness of the model.

[0150] S40: Fusing the long-range spatial similarity features and the time features to obtain preliminary fusion features, and fusing the preliminary fusion features and the local spatial features to obtain final fusion features.

[0151] After obtaining local spatial features, long-range spatial similarity features and temporal features, these three features obtained by heterogeneous attention are first fused into the multi-head self-attention block to reduce the computational complexity of the model. Specifically, the attention head is divided into three types, namely local (LocSAH) head, similarity (SimSAH) head and temporal (TAH) head, which correspond to the three types of attention mechanisms respectively. Then, interactive fusion of features is performed. Since the spatial and temporal features in traffic flow are usually interdependent, if the temporal features and spatial features are processed separately and then fused, or the temporal features and spatial features are processed sequentially, the deep and complex interactive relationship between them may not be effectively captured. Therefore, the present invention uses a hierarchical interactive fusion mechanism to capture the deep interactive relationship and independent information between temporal and spatial features in traffic flow. Specifically:

[0152] As shown in Figure 6(a), the long-range spatial similarity features are first concatenated with the temporal features and fused via a multi-layer perceptron (MLP) to learn the complex interactions between these two dimensions. This approach not only enhances the model's understanding of the long-range spatial similarity and temporal features, but also more clearly identifies their joint impact on traffic flow. For example, during rush hour, holidays, or during weather emergencies (such as heavy rain or snow), traffic flow in different regions may exhibit similar patterns of change, which may be influenced by both global temporal features and long-range similarity spatial features.

[0153] Next, the preliminary fusion features obtained in the previous step are further spliced with the short-range spatial features (i.e., local spatial features), and then a position-by-position fully connected feedforward network (referred to as feedforward network) is applied so that the fused features simultaneously contain local spatial features, long-range spatial similarity features, and time information. The above process can be expressed as follows:

[0154]

[0155] in, Indicates splicing, Z sim is the output of the similarity head, Z t is the output of the time head, Z Loc is the output of the local head, W mlp ∈R d×d is the learnable projection matrix of the MLP, W O ∈R d×d is the learnable projection matrix for the feedforward network.

[0156] Finally, following the original Transformer architecture, layer normalization and residual connections (not shown in Figure 6(a)) are introduced to output the HIFM and obtain the final fusion features. The specific operations are as follows:

[0157] X hid =LayerNorm(HIFM+Sublayer(HIFM));

[0158] Among them, X hid ∈R T×N×d Represents the final fusion feature, LayerNorm represents layer normalization, and HIFM+Sublayer (HIFM) is a residual connection method.

[0159] In S40, layer-by-layer fusion significantly enhances the interaction between features by gradually optimizing the interaction between features. It can accurately capture the deep interaction and independent information between features, and finely capture the deep spatiotemporal dependencies, thus avoiding the problems of redundancy and insufficient interaction in traditional methods.

[0160] S50: predicting traffic flow based on the final fusion features.

[0161] Specifically, to improve the propagation efficiency of features and gradients in deep learning models and effectively avoid the problem of vanishing or exploding gradients, this paper cascades spatiotemporal coding layers and designs an innovative skip connection mechanism to pass important early information directly to later layers, bypassing potential interference from intermediate layers and preserving key temporal and spatial features. This approach enhances the fluidity of gradients, optimizes the model's ability to predict long-term dependencies, and thus improves training stability.

[0162] Specifically, see Figure 6(b), L spatiotemporal coding layers are cascaded together, and the final fusion feature X output by each spatiotemporal coding layer is hid is converted to a skip dimension d by a skip connection layer (such as a 1×1 convolution layer) skip ,form Sum the outputs of all skip connection layers to obtain the final hidden state features In order to perform multi-step traffic flow prediction, a direct output strategy is adopted, and dimension conversion is performed through two layers of 1×1 convolution (Conv) to obtain the final prediction result:

[0163]

[0164] in, represents the traffic flow data at predicted time step T′, and C represents the dimension of the predicted data. This design not only significantly reduces cumulative error but also excels in computational efficiency and predictive performance. The introduction of skip connections ensures efficient propagation of information and gradients within the deep network, preserving both temporal and spatial characteristics.

[0165] In one embodiment, Figure 6(a) shows the overall architecture of the deep learning model GLFormer used in this invention. This GLFormer model fully considers both global and local information when modeling temporal and spatial dependencies. Through a hierarchical interactive fusion mechanism, the model effectively captures the deep interactions and independent information between temporal and spatial features, leading to more accurate traffic flow predictions.

[0166] The traffic flow prediction method based on global-local spatiotemporal dependency interaction provided by the present invention utilizes multiple embedding methods to embed historical traffic flow data, and uses learnable weights to synthesize the obtained multiple embedding representations, so that the model can dynamically adjust the embedding weights of different information sources according to actual conditions. In addition, the present invention utilizes a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism to encode the comprehensive embedding representation, which can effectively capture the spatiotemporal dependency in traffic flow prediction, explicitly extract long-term information, and achieve an effective balance between long-term and short-term temporal dependencies, as well as comprehensively capture the deep interactive relationship between temporal and spatial features. In addition, the present invention fuses remote spatial similarity features and temporal features to obtain preliminary fused features, and fuses the preliminary fused features with the local spatial features to obtain final fused features. Through this hierarchical interactive fusion mechanism, the interactive relationship between features is fused and optimized layer by layer, accurately capturing spatiotemporal dependencies. Taking all the above advantages into consideration, the present invention effectively improves the accuracy of traffic flow prediction.

[0167] The beneficial effects of the present invention are described below using experimental data.

[0168] Specifically, the present invention conducted experiments based on real-world datasets to validate its effectiveness. During the experiments, the overall performance of the GLFormer model used in this invention was compared with that of various existing models. Comparative experiments were conducted for different prediction time targets, and model performance was evaluated through ablation experiments and parameter studies. Finally, statistical analysis and result visualization experiments were conducted to further validate the model's advantages.

[0169] First, let's describe the datasets used in the experiments. The experiments were conducted on three real-world road datasets in California (PEMS04, PEMS07, and PEMS08). These datasets are collected in real time every 30 seconds by the PEMS system. The raw data is processed and aggregated into 5-minute intervals. Furthermore, these datasets include spatial adjacency matrices constructed based on the real-world road networks. Table 1 provides more detailed information about the datasets.

[0170] Table 1

[0171]

[0172]

[0173] Then, several existing models involved in performance comparison are introduced:

[0174] DCRNN (Diffusion Convolutional Recurrent Neural Network): A neural network model that combines a diffusion graph convolutional network and a GRU (Gated Recurrent Unit). By modeling traffic flow changes as a one-dimensional convolutional diffusion process, DCRNN effectively captures spatial correlations and uses the GRU to model time series to predict future traffic flows.

[0175] STGCN (Spatial-Temporal Graph Convolutional Network): A spatial-temporal graph convolutional network that captures spatial and temporal correlations through a series of spatiotemporal convolutional blocks. This model embeds graph convolutions within one-dimensional convolutional units to extract spatial features while leveraging causal convolutions to process temporal information, enabling effective modeling of traffic data.

[0176] GWN (Graph WaveNet): Combines adaptive graph convolution and dilated one-dimensional convolution to capture spatiotemporal dependencies. GWN uses predefined and adaptive adjacency matrices to capture spatial associations and dilated convolutions to capture temporal associations, thereby expanding the receptive field and capturing long-range dependencies.

[0177] ASTGCN (Attention Based Spatial-Temporal Graph Convolutional Network): A spatiotemporal graph convolutional network that integrates an attention mechanism. This model introduces spatial and temporal attention mechanisms before spatiotemporal convolutions, adaptively weighting different temporal and spatial factors to capture hidden complex spatiotemporal patterns. ASTGCN specifically designs components to model temporal dependencies at different time scales (e.g., recent, daily, and weekly), enabling it to learn more expressive feature representations, significantly improving the accuracy of traffic forecasts.

[0178] STSGCN (Spatial-Temporal Synchronous Graph Convolutional Network): A novel spatiotemporal synchronous graph convolutional network. By constructing local spatiotemporal subgraphs and introducing a synchronous graph convolutional module, STSGCN simultaneously identifies and captures local spatial and temporal correlations, excelling in processing highly heterogeneous spatiotemporal network data. This synchronous processing design improves the ability to capture complex spatiotemporal relationships.

[0179] STFGNN (Spatial-Temporal Fusion Graph Neural Network): A spatial-temporal fusion graph neural network for traffic flow prediction. This model introduces the concept of a "temporal graph" and designs a novel adjacency matrix generated from raw data to capture implicit correlations that spatial graphs fail to capture. STFGNN also combines dilated convolutions, a gating mechanism, and a spatial-temporal fusion graph module to effectively capture long-range and long-term spatiotemporal dependencies, enabling a more comprehensive modeling of complex traffic flow data.

[0180] GMAN (Graph Multi-Attention Network): A spatiotemporal graph neural network that leverages multiple attention mechanisms. GMAN uses spatiotemporal attention to capture dynamic spatial and nonlinear temporal correlations and incorporates a gated fusion mechanism to adaptively fuse the outputs of spatiotemporal attention. GMAN also employs a transformation attention mechanism to extract more accurate predictive representations from historical features. By introducing a transformation layer, it reduces error propagation, making the model more stable and efficient when handling complex spatiotemporal dependencies.

[0181] DSTAGNN (Dynamic Spatial-Temporal-Aware Graph Neural Network): A dynamic spatial-temporal-aware graph neural network. This model generates dynamic spatial-temporal graphs using a data-driven approach, capturing the spatiotemporal interactions between nodes. DSTAGNN utilizes a spatial-temporal attention module based on multi-order Chebyshev polynomials to explore dynamic spatial dependencies, while combining it with an enhanced gated convolution module to improve its ability to capture dynamic temporal correlations. Furthermore, an enhanced multi-head attention mechanism further accurately models the dynamic connections between nodes, improving the ability to express and predict multi-scale spatiotemporal dependencies in complex spatiotemporal networks.

[0182] PDFormer (Propagation Delay-aware Dynamic Long-range Transformer): This model captures dynamic spatial dependencies through a spatial self-attention module and uses a graph masking matrix to model short-range and long-range spatial interactions. Furthermore, PDFormer incorporates a traffic delay-aware feature transformation module to fully account for the time delay in the propagation of traffic conditions, thereby improving its ability to model long-range spatial and temporal dependencies.

[0183] To enable fair comparison with existing models, the PEMS04, PEMS07, and PEMS08 datasets are divided into training, validation, and test sets in a ratio of 6:2:2. Furthermore, a multi-step forecasting approach is employed to predict traffic flow for the next hour (12 consecutive time steps) using data from the past hour (12 consecutive time steps).

[0184] All experiments were run on a computer equipped with an RTX 3080Ti GPU and 90GB of memory, using Ubuntu 20.04, and implementing GLFormer using PyTorch 1.10.0 and Python 3.8.10. The time step was set to 5 minutes.

[0185] The experimental hyperparameters are as follows: the number of minimum non-trivial eigenvectors is k = 4, the number of spatiotemporal coding layers is L = 6, and the hidden layer dimension is d skip =64, and the kernel size of the global temporal convolution is 12. The optimizer is AdamW, the learning rate is 0.001, the batch size is 16 (for the PEMS07 dataset with a large number of nodes, the batch size is adjusted to 4), and the number of training epochs is 200.

[0186] In the experiments, the following three widely adopted metrics are used to evaluate the prediction accuracy of the model: MAE (mean absolute error), RMSE (root mean square error), and MAPE (mean absolute percentage error).

[0187] The following are the test results and analysis:

[0188] (1) Overall comparison:

[0189] Table 2 shows the average 60-minute prediction results of the proposed GLFormer and 10 existing methods on the PEMS04, PEMS07, and PMS08 datasets. The results show that the proposed GLFormer outperforms the baseline models in all indicators on all three datasets, achieving the best results.

[0190] Table 2

[0191] Dataset Model MAE MAPE (%) RMSE Pems04 GraphWaveNet 25.45 17.29 39.70 DCRNN 24.70 17.12 38.12 STGCN 22.70 14.59 35.55 ASTGCN 22.93 16.56 35.22 STSGCN 21.19 13.90 33.65 STFGNN 19.83 13.02 31.88 GMAN 19.13 13.19 31.60 DSTAGNN 19.45 12.83 31.72 PDFormer 18.32 12.06 29.99 GLFormer 18.16 11.96 29.83 Pems07 GraphWaveNet 26.85 12.12 42.78 DCRNN 25.30 11.66 38.58 STGCN 25.38 11.08 38.78 ASTGCN 28.05 13.92 42.57 STSGCN 24.26 10.20 39.03 STFGNN 22.07 9.21 35.80 GMAN 20.96 9.05 34.09 DSTAGNN 21.72 9.25 34.96 PDFormer 19.83 8.53 32.87 GLFormer 19.59 8.28 32.74 Pems08 GraphWaveNet 19.13 12.68 31.05 DCRNN 17.86 11.45 27.83 STGCN 18.02 11.40 27.78 ASTGCN 18.61 13.08 28.16 STSGCN 17.13 10.96 26.80 STFGNN 16.64 10.60 26.22 GMAN 15.30 10.13 24.91 DSTAGNN 15.81 9.96 25.06 PDFormer 13.63 9.03 23.48 GLFormer 13.48 8.94 23.18

[0192] Among them, GraphWaveNet performs relatively poorly, primarily due to its limited ability to overlay spatiotemporal layers, resulting in a limited receptive field and difficulty capturing complex multi-scale spatiotemporal dependencies. Similarly, DCRNN and STGCN rely on fixed, predefined graph structures. While they have some advantages in capturing static spatial dependencies, they cannot adapt to the dynamic nature of spatiotemporal relationships, particularly when it comes to dynamic interactions between nodes. ASTGCN introduces an attention mechanism, which improves its ability to model short-term and cyclical dependencies, but fails to effectively balance long-term and short-term dependencies, and its capture of long-term dependencies remains limited. STSGCN emphasizes the simultaneous capture of local spatial and temporal features, performing well in highly heterogeneous spatiotemporal networks. However, its performance is significantly affected by data size. When data is insufficient, the construction of local spatiotemporal subgraphs may not fully reflect global characteristics, limiting the model's applicability. GMAN excels in capturing nonlinear dynamic dependencies through a multi-attention mechanism, but its weighted spatiotemporal attention fails to explicitly model the interaction between local and global dependencies. STFGNN and DSTAGNN use dynamic graph structure modeling, which significantly improves the model's ability to capture spatial dependencies that change over time. However, these methods generally focus on dynamic changes over large time spans and lack sensitivity to local fluctuations over short time scales. PDFormer significantly improves performance in traffic condition propagation scenarios by introducing a traffic delay perception module, but it is insufficient in capturing deep interactions between spatiotemporal features, and the extraction of long-term time information is relatively limited. Therefore, PDFormer performs worse than the GLFormer of the present invention. In addition, many existing models still rely on LSTM or GRU to model time dependencies. Although these methods can capture long-term and short-term dependencies in time series, they may not be able to effectively combine these two dependencies when dealing with long time spans or complex time series relationships, resulting in inaccurate prediction results.

[0193] The GLFormer of the present invention takes into account global and local spatiotemporal information in its design, aiming to comprehensively and deeply model spatial and temporal dependencies. In terms of spatial dependency modeling, a graph mask spatial self-attention mechanism is adopted to comprehensively capture complex spatial dependencies. In terms of temporal dependency modeling, the temporal features extracted by the attention mechanism are fused with the long-term temporal information captured by the global convolution, thereby achieving an effective fusion of long-term and short-term temporal features. In addition, with the help of a hierarchical interactive fusion mechanism, the interaction between features is optimized layer by layer to finely capture spatiotemporal dependencies. Through this design, the GLFormer model of the present invention avoids the problems of insufficient spatial and temporal dependency modeling caused by excessive emphasis on the local or global, as well as insufficient interaction between spatiotemporal features. Therefore, the GLFormer model of the present invention outperforms the baseline method in performance, especially on the PEMS07 dataset with the largest graph scale and the longest time step.

[0194] In order to evaluate the performance of the GLFormer model of the present invention in long-term and short-term prediction tasks, a number of existing models were selected, and the prediction performance of these existing models and GLFormer at different prediction step sizes were compared under the same experimental conditions. Specifically, Table 3 shows the performance of GLFormer and the selected baseline models on the PEMS04 and PEMS08 datasets, and compares the performance of 15-minute, 30-minute, and 60-minute prediction tasks. The results show that GLFormer significantly outperforms the baseline model in all evaluation indicators, whether for short-term or long-term prediction tasks. This performance improvement can be attributed to the concise and efficient module design in the GLFormer model, which aims to optimize the performance of long-term and short-term prediction tasks. In order to more intuitively show the comparison of the prediction performance of the baseline model and GLFormer at different prediction step sizes, Figure 7 and Figure 8 The data in Table 3 are visualized in the form of a line graph, where Figure 7 The prediction performance comparison of the GLFormer model of the present invention and several existing models at different prediction step sizes on the PEMS04 dataset is shown; Figure 8 The prediction performance comparison of the GLFormer model of the present invention and several existing models at different prediction step sizes on the PEMS08 dataset is shown.

[0195] Table 3

[0196]

[0197]

[0198] In general, as the time interval of the prediction target increases, the difficulty of prediction also increases, which leads to an increase in the prediction error. Figure 7 and Figure 8 As shown, models using predefined graph structures (such as DCRNN and STGCN) perform similarly for short-term predictions, but their accuracy decreases significantly as the prediction interval increases. In contrast, PDFormer, which uses an attention mechanism, shows a more gradual performance degradation. This is primarily because these models only consider the influence of neighboring nodes and fail to capture the dynamic spatiotemporal dependencies arising from temporal changes. PDFormer, through its attention-based architecture, better models the dynamic spatiotemporal dependencies that are crucial for long-term predictions. However, while PDFormer is able to capture both long-term and short-term dependencies, these dependencies are typically based on the dynamic relationship between temporal segments. Therefore, when the similarity between temporal segments is weak or the data is highly volatile, the model may struggle to accurately capture both dependencies, resulting in inferior performance compared to our GLFormer model. Results show that GLFormer outperforms the selected baseline models across all evaluation metrics, both for short-term and long-term prediction tasks. This superior performance is attributed to its unique model design, which fully considers both global and local influences in spatiotemporal modeling, explicitly extracts long-term temporal information, and effectively integrates long-term and short-term temporal features. In particular, it is based on the innovation of the hierarchical interactive fusion mechanism, which can deeply explore and integrate the spatiotemporal features of different scales while hierarchically and finely modeling spatiotemporal dependencies.

[0199] (2) Ablation experiment:

[0200] To further evaluate the effectiveness of each component in GLFormer, we conducted experiments with the following variants:

[0201] RemDWM: This variant removes the dynamic weighting mechanism in the data embedding layer and directly adds the embedded data, without dynamically adjusting the weights of each component. This variant aims to examine the contribution of the dynamic weighting mechanism to model performance.

[0202] RemHIFA: In the spatiotemporal encoding layer, the hierarchical interactive fusion attention layer was removed, leaving only the three-attention module alone. This experiment aimed to verify whether hierarchical interactive fusion attention plays a key role in the model's spatiotemporal information fusion.

[0203] RemGTConv: In Global TSA, the attention mechanism is retained but the GTConv module is removed. This variant is used to examine the importance of the GTConv module in extracting long-term temporal information.

[0204] During the experiment, we selected the PEMS08 dataset (with the smallest graph size) and the PEMS07 dataset (with the largest graph size) as representative datasets for ablation experiments. The experimental results show that GLFormer outperforms its variants, validating the effectiveness of each module in GLFormer. Table 4 shows the average comparison of prediction performance over 60 minutes.

[0205] Table 4

[0206] Dataset Module combination MAE MAPE (%) RMSE PMES08 RemDWM 13.56 9.04 23.31 RemHIFA 13.58 9.03 23.33 RemGTConv 13.61 9.07 23.47 GLFormer 13.48 8.94 23.18 PMES07 RemDWM 19.69 8.42 32.80 RemHIFA 19.70 8.40 32.82 RemGTConv 19.77 8.49 32.85 GLFormer 19.59 8.28 32.74

[0207] The ablation experiment results are shown in Figure 9 and Figure 10 , Figure 9 The prediction performance of the proposed GLFormer model and its three variants under different time settings on the PEMS08 dataset is demonstrated; Figure 10 This paper demonstrates the prediction performance of the GLFormer model and its three variants on the PEMS07 dataset under different time settings. The different prediction durations represent short-term and long-term predictions, respectively. Ablation experiments show that each component of the model contributes significantly to the overall performance improvement, leading to the following conclusions:

[0208] like Figure 9 and Figure 10 As shown in the figure, removing the dynamic weighting mechanism (RemDWM) significantly impacts short-term prediction performance. Due to the lack of a dynamic weighting mechanism, the model is unable to adjust the contribution of different embeddings based on the actual situation, resulting in a significant decrease in fusion effect, especially in short-term prediction tasks. However, in long-term prediction tasks, since other components can effectively capture long-term temporal features, removing the dynamic weighting mechanism has a smaller impact on performance, but still demonstrates its importance in improving overall performance.

[0209] After removing the Hierarchical Interaction Fusion Attention (RemHIFA) module, the model's performance on long-term prediction tasks significantly degraded. This demonstrates the irreplaceable role of the HIFA module in deeply mining the complex interactions and independent information between temporal and spatial features. In short-term prediction tasks, due to the lower complexity of feature interactions, removing HIFA only slightly degraded the model's performance. This demonstrates that while simple feature concatenation methods are still effective for short-term prediction, the HIFA module is particularly important in modeling long-term dependencies.

[0210] Removing the GTConv module (RemGTConv) significantly affects both long-term and short-term prediction performance. Figure 9 and Figure 10As shown in the figure, the GTConv module plays a key role in modeling long-term temporal features. Removing this module leads to a significant drop in performance on long-term prediction tasks. In addition, in short-term prediction tasks, the attention mechanism based on time slice similarity can still effectively capture temporal dependencies. Removing this module also leads to a significant drop in performance, further verifying the importance of the GTConv module in effectively fusing long-term and short-term information.

[0211] Overall, Table 4 and Figure 9 and Figure 10 The results collectively demonstrate the rationality of the design of the GLFormer model and the effectiveness of each module, further highlighting its powerful performance in traffic flow prediction.

[0212] (3) Parameter study:

[0213] This section of the experiment conducted a series of parameter studies to explore the impact of hyperparameter settings on model performance. The experiments were conducted on the PEMS08 dataset, focusing on control variables that significantly affect model performance. These variables include the number of stacked spatiotemporal coding layers, the convolution kernel size (KS) of the GTConv module, and the batch size. The following tables show the experimental results of these parameter studies, as detailed in Tables 5, 6, and 7. Table 5 shows the impact of the number of stacked spatiotemporal coding layers on the performance of the GLFormer model, Table 6 shows the impact of the convolution kernel size (KS) of the GTConv module on the performance of the GLFormer model, and Table 7 shows the impact of batch size on the performance of the GLFormer model.

[0214] Table 5

[0215] Number of stacking layers MAE MAPE (%) RMSE 2 13.89 9.25 24.01 4 13.67 9.15 23.74 6 13.48 8.96 23.18 8 13.63 9.18 23.65

[0216] Table 6

[0217] Convolution kernel size MAE MAPE (%) RMSE 6 13.72 9.12 23.66 9 13.61 9.05 23.51 12 13.48 8.96 23.18 Batch Size MAE MAPE (%) RMSE 8 13.64 9.12 23.51 16 13.48 8.96 23.18 32 13.69 9.17 23.54

[0218] Table 7

[0219]

[0220]

[0221] It can be observed that the number of stacked spatiotemporal encoding layers significantly affects model performance. This structure must strike a balance between fully capturing spatiotemporal features and controlling architectural complexity. As shown in Table 5, the MAE and RMSE metrics continue to improve as the number of layers increases from 2 to 6, attributed to the deep network's ability to progressively extract multi-scale spatiotemporal features. However, when the number of layers exceeds 6, reaching 8, the performance curve shows an inflection point, revealing the potential risks of deep architectures.

[0222] It is worth noting that architectures with insufficient layers (e.g., 2 to 4 layers) struggle to construct effective feature propagation paths, limiting their ability to model long-range spatiotemporal dependencies. This structural flaw is particularly pronounced in complex scenarios. When the layers are stacked excessively (e.g., 8 layers), while the theoretical receptive field expands, the actual lengthening of the gradient propagation path exacerbates signal attenuation, reducing the efficiency of updating the underlying network parameters. Furthermore, feature reuse in deep architectures can cause information interference, leading to distortion of key features during transmission. This negative effect is more likely to induce overfitting when training data is limited.

[0223] The experimental results in Table 5 demonstrate that the six-layer spatiotemporal encoding architecture exhibits optimal robustness: it has a sufficiently deep feature transformation module to decouple the spatiotemporal dimensions, and maintains the stability of the gradient flow through residual connections. This balanced design avoids the limited representation capabilities of shallow networks while also mitigating the risk of information degradation in deeper networks, effectively improving prediction accuracy.

[0224] Table 6 shows that the convolution kernel size in the GTConv module has a significant impact on temporal feature extraction, especially when combined with the attention mechanism. Larger convolution kernels (such as 12) can capture global information over longer time spans by expanding the receptive field, thereby more effectively identifying long-term trends. Table 6 shows the impact of different convolution kernel sizes on model performance. While smaller convolution kernels (such as 9 and 6) can capture long-term temporal dependencies, their smaller receptive fields prevent them from effectively capturing long-term dependencies spanning multiple time steps, resulting in poor performance when processing complex time series. Specifically, when the convolution kernel size is 12, the model is able to consider information from the past 12 time steps, capturing cyclical changes and long-term patterns in traffic flow, which is crucial for traffic flow prediction. In contrast, when the convolution kernel size is 9 or 6, the model focuses primarily on shorter-term temporal variations and has difficulty effectively extracting global trends and long-term dependencies. Therefore, a convolution kernel size of 12 improves the accuracy of long-term temporal information extraction, thereby improving prediction accuracy. This demonstrates that larger convolution kernels, by expanding the receptive field and combining with the attention mechanism, better achieve the effective fusion of long-term and short-term temporal features.

[0225] Batch size is a key factor in training deep learning models, directly impacting the stability and efficiency of the optimization process. The experiments in Table 7 show that a batch size of 16 achieves optimal model performance, with the MAE and RMSE reaching their lowest values. Larger batch sizes (e.g., 32) lead to a slight decrease in performance, likely because larger batches make gradient updates during training less refined, resulting in each update containing too much information, which in turn reduces the model's learning ability. On the other hand, smaller batch sizes (e.g., 8) frequently update model parameters, which can introduce higher noise during the update process and cause the model to become trapped in a local minimum. Furthermore, due to frequent parameter updates, models trained with small batches can experience unstable convergence. This suggests that there is a trade-off between the impact of batch size on model performance, and choosing an appropriate batch size can improve both model efficiency and accuracy.

[0226] (4) Statistical analysis:

[0227] To verify the performance difference between the GLFormer model and the existing models in Table 2, Friedman test and Nemenyi post hoc test were performed. The Friedman test assumes that all k compared methods perform consistently on N datasets. The first step is to calculate the adjustment statistic T according to the formula F and check statistics

[0228]

[0229]

[0230] Among them, r i represents the average ranking of the i-th model. In addition, T F The second step is to compare T F The hypothesis is tested with the corresponding critical value. If the hypothesis is rejected, it indicates that there is a significant difference in the performance of the compared models. Next, a post hoc test is needed to further distinguish the performance differences between the algorithms.

[0231] The Nemenyi test is a commonly used post hoc test method. The Nemenyi test uses the formula to calculate the critical distance (CD) to reflect the difference between the average ranking values of each method:

[0232]

[0233] Among them, q αDenotes the critical value of the Tukey distribution, and CD is the critical value of the Nemenyi test. When the difference in the average ranking of the two methods exceeds the critical distance range, it indicates that there is a significant difference in their performance; otherwise, the performance difference is not significant.

[0234] In this experiment, N=3, k=10. When α=0.05, according to T F The calculation formula for calculating MAE, RMSE and MAPE is T F From Table 8, we can see that the MAE, RMSE, and MAPE values all exceed the threshold of 2.4563.

[0235] Table 8

[0236] Metrics <![CDATA[T F ]]> MAE 2.9020 RMSE 3.1518 MAPE 4.3854

[0237] In other words, the assumption that all algorithms have the same performance has been rejected. Therefore, a Nemenyi test is needed for further verification. According to the CD calculation formula, CD = 8.1924 is calculated. Then, a Friedman test plot is generated. Figure 11 The Friedman test plot is a graph showing the performance difference between the GLFormer model of the present invention and several existing models. For each algorithm, the blue dot represents its average rank. The horizontal line centered on the blue dot represents the critical distance. If the horizontal lines do not overlap, it indicates a significant difference.

[0238] After comprehensive consideration, it is concluded that the GLFormer model proposed in this paper shows significant differences in MAE, RMSE and MAPE compared with STFGNN, STSGCN, ASTGCN, STGCN, DCRNN, Graph WaveNet and GMAN, but does not show significant differences compared with DSTAGNN and PDFormer, which have performed best in recent years. Figure 11 As shown in Figure 2, the method of the present invention performs best in the average ranking when using any indicator. In summary, the method of the present invention is statistically significantly better than other methods.

[0239] (5) Visualization of prediction results:

[0240] This section conducts traffic flow prediction experiments on the PEMS08 dataset, comparing the predicted results with the actual values for 2,500 time steps. It also conducts an in-depth analysis of the data for three specific time intervals: all hours of the second day, from noon on the fifth day to noon on the sixth day, and from noon on the seventh day to noon on the eighth day (indicated by the green dashed boxes). Figure 12 、 Figure 13 and Figure 14The comparison results of the GLFormer model of the present invention on the traffic flow of 15 minutes, 30 minutes and 60 minutes at node 0 are shown respectively. In order to further verify the consistency of the performance of the model at different nodes, Figure 15 The comparison between the predicted results of GLFormer of the present invention and the actual value of the 60-minute traffic flow at node 1 is shown.

[0241] from Figures 12-15 The following points can be observed: (1) GLFormer demonstrates superiority in multi-scale temporal dependency modeling, maintaining high prediction accuracy at multiple time scales. Whether it is short-term, medium-term, or long-term prediction, it can effectively reflect traffic flow changes and show good adaptability. (2) GLFormer demonstrates a strong ability to capture drastic changes in traffic flow, especially in the transition phase between peak and trough periods, accurately capturing traffic flow changes, demonstrating its excellent spatiotemporal dependency modeling capabilities. (3) GLFormer performs stably in cross-node prediction tasks, verifying its excellent cross-node generalization ability and its ability to adapt to traffic flow prediction needs in different geographical locations.

[0242] In summary, GLFormer demonstrates exceptional ability in accurately capturing traffic patterns across heterogeneous nodes and time periods. Experimental results demonstrate that GLFormer outperforms other baseline models on three real-world datasets, achieving state-of-the-art performance and validating its effectiveness in modeling spatiotemporal dependencies. The experimental results highlight the model's superior performance in capturing spatiotemporal dependencies, particularly in identifying abrupt traffic fluctuations during the transition between peak and off-peak hours, thus demonstrating its robust spatiotemporal dependency modeling capabilities.

[0243] In summary, the traffic flow prediction method based on global-local spatiotemporal dependency interaction proposed in the present invention fully considers the influence of global and local factors when modeling spatiotemporal dependencies, and gradually optimizes the interaction between features through a hierarchical interactive fusion mechanism, thereby accurately capturing spatiotemporal dependencies. In the process of spatial dependency modeling, the present invention adopts a graph mask spatial self-attention mechanism to model dynamic spatial dependencies from both short-range and long-range perspectives, striving to fully capture complex spatial dependencies. In temporal dependency modeling, the present invention combines the attention mechanism that captures overall temporal dependencies with global temporal convolution that effectively extracts long-term information, thereby improving the accuracy of long-term information extraction and effectively balancing long-term and short-term temporal dependencies. In addition, the present invention adopts an improved data embedding technology, which enables the model to dynamically adjust the embedding weights of different information sources according to actual conditions, so as to better cope with the dynamic characteristics of traffic flow.

[0244] The method provided in the embodiments of the present invention can be applied to electronic devices. Specifically, the electronic devices can be desktop computers, portable computers, smart mobile terminals, servers, etc. This is not limited here; any electronic device that can implement the present invention falls within the scope of protection of the present invention.

[0245] Corresponding to the above-mentioned traffic flow prediction method based on global-local spatiotemporal dependency interaction, a traffic flow prediction device based on global-local spatiotemporal dependency interaction includes:

[0246] Acquisition module, used to obtain historical traffic flow data;

[0247] an embedding module, configured to embed the historical traffic flow data using a plurality of embedding methods, and synthesize the obtained multiple embedding representations to obtain a comprehensive embedded representation of the historical traffic flow data;

[0248] an encoding module for encoding the comprehensive embedding representation using a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism to obtain local spatial features, long-range spatial similarity features, and temporal features; wherein the similarity is the similarity of traffic flow states;

[0249] A fusion module is used to fuse the long-range spatial similarity feature and the time feature to obtain a preliminary fusion feature, and to fuse the preliminary fusion feature and the local spatial feature to obtain a final fusion feature;

[0250] The prediction module is used to predict the traffic flow according to the final fusion features.

[0251] Optionally, the embedding module includes: a high-dimensional embedding submodule, a time period embedding submodule and a spatial embedding submodule;

[0252] The high-dimensional embedding submodule is used to map the historical traffic flow data to a high dimension through a fully connected layer to obtain a high-dimensional embedding representation;

[0253] The time period embedding submodule includes: weekly period embedding, daily period embedding and time position encoding; the weekly period embedding is used to convert the time in the historical traffic flow data into a weekly index and generate a high-dimensional weekly embedding representation; the daily period embedding is used to convert the time in the historical traffic flow data into a minute index and generate a high-dimensional daily embedding representation; the time position encoding is a time position encoding introduced into the historical traffic flow data using the position encoding method in the Transformer model;

[0254] The spatial embedding submodule is used to capture the spatial features of the historical traffic flow data through a spatial graph Laplace embedding method to obtain a high-dimensional spatial embedding representation.

[0255] Optionally, the embedding module synthesizes the multiple embedded representations to obtain a comprehensive embedded representation of the historical traffic flow data, including:

[0256] X emb =a data X data +a spe X spe +a w X w +a d X d +a tpe X tpe ;

[0257] Among them, X data is the high-dimensional embedding representation, X spe is the spatial embedding representation, X w is the week embedding representation, X d is the embedding representation of the day, X tpe Encode the time position; a data 、a spe 、a w 、a d 、a tpe X data 、X spe 、X w 、X d 、X tpe The corresponding learnable weight coefficient, X emb is a comprehensive embedding representation of the historical traffic flow data.

[0258] It should be noted that, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0259] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A traffic flow prediction method based on global-local spatiotemporal dependency interaction, characterized in that: include: Obtain historical traffic flow data; Embedding the historical traffic flow data using a plurality of embedding methods respectively, and synthesizing the obtained multiple embedding representations using learnable weights to obtain a comprehensive embedding representation of the historical traffic flow data; The comprehensive embedding representation is encoded using a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism to obtain local spatial features, long-range spatial similarity features, and temporal features; wherein the similarity is the similarity of traffic flow states; fusing the remote spatial similarity feature and the temporal feature to obtain a preliminary fused feature, and fusing the preliminary fused feature and the local spatial feature to obtain a final fused feature; Traffic flow is predicted based on the final fusion features.

2. The traffic flow prediction method based on global-local spatiotemporal dependency interaction according to claim 1 is characterized in that: The multiple embedding methods include: high-dimensional embedding, time period embedding and spatial embedding; wherein, The high-dimensional embedding is used to map the historical traffic flow data to a high dimension through a fully connected layer to obtain a high-dimensional embedding representation; The time period embedding includes: weekly period embedding, daily period embedding and time position coding; The weekly period embedding is used to convert the time in the historical traffic flow data into a weekly index and then generate a high-dimensional weekly embedding representation; The daily period embedding is used to convert the time in the historical traffic flow data into minute indexes and generate a high-dimensional daily embedding representation; The time position coding is a time position coding introduced into the historical traffic flow data by using the position coding method in the Transformer model; The spatial embedding is used to capture the spatial features of the historical traffic flow data through a spatial graph Laplace embedding method to obtain a high-dimensional spatial embedding representation.

3. The traffic flow prediction method based on global-local spatiotemporal dependency interaction according to claim 2 is characterized in that: The multiple embedded representations obtained by synthesis, to obtain a comprehensive embedded representation of the historical traffic flow data, include: X emb =a data X data +a spe X spe +a w X w +a d X d +a tpe X tpe ; Among them, X data is the high-dimensional embedding representation, X spe is the spatial embedding representation, X w is the week embedding representation, X d is the day embedding representation, X tpe Encode the time position; a data 、a spe 、a w 、a d 、a tpe X data 、X spe 、X w 、X d 、X tpe The corresponding learnable weight, X emb is a comprehensive embedding representation of the historical traffic flow data.

4. The traffic flow prediction method based on global-local spatiotemporal dependency interaction according to claim 1 is characterized in that: The method of encoding the comprehensive embedding representation by using the local spatial self-attention mechanism to obtain the local spatial features includes: At each time step, generating a first query vector, a first key vector, and a first value vector for the comprehensive embedding representation, thereby achieving feature modeling of the spatial dimension and the traffic flow dimension at the time step; Calculating a global spatial attention weight based on the first query vector and the first key vector by a scaled dot product operation; Generate a global spatial attention matrix based on the global spatial attention weight and the first value vector, so as to describe spatially close nodes using the global spatial attention matrix; the nodes are nodes in the traffic scene to which the historical traffic flow data belongs; Calculate the local space matrix based on the distance information between nodes; Generate the local spatial features according to the global spatial attention matrix, the local spatial matrix and the first value vector.

5. The traffic flow prediction method based on global-local spatiotemporal dependency interaction according to claim 4 is characterized in that: The method of encoding the comprehensive embedding representation by using the similarity-based spatial self-attention mechanism to obtain the long-range spatial similarity feature includes: The dynamic time warping (DTW) algorithm is used to determine the node set that is similar to the historical traffic flow data of each node as the similarity cluster of the node, and a spatial similarity matrix between the nodes is constructed based on the determination results. For each node, the historical traffic flow data of the centroid of the similar cluster is determined using the K-shape clustering algorithm based on the historical traffic flow data of each node in its similar cluster; constructing a historical similarity matrix for all nodes based on the comprehensive embedding and the historical traffic flow data of each centroid; updating the first key vector according to the historical similarity matrix; Calculating a long-range spatial similarity attention weight based on the first query vector and the updated first key vector by a scaled dot product operation; The long-range spatial similarity feature is generated according to the long-range spatial similarity attention weight, the spatial similarity matrix and the first value vector.

6. The traffic flow prediction method based on global-local spatiotemporal dependency interaction according to claim 1 is characterized in that: The method of encoding the comprehensive embedding representation using the global temporal self-attention mechanism to obtain the temporal feature includes: Based on the comprehensive embedding representation, a multi-head self-attention mechanism is used to obtain the temporal attention distribution; Extracting global temporal features from the comprehensive embedding representation using a convolution kernel of size (12, 1); The temporal feature is obtained according to the temporal attention distribution and the global temporal feature.

7. A traffic flow prediction device based on global-local spatiotemporal dependency interaction, characterized in that: include: Acquisition module, used to obtain historical traffic flow data; an embedding module, configured to embed the historical traffic flow data using a plurality of embedding methods, and synthesize the obtained multiple embedding representations to obtain a comprehensive embedded representation of the historical traffic flow data; an encoding module for encoding the comprehensive embedding representation using a preset local spatial self-attention mechanism, a similarity-based spatial self-attention mechanism, and a global temporal self-attention mechanism to obtain local spatial features, long-range spatial similarity features, and temporal features; wherein the similarity is the similarity of traffic flow states; A fusion module is used to fuse the long-range spatial similarity feature and the time feature to obtain a preliminary fusion feature, and to fuse the preliminary fusion feature and the local spatial feature to obtain a final fusion feature; The prediction module is used to predict the traffic flow according to the final fusion features.

8. The traffic flow prediction device based on global-local spatiotemporal dependency interaction according to claim 7, characterized in that: The embedding module includes: a high-dimensional embedding submodule, a time period embedding submodule and a spatial embedding submodule; The high-dimensional embedding submodule is used to map the historical traffic flow data to a high dimension through a fully connected layer to obtain a high-dimensional embedding representation; The time period embedding submodule includes: weekly period embedding, daily period embedding and time position encoding; the weekly period embedding is used to convert the time in the historical traffic flow data into a weekly index and generate a high-dimensional weekly embedding representation; the daily period embedding is used to convert the time in the historical traffic flow data into a minute index and generate a high-dimensional daily embedding representation; the time position encoding is a time position encoding introduced into the historical traffic flow data using the position encoding method in the Transformer model; The spatial embedding submodule is used to capture the spatial features of the historical traffic flow data through a spatial graph Laplace embedding method to obtain a high-dimensional spatial embedding representation.

9. The traffic flow prediction method based on global-local spatiotemporal dependency interaction according to claim 8 is characterized in that: The embedding module synthesizes the multiple embedded representations to obtain a comprehensive embedded representation of the historical traffic flow data, including: X emb =a data X data +a spe X spe +a w X w +a d X d +a tpe X tpe ; Among them, X data is the high-dimensional embedding representation, X spe is the spatial embedding representation, X w is the week embedding representation, X d is the day embedding representation, X tpe Encode the time position; a data 、a spe 、a w 、a d 、a tpe X data 、X spe 、X w 、X d 、X tpe The corresponding learnable weight coefficient, X emb is a comprehensive embedding representation of the historical traffic flow data.

Images