Lightweight space-time traffic flow prediction method based on GAT and Mama

Abstract

The invention discloses a lightweight space-time traffic flow prediction method based on GAT and Mama, and relates to the technical field of intelligent traffic systems. According to the method, the feature expression ability of the model is improved through multi-dimensional feature learning (time, date and space-time) of an embedded layer, rich features of input data can be deeply captured, and the learning ability of the model is enhanced; a graph attention module is used to adaptively pay attention to the relative importance between nodes, so that the graph structure relationship in the data can be more effectively captured and utilized, and the analysis accuracy of the model on the spatial data is improved; the GAT-Mama module optimizes the time sequence analysis and spatial data processing capability, and the model can more accurately capture the dynamic dependency relationship between time points and between spatial positions through a multi-layer graph attention mechanism and residual connection; the GAT-Mama layer structure supports flexible adjustment, a user can adjust the depth of the model according to specific task requirements so as to adapt to spatio-temporal data prediction tasks with different complexities, and good adaptability is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation systems, and in particular to a lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba. Background Art

[0002] Accurate traffic forecasting is essential for intelligent transportation systems, which enables proactive traffic management, alleviates congestion, and supports urban planning decisions. With the acceleration of urbanization and the increasing complexity of transportation networks, modeling and forecasting of traffic flows face significant challenges, mainly due to the inherent spatiotemporal dependencies in the data. Traditional statistical methods often have difficulty effectively capturing the nonlinear and dynamic characteristics of traffic data. In recent years, advances in deep learning have promoted the application of neural network architectures in modeling spatial and temporal dependencies. Recurrent neural networks (RNNs), especially long short-term memory (LSTM) networks and gated recurrent units (GRU), have been widely used in time series forecasting. However, RNN-based models have difficulties in capturing spatial dependencies and are difficult to effectively model long-term temporal dependencies due to problems such as gradient vanishing. Existing traffic forecasting models have limitations in dealing with the spatiotemporal dependencies of traffic data. Traditional methods often rely on fixed graph structures and have difficulty adapting to dynamic changes in transportation networks. Although some advanced models, such as Transformer-based models, perform well in long-range predictions, their high computational complexity limits their application in real-time predictions. Therefore, a new model is needed to improve the accuracy of traffic prediction while reducing the computational complexity.

[0003] In order to solve the spatial dependency problem, graph neural networks (GNNs) were introduced to effectively model the relational structure of traffic networks. Diffused Convolutional Recurrent Neural Network (DCRNN) and Graph WaveNet (GWNet) combine graph convolution with temporal modeling techniques to process spatiotemporal data. In addition, Adaptive Graph Convolutional Recurrent Network (AGCRN) further enhances the adaptability of the model by dynamically updating the spatial relationships between nodes. However, these methods usually rely on predefined or static graph structures, which limits their adaptability to changing traffic patterns.

[0004] Transformer-based models have attracted widespread attention due to their ability to effectively capture long-term dependencies using self-attention mechanisms. Models such as Graph Multi-Head Attention Network (GMAN), PDFormer, and Spatio-Temporal Autoencoder Transformer (STAEfrore) use multi-head attention mechanisms to model spatio-temporal correlations. Although these models perform well on complex data, they are accompanied by high computational complexity, making them less suitable for real-time prediction applications. Although the Reformer approach aims to reduce this complexity, it still has difficulties in achieving a balance between efficiency and accuracy.

[0005] State-space models have been revisited in sequence modeling due to their ability to efficiently capture long-term dependencies. The Selective State-Space Model (Mamba) introduces a method that combines state-space models with neural networks to effectively model complex temporal dynamics without the computational overhead of traditional recurrent units.

[0006] Therefore, a new solution to the above problems needs to be proposed. Summary of the invention

[0007] The purpose of the present invention is to provide a lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba, specifically a new model for spatiotemporal traffic prediction - GAT-Mamba, which combines Graph Attention Networks (GAT) and Selective State Space Models (Mamba) to improve the accuracy of traffic prediction at multiple time scales to solve the technical problems raised in the background technology.

[0008] To achieve the above object, the present invention provides the following technical solution: a lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba, comprising at least the following steps:

[0009] S1: Building a spatiotemporal traffic prediction model, which is used to improve the accuracy of traffic prediction at multiple time scales, and includes an embedding layer, a GAT-Mamba module, and a regression layer;

[0010] S2: Use the embedding layer to represent the original features of historical traffic data in high dimensions and generate feature representations suitable for model input;

[0011] S3: The GAT-Mamba module and regression layer perform spatiotemporal dependency analysis on traffic data. The generated features are processed by the regression layer module to generate the final prediction output for each node in the prediction range.

[0012] Furthermore, the embedding layer generates a high-dimensional spatial representation by integrating feature embedding, periodic embedding and spatiotemporal adaptive embedding.

[0013] Furthermore, the S2 at least includes the following steps:

[0014] Feature embedding uses a fully connected layer to calculate the embedding representation of each node data, that is, This fully connected layer is applied independently to each node at each time step, see the following formula:

[0015]

[0016] in: is the embedding representation of node i at time step t, which is a high-dimensional vector representing the characteristics of the node; It represents the original input characteristics of node i at time t; is the traffic characteristic of node i at time t; time t and day t These three original features are combined into a 3D vector and input into the fully connected layer. and are the learnable weight matrix and bias vector, respectively, d f is the dimension of each node embedding, i.e., the representation size of the node in the embedding space;

[0017] Then obtain the high-dimensional representation of the original features from each node;

[0018] Periodic embedding uses a learnable embedding tensor to represent the weekday and timestamp day features, denoted as and Where N w and N d Represents the number of working days and timestamps (i.e. 7 days, 24 hours), d f is the dimension of the embedding. At each time step, from T w and T d The corresponding weekday and timestamp embeddings are retrieved from ; the weekday and timestamp embeddings are then concatenated and broadcasted to form a periodic embedding of the traffic time series of length T Where N is the number of spatial nodes, and E is embedded periodically. p Effectively encodes weekly and daily temporal patterns, enhances the model's ability to capture repetitive trends in traffic data, and preserves the location information of the time series;

[0019] Time series embedding, using the spatiotemporal adaptive embedding method, denoted as where d a is the dimension of spatiotemporal adaptive embedding to capture complex spatiotemporal relationships;

[0020] By concatenating the above embeddings, we get the hidden spatiotemporal representation The hidden dimension d h =3d f +d a , which is the total dimension after splicing.

[0021] Furthermore, the GAT-Mamba module in S3 integrates the graph attention network and the Mamba model to enhance the model's ability to process graph-structured traffic data and capture complex relationships in spatial and temporal dimensions;

[0022] The integrated graph attention network is GAT;

[0023] The GAT-Mamba module not only utilizes flow as the unique feature of each node, but also takes periodic data, i.e., time and day, as multiple features;

[0024] The GAT-Mamba module is divided into a temporal GAT-Mamba block and a spatial GAT-Mamba block. The temporal GAT-Mamba block effectively processes node features, captures the temporal dynamics of traffic flow, and makes full use of long-term dependencies. The spatial GAT-Mamba block further enhances the model's understanding and representation of traffic relations by analyzing data in the spatial dimension.

[0025] Furthermore, the regression layer in S3 maps the features processed by the GAT-Mamba module to the target space, thereby achieving accurate prediction of future traffic flow.

[0026] Furthermore, the S3 at least includes the following steps:

[0027] Given a hidden spatiotemporal node feature representation T is the number of time steps, N is the number of nodes, d h is the characteristic dimension;

[0028] At each time step t∈{1,...,T}, the temporal GAT-Mamba layer uses GAT with four attention heads to capture the dynamic interactions between nodes, i.e., at each time step tGAT processes node features See the following formula:

[0029]

[0030] Where G(V,E) represents the traffic network graph, which contains nodes V and edges reflects the underlying road network structure necessary to capture spatial relationships;

[0031] Collecting the outputs of all time steps, we get the following formula:

[0032]

[0033] To stabilize the learning process and promote convergence, we add the original input Z to the output Z out A residual connection is added to the sum and layer normalization is applied to produce the final output, see the following formula:

[0034]

[0035] Residual connections help alleviate issues such as vanishing gradients, while layer normalization ensures that feature distribution remains consistent across layers;

[0036] The temporal GAT-Mamba block uses the Mamba model along the time axis to capture temporal interactions after performing GAT, as shown below:

[0037]

[0038] The residual connection is realized, see the following formula:

[0039]

[0040] Where Z Mamba It represents the final output after applying the Mamba model and residual connection and then layer normalization;

[0041] By stacking three GAT-Mamba modules, a temporal GAT-Mamba layer is constructed, which enhances the model's ability to capture complex temporal dependencies;

[0042] By applying the Mamba model along the spatial dimension within the GAT-Mamba module, the spatial GAT-Mamba block is obtained, and then the spatial GAT-Mamba layer is constructed;

[0043] To generate the final prediction, the output of the spatiotemporal GAT-Mamba layer is Reshape by flattening the time and feature dimensions of each node, resulting in

[0044] The regression layer then calculates the prediction value as:

[0045]

[0046] in, is the predicted output of each node on the prediction range T′, represents the learnable weight, b o ∈R T′ is the deviation vector.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. The present invention improves the feature expression ability of the model through multi-dimensional feature learning (time, date, time and space) of the embedding layer, which can deeply capture the rich features of the input data and enhance the learning ability of the model.

[0049] 2. The present invention uses a graph attention module to adaptively focus on the relative importance of nodes, so it can more effectively capture and utilize the graph structure relationship in the data and improve the accuracy of the model's analysis of spatial data.

[0050] 3. The GAT-Mamba module of the present invention optimizes the time series analysis and spatial data processing capabilities. Through the multi-layer graph attention mechanism and residual connection, the model can more accurately capture the dynamic dependencies between time points and spatial locations.

[0051] 4. The GAT-Mamba layer structure of the present invention supports flexible adjustment. Users can adjust the depth of the model according to specific task requirements to adapt to spatiotemporal data prediction tasks of different complexities, and has good adaptability.

[0052] 5. The present invention excels in computational efficiency. It provides an optimized computational method through the integration of Mamba, significantly reduces computational complexity, and makes the model more suitable for real-time prediction. It can also dynamically adapt to changes in the traffic network and improve prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0054] Figure 1 This is a schematic diagram of the GAT-Mamba model architecture;

[0055] Figure 2 This is a schematic diagram of GAT Attention;

[0056] Figure 3 Schematic diagram of GAT Multi-Head Attention;

[0057] Figure 4 This is a schematic diagram of the METR-LA dataset;

[0058] Figure 5 Schematic diagram of the PEMS-BAY dataset. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0060] See also Figure 1-Figure 3, a lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba, comprising at least the following steps:

[0061] S1: Build a spatiotemporal traffic prediction model. The spatiotemporal traffic prediction model is used to improve the accuracy of traffic prediction at multiple time scales. The spatiotemporal traffic prediction model includes an embedding layer, a GAT-Mamba module, and a regression layer.

[0062] S2: Use the embedding layer to represent the original features of historical traffic data in high dimensions and generate feature representations suitable for model input;

[0063] S3: The GAT-Mamba module and regression layer perform spatiotemporal dependency analysis on traffic data. The generated features are processed by the regression layer module to generate the final prediction output for each node in the prediction range.

[0064] The embedding layer generates a high-dimensional spatial representation by integrating feature embedding, periodic embedding, and spatiotemporal adaptive embedding.

[0065] S2 includes at least the following steps:

[0066] Feature embedding uses a fully connected layer to calculate the embedding representation of each node data, that is, This fully connected layer is applied independently to each node at each time step, see the following formula:

[0067]

[0068] in: is the embedding representation of node i at time step t, which is a high-dimensional vector representing the characteristics of the node; It represents the original input characteristics of node i at time t; is the traffic characteristic of node i at time t; time t and day t These three original features are combined into a 3D vector and input into the fully connected layer. and are the learnable weight matrix and bias vector, respectively, d f is the dimension of each node embedding, i.e., the representation size of the node in the embedding space;

[0069] Then obtain the high-dimensional representation of the original features from each node;

[0070] Periodic embedding uses a learnable embedding tensor to represent the weekday and timestamp day features, denoted as and Where N w and N dRepresents the number of working days and timestamps (i.e. 7 days, 24 hours), d f is the dimension of the embedding. At each time step, from T w and T d The corresponding weekday and timestamp embeddings are retrieved from the . Then the weekday and timestamp embeddings are concatenated and broadcasted to form a periodic embedding of a traffic time series of length T. Where N is the number of spatial nodes, and E is embedded periodically. p Effectively encodes weekly and daily temporal patterns, enhances the model's ability to capture repetitive trends in traffic data, and preserves the location information of the time series;

[0071] The temporal relationship in traffic time series is controlled not only by periodicity but also by temporal order. For example, a time window in a traffic time series is more similar to its adjacent time windows, while time series from different sensors usually show different temporal patterns. Time series embedding uses a spatiotemporal adaptive embedding method, denoted as where d a is the dimension of spatiotemporal adaptive embedding to capture complex spatiotemporal relationships;

[0072] By concatenating the above embeddings, we get the hidden spatiotemporal representation The hidden dimension d h =3d f +d a , which is the total dimension after splicing.

[0073] The GAT-Mamba module in S3 integrates the graph attention network and the Mamba model to enhance the model's ability to process graph-structured traffic data and capture complex relationships in spatial and temporal dimensions. The GAT-Mamba model is designed to capture the complex spatiotemporal dependencies inherent in traffic prediction tasks.

[0074] The integrated graph attention network is GAT;

[0075] The GAT-Mamba module not only utilizes flow as the unique feature of each node, but also takes periodic data, i.e., time and day, as multiple features;

[0076] The GAT-Mamba module is divided into a temporal GAT-Mamba block and a spatial GAT-Mamba block. The temporal GAT-Mamba block effectively processes node features, captures the temporal dynamics of traffic flow, and makes full use of long-term dependencies. The spatial GAT-Mamba block further enhances the model's understanding and representation of traffic relations by analyzing data in the spatial dimension.

[0077] The regression layer in S3 maps the features processed by the GAT-Mamba module to the target space, achieving accurate prediction of future traffic flow.

[0078] S3 includes at least the following steps:

[0079] Given a hidden spatiotemporal node feature representation T is the number of time steps, N is the number of nodes, d h is the characteristic dimension;

[0080] At each time step t∈{1,...,T}, the temporal GAT-Mamba layer uses GAT with four attention heads to capture the dynamic interactions between nodes, i.e., at each time step tGAT processes node features See the following formula:

[0081]

[0082] Where G(V,E) represents the traffic network graph, which contains nodes V and edges reflects the underlying road network structure necessary to capture spatial relationships;

[0083] Collecting the outputs of all time steps, we get the following formula:

[0084]

[0085] To stabilize the learning process and promote convergence, we add the original input Z to the output Z out A residual connection is added to the sum and layer normalization is applied to produce the final output, see the following formula:

[0086]

[0087] Residual connections help alleviate issues such as vanishing gradients, while layer normalization ensures that feature distribution remains consistent across layers;

[0088] The temporal GAT-Mamba block uses the Mamba model along the time axis to capture temporal interactions after performing GAT, as shown below:

[0089]

[0090] The residual connection is realized, see the following formula:

[0091]

[0092] Where Z Mamba It represents the final output after applying the Mamba model and residual connection and then layer normalization;

[0093] By stacking three GAT-Mamba modules, a temporal GAT-Mamba layer is constructed, which enhances the model's ability to capture complex temporal dependencies;

[0094] By applying the Mamba model along the spatial dimension within the GAT-Mamba module, the spatial GAT-Mamba block is obtained, and then the spatial GAT-Mamba layer is constructed;

[0095] To generate the final prediction, the output of the spatiotemporal GAT-Mamba layer is Reshape by flattening the time and feature dimensions of each node, resulting in

[0096] The regression layer then calculates the prediction value as:

[0097]

[0098] in, is the predicted output of each node on the prediction range T′, represents the learnable weight, b o ∈R T′ is the deviation vector.

[0099] The Graph Attention Network (GAT) module uses a self-attention mechanism to weigh the influence of nodes in traffic networks, allowing dynamic importance assignment based on the connectivity and features of nodes. This is particularly beneficial in traffic networks because not all connections contribute equally to traffic flow and congestion patterns.

[0100] The Graph Attention Network (GAT) was proposed by Velickovic et al. The attention mechanism introduced by GAT allows it to dynamically adjust the attention score according to the differences in node features. The specific process is to rely on the attention mechanism to calculate the weight of each point to its first-order neighbor node and perform a first-order convolution operation. In addition, the introduction of the attention mechanism allows it to be calculated in parallel, greatly improving the calculation speed.

[0101] Next, we take the target node v i For example, during the GAT calculation process, all nodes in the graph will pass through a fully connected layer to achieve the transformation of feature dimensions. After that, they will pass through the fully connected layer, nonlinear activation function and normalization operation in sequence to obtain the attention scores between nodes. In order to preserve the graph structure information, GAT's attention calculation is only performed between neighboring nodes. The specific calculation formula is as follows:

[0102]

[0103] where αij Represents node v i and its neighbor node v j The attention score between them, exp represents exponential operation, LeakyReLU is a nonlinear activation function, N(i) is the set of neighbor nodes, is the parameter of the attention mechanism, W∈R P×Q Indicates the parameter when the node feature is transformed, h i ∈R P Represents node v i P-dimensional features, || represents the concatenation operation. According to the above formula

[0104]

[0105] The central node v can be obtained i To its neighbor node v j The attention score is then multiplied by the corresponding neighbor node feature and summed up as the node v i The new embedding representation. The specific calculation formula is shown as follows:

[0106]

[0107] in Represents the l-th layer node v j The output of , σ is the activation function, and W is the parameter of feature transformation.

[0108] Attention mechanism used in the model of the present invention By weight vector Parameterize and apply LeakyReLU activation, such as Figure 2 . Figure 3 Illustration of multi-head attention (K=3 heads) of node 1 on its neighborhood. Different arrow styles and colors indicate independent attention computations. Concatenating or averaging the aggregated features of each head yields

[0109]

[0110] Through its efficient selective state update mechanism and feedback loop mechanism, Mamba can accurately capture the long-term temporal dynamics of traffic data, providing a new and effective method for traffic forecasting.

[0111] The Mamba model, by Gu and Dao, implements the Selective State Space Model (SSM) architecture, which selectively preserves state information in the time axis in sequence modeling, which is crucial for understanding time series data.

[0112] This model is designed to efficiently capture the dynamic changes of long time series data without the computational burden imposed by traditional recurrent units. In the context of traffic forecasting, the Mamba model plays a key role in handling the time dimension of data, capturing the changing patterns of traffic flow dynamics. Here is how it works:

[0113] Before starting the Mamba model, the state needs to be initialized and the initial state h needs to be set. (t-1) , which may be a zero vector or a specific method based on historical data. This state evolves as the model runs and reflects the current traffic conditions. The core of Mamba is the selective state update mechanism, which involves input gating, state transfer and state fusion. Input gating filters valid inputs through the Sigmoid activation function; state transfer efficiently updates the state through linear transformations (such as matrix multiplication); state fusion mixes new and old states to ensure a smooth transition. After updating the state, Mamba maps it to the output layer to predict future traffic flow, and adds nonlinear functions such as ReLU or Tanh to improve accuracy. In addition, the model contains a feedback loop mechanism that feeds predictions back to the system to adjust parameters, relies on the back propagation algorithm to correct errors, optimize performance, and make predictions more accurate and reliable.

[0114] In summary:

[0115] The present invention integrates a graph attention network (GAT) with a selective state space model (Mamba) to improve the accuracy of traffic prediction at multiple time scales, including an embedding layer, GAT-Mamba and a regression layer. The embedding layer mainly represents the original features of historical traffic data in high dimensions to generate a feature representation suitable for model input. The GAT module uses a self-attention mechanism to dynamically weigh the influence between nodes in the traffic network and flexibly capture spatial dependencies. The Mamba model can efficiently capture long-term temporal dynamics while avoiding the computational burden brought by traditional recurrent units. GAT-Mamba is divided into temporal and spatial GAT-Mamba modules, wherein the temporal GAT-Mamba module effectively processes node features, captures the temporal dynamics of traffic flow, and makes full use of long-term dependencies. The spatial GAT-Mamba module further enhances the model's understanding and representation of traffic relations by analyzing data in the spatial dimension. The regression layer maps the features processed by GAT-Mamba to the target space to achieve accurate prediction of future traffic flow. By combining graph attention networks with selective state space modeling, the present invention innovatively solves the shortcomings of traditional traffic prediction models in dealing with spatiotemporal dependencies, significantly improving the accuracy and efficiency of traffic flow prediction, especially in dynamic traffic modes.

[0116] Based on the above content, the following experiment is proposed for demonstration:

[0117] Experimental setup:

[0118] This case is implemented using the open source software library PyTorch (version 2.1.1) on a workstation equipped with a single NVIDIA GeForce RTX 4090 GPU, using the METRLA, PEMS-BAY, PEMS03, PEMS04, PEMS07, and PEMS08 benchmark traffic datasets. The METR-LA and PEMS-BAY datasets are divided into training, validation, and test sets in a ratio of 7:1:2. PEMS03, PEMS04, PEMS07, and PEMS08 are divided into training, validation, and test sets in a ratio of 6:2:2. The embedding dimension d f Set to 24, d a The number of spatial and temporal GAT-Mamba layers is 3. The number of attention heads in GAT is set to 4. The Adam optimizer is used with an initial learning rate of 0.001 and a batch size of 16. When the validation error converges in 30 consecutive training cycles, an early stopping mechanism is performed to reduce the training time.

[0119] Model evaluation indicators:

[0120] The present invention uses three widely used metrics in traffic prediction tasks: mean absolute error (MAE, Equation 11), root mean square error (RMSE, Equation 12), and mean absolute percentage error (MAPE, Equation 13).

[0121]

[0122]

[0123] Benchmark datasets Figure 4 and Figure 5 ;

[0124] To evaluate the performance of GAT-Mamba, we use several benchmark traffic datasets, including METR-LA, PEMS-BAY, PEMS03, PEMS04, PEMS07, and PEMS08. These datasets are commonly used for traffic forecasting and analysis, especially in the context of graph neural networks and time series modeling. Figure 4 and Figure 5 Visualization of the node distribution of the METR-LA and PEMS-BAY datasets.

[0125] Experimental results:

[0126] As shown in Tables 1 and 2, GAT-Mamba outperforms traditional models such as GWNet, DCRNN, and STGCN in short-term (15 minutes), medium-term (30 minutes), and long-term (60 minutes) prediction accuracy. The GAT-Mamba model achieves new breakthroughs in traffic forecasting and outperforms existing models on multiple time frames and datasets. Its innovative use of graph attention and selective state space provides more accurate, stable, and reliable predictions, making it a valuable tool for advanced traffic management and planning. The scalability and computational efficiency of the model further emphasize its applicability in real-world scenarios, which is expected to significantly improve the operation of transportation systems.

[0127] Table 1 Performance of each model on METR-LA and PEMS-BAY datasets

[0128]

[0129]

[0130] The results show that GAT-Mamba consistently outperforms existing state-of-the-art models in terms of forecast accuracy at different time horizons. Compared to the baseline model, GAT-Mamba achieves a 17.28% reduction in mean absolute error (MAE).

[0131] Table 2 Performance of each model on PEMS03, 04, 07 and 08

[0132]

[0133] According to the experiments, the best periods for METR-LA, PEMS-BAY, PEMS03, PEMS04, PEMS07 and PEMS08 are 21, 12, 27, 39, 37 and 47, respectively.

[0134] Ablation experiment:

[0135] In order to evaluate the importance of each component of the model, an ablation experiment was conducted on the GAT-Mamba model to evaluate the importance of each component by analyzing their contribution to the performance on the METR-LA and PEMS-BAY datasets, as shown in Table 4.

[0136] Table 3 Ablation studies on METR-LA and PEMS-BAY datasets

[0137]

[0138] GAT-Mamba w / o GAT: Removing the GAT component from GAT-Mamba leads to an increase in error metrics across all prediction horizons (MAE, RMSE, MAPE), highlighting the critical role of the graph attention mechanism in effectively modeling traffic data.

[0139] GAT-Mamba (w / o Temp) with Removal of Temporal Components: Omitting the temporal components slightly deteriorates the performance of the model, indicating their importance in capturing time-dependent traffic patterns.

[0140] Removing the spatial component leads to higher error metrics, highlighting their necessity to understand spatial relationships within the data.

[0141] GAT-Mamba w / o both spatial and temporal components: This configuration, lacking both spatial and temporal components, exhibits the most significant performance drop, validating the synergistic importance of these elements in the model architecture.

[0142] In summary, the fully integrated GAT-Mamba model demonstrates superior performance on all measured metrics and prediction horizons compared to the ablated variants, emphasizing that the spatial and temporal components of GAT and Mamba are indispensable for achieving high accuracy and robustness in traffic forecasting tasks.

[0143] Summary based on the above:

[0144] This paper proposes a spatiotemporal traffic prediction method based on graph attention network (GAT) and selective state space model (Mamba) by integrating graph attention network and selective state space model to improve the accuracy of traffic prediction under multiple time windows. Extensive evaluation of the model on multiple benchmark datasets including METR-LA and PEMS-BAY shows that it consistently outperforms existing state-of-the-art models.

[0145] Ablation studies further highlight the critical role of the spatial and temporal GAT-Mamba components in the model. Removing either component leads to a drop in performance, thus emphasizing their importance in capturing the complex spatiotemporal dependencies that characterize traffic data. These components are not merely additive but rather have a synergistic effect that significantly enhances the model’s predictive power.

[0146] Future work will focus on expanding the application scope of GAT-Mamba to other fields, such as weather forecasting and dynamic network analysis, where graph-based spatiotemporal data are prevalent. It is also planned to explore further improvements to the model architecture to improve its interpretability and adaptability to rapidly changing data patterns. It is hoped that this will help build smarter and more adaptive traffic management systems and pave the way for further research in the field of graph-based predictive analytics.

[0147] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba, characterized by: At least the following steps are included: S1: Building a spatiotemporal traffic prediction model, which is used to improve the accuracy of traffic prediction at multiple time scales, and includes an embedding layer, a GAT-Mamba module, and a regression layer; S2: Use the embedding layer to represent the original features of historical traffic data in high dimensions and generate feature representations suitable for model input; S3: The GAT-Mamba module and regression layer perform spatiotemporal dependency analysis on traffic data. The generated features are processed by the regression layer module to generate the final prediction output for each node in the prediction range.

2. The lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba according to claim 1 is characterized by: The embedding layer generates a high-dimensional spatial representation by integrating feature embedding, periodic embedding, and spatiotemporal adaptive embedding.

3. The lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba according to claim 2 is characterized by: The S2 at least comprises the following steps: Feature embedding uses a fully connected layer to calculate the embedding representation of each node data, that is, This fully connected layer is applied independently to each node at each time step, see the following formula: in: is the embedding representation of node i at time step t, which is a high-dimensional vector representing the characteristics of the node; It represents the original input characteristics of node i at time t; is the traffic characteristic of node i at time t; time t and day t These three original features are combined into a 3D vector and input into the fully connected layer. and are the learnable weight matrix and bias vector, respectively, d f is the dimension of each node embedding, i.e., the representation size of the node in the embedding space; Then obtain the high-dimensional representation of the original features from each node; Periodic embedding uses a learnable embedding tensor to represent the weekday and timestamp day features, denoted as and Where N w and N d Represents the number of working days and timestamps, d f is the dimension of the embedding, at each time step, from T w and T d The corresponding weekday and timestamp embeddings are retrieved from ; the weekday and timestamp embeddings are then concatenated and broadcasted to form a periodic embedding of the traffic time series of length T Where N is the number of spatial nodes, and E is embedded periodically. p Effectively encodes weekly and daily temporal patterns, enhances the model's ability to capture repetitive trends in traffic data, and preserves the location information of the time series; Time series embedding, using the spatiotemporal adaptive embedding method, denoted as where d a is the dimension of spatiotemporal adaptive embedding to capture complex spatiotemporal relationships; By concatenating the above embeddings, we get the hidden spatiotemporal representation The hidden dimension d h =3d f +d a , which is the total dimension after splicing.

4. The lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba according to claim 1 is characterized by: The GAT-Mamba module in S3 integrates the graph attention network and the Mamba model to enhance the model's ability to process graph-structured traffic data and capture complex relationships in spatial and temporal dimensions. The integrated graph attention network is GAT; The GAT-Mamba module not only utilizes flow as the unique feature of each node, but also takes periodic data, i.e., time and day, as multiple features; The GAT-Mamba module is divided into a temporal GAT-Mamba block and a spatial GAT-Mamba block. The temporal GAT-Mamba block effectively processes node features, captures the temporal dynamics of traffic flow, and makes full use of long-term dependencies. The spatial GAT-Mamba block further enhances the model's understanding and representation of traffic relations by analyzing data in the spatial dimension.

5. The lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba according to claim 4 is characterized by: The regression layer in S3 maps the features processed by the GAT-Mamba module to the target space, thereby achieving accurate prediction of future traffic flow.

6. The lightweight spatiotemporal traffic flow prediction method based on GAT and Mamba according to claim 5 is characterized by: The S3 at least includes the following steps: Given a hidden spatiotemporal node feature representation T is the number of time steps, N is the number of nodes, d h is the characteristic dimension; At each time step t∈{1,...,T}, the temporal GAT-Mamba layer uses GAT with four attention heads to capture the dynamic interactions between nodes, i.e., at each time step tGAT processes node features See the following formula: Where G(V,E) represents the traffic network graph, which contains nodes V and edges reflects the underlying road network structure necessary to capture spatial relationships; Collecting the outputs of all time steps, we get the following formula: To stabilize the learning process and promote convergence, we add the original input Z to the output Z out A residual connection is added to the sum and layer normalization is applied to produce the final output, see the following formula: Residual connections help alleviate issues such as vanishing gradients, while layer normalization ensures that feature distribution remains consistent across layers; The temporal GAT-Mamba block uses the Mamba model along the time axis to capture temporal interactions after performing GAT, as shown below: The residual connection is realized, see the following formula: Where Z Mamba It represents the final output after applying the Mamba model and residual connection and then layer normalization; By stacking three GAT-Mamba modules, a temporal GAT-Mamba layer is constructed, which enhances the model's ability to capture complex temporal dependencies; By applying the Mamba model along the spatial dimension within the GAT-Mamba module, the spatial GAT-Mamba block is obtained, and then the spatial GAT-Mamba layer is constructed; To generate the final prediction, the output of the spatiotemporal GAT-Mamba layer is Reshape by flattening the time and feature dimensions of each node, resulting in The regression layer then calculates the prediction value as: in, is the predicted output of each node on the prediction range T′, represents the learnable weight, b o ∈R T′ is the deviation vector.

Images