Traffic flow prediction method based on space-time mask hourglass Transform network

Abstract

The invention discloses a traffic flow prediction method based on a space-time mask hourglass Transform network, and the method comprises the following steps: inputting space-time traffic flow data into a long-term trend feature extractor with a space-time mask hourglass for space-time decoupling and feature extraction, and outputting a long-term trend implicit representation; inputting the implicit expression of the long-term trend into a periodic trend feature extractor, and outputting the implicit expression of the periodic trend; and inputting the implicit representation of the periodic trend into a short-term trend feature extractor, and outputting a traffic flow prediction result. The method can effectively improve the prediction precision of the model in long-time prediction and periodic traffic fluctuation scenes.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation, and more specifically, relates to a traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network. Background Technology

[0002] Traffic flow prediction is one of the core research directions of Intelligent Transportation Systems (ITS), playing a crucial role in traffic management and travel planning. By accurately predicting future traffic conditions, drivers can plan optimal routes in advance, thereby reducing travel time, alleviating traffic congestion, and improving overall network efficiency. Simultaneously, traffic management departments can also use the prediction results to dynamically allocate resources and optimize traffic signal control, which has significant forward-looking decision-making value, especially in highly congested areas.

[0003] Existing traffic flow prediction methods mainly include statistical methods, machine learning methods, and deep learning methods. Early studies mostly used statistical models such as historical average (HA), autoregressive integral moving average (ARIMA), and Kalman filtering. These methods can describe the overall trend of time series changes, but their ability to characterize nonlinear features and complex spatial dependencies is limited, making them difficult to adapt to dynamically changing traffic scenarios. Subsequently, machine learning methods such as hidden Markov models and spatiotemporal K-nearest neighbors (STKNN) were introduced into the field of traffic prediction. By introducing mechanisms such as road segment clustering and spatiotemporal weights, these methods have improved the adaptability of models to changes in traffic conditions to some extent. However, they still mainly rely on shallow feature representations and lack the ability to model complex spatiotemporal dependencies.

[0004] In recent years, deep learning methods have been widely applied to traffic flow prediction. Models such as Convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), Graph Convolutional Networks (GCNNs), and Transformers have effectively improved the ability to characterize nonlinear features and complex spatiotemporal dependencies. For example, a prediction framework combining CNNs and LSTMs can simultaneously extract spatial features and temporal dependencies; graph convolutional models can characterize the topological structure of road networks; and models based on attention mechanisms and Transformers further enhance the ability to interact with features across time periods and nodes, achieving good results in terms of prediction accuracy and robustness.

[0005] Despite extensive research, existing traffic flow prediction methods still have certain limitations. On the one hand, spatial and temporal features are often jointly modeled, resulting in insufficient decoupling of spatiotemporal features and making it difficult to independently capture complex dynamic relationships. On the other hand, most methods focus on short-term historical information and fail to fully exploit the long-term periodic patterns prevalent in traffic data, leading to performance degradation in cross-day or long-term prediction tasks. Furthermore, existing methods still have limited ability to model global spatial dependencies, often focusing on local neighborhood nodes and neglecting potential relationships between distant nodes, thus limiting their generalization ability in cross-regional scenarios.

[0006] A prior art patent with publication number CN114299723A proposes a traffic flow prediction method. The method includes: acquiring a training set containing periodic data and recent data, where the periodic data is historical traffic data with a weekly cycle, and the recent data is historical traffic data for a recent time period. The traffic data includes traffic flow, traffic speed, and road occupancy. A traffic flow prediction model is constructed, using the periodic data and recent data as input. A localized spatiotemporal graph is constructed to simulate the spatiotemporal dependencies in the data, and a spatiotemporal dynamic correlation in the data is simultaneously captured based on an attention mechanism to extract periodic features. Simultaneously, a pattern transfer component is used to calculate the similarity between the periodic features and recent data, and the periodic features are selectively transferred and integrated into the prediction target data. Finally, the collected current-time traffic data is input into the trained model to predict traffic flow for future time periods, thereby improving the accuracy of urban traffic flow prediction. However, this method mainly relies on the fusion modeling of historical periodic data and recent data. In expressing spatiotemporal dependencies, it only uses a localized spatiotemporal graph and an attention mechanism to extract the dynamic correlation between periodic and recent features, lacking multi-scale decoupling and global modeling of long-term spatiotemporal dependencies. Summary of the Invention

[0007] To overcome the problems of insufficient decoupling of spatiotemporal features and inadequate utilization of long-term periodic trends in existing traffic flow prediction methods, this invention provides a traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network.

[0008] The primary objective of this invention is to solve the aforementioned technical problems. The technical solution of this invention is as follows: This invention provides a traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network, comprising the following steps: Spatiotemporal traffic flow data is input into a long-term trend feature extractor with a spatiotemporal mask hourglass for spatiotemporal decoupling and feature extraction, and the output is an implicit representation of the long-term trend. Input the long-term trend implicit representation into the periodic trend feature extractor and output the periodic trend implicit representation. The implicit representation of the cyclical trend is input into the short-term trend feature extractor, and the traffic flow prediction results are output.

[0009] Furthermore, the long-term trend feature extractor includes an encoder and a decoder. Spatiotemporal traffic flow data is input into the long-term trend feature extractor, and an implicit representation of the long-term trend is output, including the following steps: X long Slice and convert into data blocks X of equal length with duration k. patched ; Transform data blocks of equal length X patched Input encoder, output implicit representation H encoder ; Implicit representation H encoder Input to decoder, output implicit representation H of long-term trend long .

[0010] Further, inputting time-equal length data blocks into the encoder and outputting an implicit representation includes the following steps: By mapping data blocks of equal duration to a high-dimensional embedding space, a vector representation is obtained. The vector representation is input into the two-dimensional position encoding module, and the position encoding is superimposed on the vector representation to obtain the vector representation with position encoding; The vector representation with position encoding is input into the temporal mask hourglass Transformer encoder and the spatial mask Transformer encoder respectively, and the temporal dimension feature representation and the spatial dimension feature representation are output respectively. The implicit representation is obtained by concatenating the time dimension feature representation and the spatial dimension feature representation.

[0011] Furthermore, the vector representation with positional encoding is input to the temporal mask hourglass Transformer encoder, and the output temporal dimension feature representation is generated, including the following steps: The time-step traffic flow data represented by the input vector is masked using a random masking strategy with a preset masking ratio to obtain masked data; The masked data is input into a Transformer encoding layer with an hourglass network structure, and the time dimension features are output. The expression is as follows:

[0012] in, (·) represents a masking operation in the time dimension. Indicates the location encoding in the time dimension. This represents the input vector representation. This represents a pre-trained Transformer encoder with an hourglass network structure.

[0013] Furthermore, the Transformer encoder with an hourglass network structure includes a first downsampling layer, a second downsampling layer, a bottleneck encoder, a first upsampling layer, and a second upsampling layer. The mask data is input into the Transformer encoder with the hourglass network structure, and the time-dimensional features are output, including the following steps: X after masking input Input the first downsampling layer and output the first downsampling feature skip1; Input skip1 into the second downsampling layer and output the second downsampling feature skip2; Input skip2 into the bottleneck encoder and output the first encoded feature X1; Input X1 and skip2 (which is connected to skip) into the first upsampling layer to output the second encoded feature X2. Input X2 and the skip1 connection into the second upsampling layer to output the time dimension features.

[0014] Furthermore, the vector representation with positional encoding is input to the spatial mask Transformer encoder, and the output is a spatial dimension feature representation, including the following steps: The spatial node traffic flow data represented by the input vector is masked using a random masking strategy with a preset masking ratio to obtain masked data. The masked data is input into a pre-trained Transformer encoder, which outputs spatial dimension features. The expression is as follows:

[0015] in, (·) represents a masking operation for spatial dimensions. Indicates spatial dimension location encoding, SMTE represents the input vector representation and represents the pre-trained transformer encoder.

[0016] Furthermore, the implicit representation is input into the decoder, and the output is an implicit representation of the long-term trend, including the following steps: Implicit representation H encoder The input linear layer performs spatial mapping, preserving the feature dimension, and outputs an implicit representation that fits the decoder. The expression is as follows:

[0017] in, and All of these represent learnable parameters; right Position encoding is performed; for the unmasked portion, it is implicitly represented. Adding this to the corresponding location information yields the temporal feature representation T of the unmasked portion. unmasked Spatial feature representation S unmasked For the mask portion, the learned mask vector is filled in at the mask positions, and the mask vector is added to the corresponding mask position information to obtain the reconstructed temporal feature representation T of the mask portion. masked and spatial feature representation S masked ; T respectively masked、 T unmasked and S unmasked S masked The two methods are combined to obtain a complete temporal and spatial representation with temporal and spatial order information. Input the obtained complete time and space representations into the Transformer decoder. Perform restoration and reconstruction to obtain the reconstructed feature representation. The expression is as follows:

[0018] in, and These represent the temporal and spatial characteristics of the unmasked portion, respectively. and These represent the temporal and spatial representations reconstructed from the mask portion, respectively; Reconstruct feature representation The input is a linear output layer, which outputs an implicit representation of the final, fully reconstructed long-term trend features. The expression is as follows:

[0019] in, and This represents the learnable parameters.

[0020] Furthermore, the implicit representation of the long-term trend is input into the periodic trend feature extractor, and the implicit representation of the periodic trend is output, including the following steps: Implicit representation of long-term trend characteristics Input the periodic trend extractor PTE, output periodic features and daily cycle characteristics The expression is as follows:

[0021] Implicit representation of long-term trend characteristics Extracting current time features and current spatial characteristics The expression is as follows:

[0022] in, Implicit representation of long-term trend characteristics Get the last slice block from the middle; Periodic characteristics Daily cycle characteristics Current time characteristics and current spatial characteristics Input feature fusion module, output comprehensive features; By inputting the comprehensive features into the multilayer perceptron module, an implicit representation of the periodic trend is obtained. The expression is as follows:

[0023] MLP stands for Multilayer Perceptron Module.

[0024] Furthermore, the periodic trend extractor is a dynamic graph convolution module.

[0025] Furthermore, the short-term trend feature extractor is a spatiotemporal prediction network structure based on graph convolution and wavelet mechanisms. The implicit representation of the periodic trend is input into the short-term trend feature extractor, and the traffic flow prediction result is output. The expression is as follows:

[0026] Where t represents the current time step, This indicates the time step for traffic flow forecasting; STE stands for Short-Term Trend Feature Extractor. Indicates current time characteristics and current spatial characteristics .

[0027] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention proposes a complete and effective traffic flow prediction method. Using long-term traffic flow data as input, it employs long-term trend feature extractors, periodic trend feature extractors, and short-term trend feature extractors to perform hierarchical modeling and collaborative fusion of long-term, periodic, and short-term trend features in the traffic flow data. This effectively improves the prediction accuracy of the method in long-term prediction and periodic traffic fluctuation scenarios. Simultaneously, by decoupling and extracting features from spatiotemporal traffic flow data, the dependencies between the time and spatial dimensions can be more fully modeled, which is beneficial for capturing the complex dynamic correlations of traffic flow evolution over time and improving the overall traffic flow prediction performance. Attached Figure Description

[0028] To make the objectives and technical solutions of this invention clearer, the following drawings are provided and described: Figure 1 A flowchart of a traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network provided in an embodiment of the present invention; Figure 2 A schematic diagram of a traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a Transformer encoder with an hourglass network structure provided in an embodiment of the present invention. Detailed Implementation

[0029] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0031] Example 1: This invention provides a traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network, such as... Figure 1 The diagram shows a flowchart of a traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network. Figure 2 The diagram shows a model of a traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network provided by the present invention. The specific steps are as follows: S1: Input the spatiotemporal traffic flow data into the long-term trend feature extractor with a spatiotemporal mask hourglass for spatiotemporal decoupling and feature extraction, and output the implicit representation of the long-term trend.

[0032] like Figure 2 As shown in (a), the model uses the original spatiotemporal traffic flow data as input, and the original spatiotemporal traffic flow data X over two weeks is used as input. long The data is converted into a data block X of equal duration by slicing it into a slicing operation with a slice size of 12. patched The implicit representation h of the long-term trend is obtained through the long-term trend feature extractor. long .

[0033] More specifically, the long-term trend feature extractor is a Spatiotemporal Mask Hourglass Transformer (STMHT) module, such as... Figure 2 As shown, it includes an encoder and a decoder, where the encoder will convert X... patched Encoding yields the latent implicit representation H of the features. encoder The decoder obtains the implicit representation H of the final long-term trend. long This prepares for the next step of extracting cyclical trend features.

[0034] The spatiotemporal traffic flow data is input into the long-term trend feature extractor, which outputs an implicit representation of the long-term trend, including the following steps: S1.1: Transfer the spatiotemporal traffic flow data X long Slice and convert into data blocks X of equal length with duration k (1 hour in this example). patched .

[0035] S1.2: Divide the data blocks X of equal duration into equal lengths. patched Input encoder, output implicit representation H encoder .

[0036] In the encoder stage, the long spatiotemporal traffic flow input data is first segmented into fixed-length segments using a slice encoding embedding operation, and these segments are mapped to a high-dimensional embedding space for subsequent processing. Next, a two-dimensional position encoding module overlays position codes onto the mapped spatiotemporal traffic flow data, allowing the subsequent model to obtain the position of each input in both the temporal and spatial dimensions. Subsequently, different mask pre-training architectures are set up in the temporal and spatial dimensions to obtain feature encodings in both dimensions.

[0037] More specifically, inputting time-equal-length data blocks into the encoder and outputting an implicit representation includes the following steps: S1.2.1: Map data blocks of equal duration to a high-dimensional embedding space to obtain a vector representation.

[0038] S1.2.2: Input the vector representation into the two-dimensional position encoding module, and superimpose the position encoding on the vector representation to obtain the vector representation with position encoding.

[0039] S1.2.3: Input the vector representation with position encoding into the temporal mask hourglass Transformer encoder and the spatial mask Transformer encoder respectively, and output the temporal dimension feature representation and the spatial dimension feature representation.

[0040] In the time dimension, the proposed mask pre-training architecture is the Temporal Masked Hourglass Transformer Encoder (TMHTE), which uses a random masking strategy with a preset masking ratio to mask traffic flow data at some time steps. The masked data is then encoded with time dimension features through a Transformer encoding layer with an hourglass network structure.

[0041] More specifically, the input temporal mask hourglass Transformer encoder is represented by a vector with positional encoding, and the output is a temporal dimension feature representation, including the following steps: S1.2.3.1: Use a random masking strategy with a preset masking ratio (25% in this embodiment) to mask the time-step traffic flow data represented by the input vector (traffic flow data of some time steps) to obtain masked data; S1.2.3.2: Input the masked data into the Transformer encoding layer with an hourglass network structure, and output the time dimension features. The expression is as follows:

[0042] in, (·) represents a masking operation in the time dimension. Indicates the location encoding in the time dimension. This represents the input vector representation. This represents a pre-trained Transformer encoder with an hourglass network structure, consisting of two parts: downsampling and upsampling. Figure 3 As shown, in the downsampling stage, features are extracted from long sequence input data through two layers of Transformer encoders and the sequence length is downsampled step by step; then the features are reduced in dimensionality in the bottleneck layer to model global dependencies in the low-dimensional space, and then the dimensionality is increased back to the original feature dimension; in the upsampling stage, the sequence length is gradually restored and fused with the intermediate features from the downsampling stage, and then restored to the original sequence shape by the decoder, thereby realizing multi-scale feature aggregation and global modeling.

[0043] More specifically, the Transformer encoder with an hourglass network structure includes a first downsampling layer, a second downsampling layer, a bottleneck encoder, a first upsampling layer, and a second upsampling layer. The masked data is input into the Transformer encoder with the hourglass network structure, and the output is temporal dimension features, including the following steps: S1.2.3.2.1.1: Mask the data X input Input the first downsampling layer and output the first downsampling feature skip1.

[0044] S1.2.3.2.1.2: Input skip1 into the second downsampling layer and output the second downsampling feature skip2.

[0045] S1.2.3.2.1.3: Input skip2 into the bottleneck encoder and output the first encoded feature X1.

[0046] S1.2.3.2.1.4: Input X1 and skip2 (a skip connection) into the first upsampling layer and output the second encoded feature X2.

[0047] S1.2.3.2.1.5: Input X2 and the skip1 connection into the second upsampling layer to output the time dimension features.

[0048] The proposed spatial mask Transformer encoder uses position-encoded vector representations as input and outputs spatial dimension feature representations. In the spatial dimension, the pre-trained mask architecture is a Spatial Mask Transformer Encoder (SMTE). This includes the following steps: S1.2.3.2.2.1: Use a random masking strategy with a preset masking ratio (25% in this embodiment) to mask the spatial node traffic flow data represented by the input vector to obtain masked data; S1.2.3.2.2.2: Input the masked data into the pre-trained Transformer encoder, and output the spatial dimension features. The expression is as follows:

[0049] in, (·) represents a masking operation for spatial dimensions. Indicates spatial dimension location encoding, SMTE represents the input vector representation and represents the pre-trained standard transformer encoder.

[0050] S1.2.4: Concatenate the temporal and spatial feature representations to obtain the implicit representation of the unmasked spatiotemporal dimension encoding. The expression is as follows:

[0051] S1.3: Implicitly represent H encoder Input to decoder, output implicit representation H of long-term trend long .

[0052] In the decoder stage, the implicit representation of the unmasked spatiotemporal dimensions obtained in the encoder stage is remapped to the space required by the decoder through a linear layer, keeping the feature dimensions unchanged, but allowing a lightweight transformation to be learned.

[0053] More specifically, the implicit representation is input into the decoder, and the output is an implicit representation of the long-term trend, including the following steps: The implicit representation H of the spatiotemporal dimension of the unmasked portion obtained during the encoder stage. encoder The input linear layer is remapped to the space required by the decoder, keeping the feature dimension unchanged, and the output is an implicit representation of the spatiotemporal dimension of the unmasked portion that fits the decoder space. The expression is as follows:

[0054] in, and All of these represent learnable parameters.

[0055] right Position encoding is performed; for the unmasked portion, it is implicitly represented. Adding this to the corresponding location information yields the temporal feature representation T of the unmasked portion. unmasked Spatial feature representation S unmasked For the mask portion, the learned mask vector is filled in at the mask positions, and the mask vector is added to the corresponding mask position information to obtain the reconstructed temporal feature representation T of the mask portion. masked and spatial feature representation S masked ; T respectively masked、 T unmasked and S unmasked S masked The two methods are combined to obtain a complete temporal and spatial representation with temporal and spatial order information. Input the obtained complete time and space representations into the Transformer decoder. By performing restoration and reconstruction, a complete feature representation with full spatiotemporal information is obtained. The expression is as follows:

[0056] in, and These represent the temporal and spatial characteristics of the unmasked portion, respectively. and These represent the temporal and spatial representations reconstructed from the mask portion, respectively; Reconstruct feature representation The input is a linear output layer, which outputs an implicit representation of the final, fully reconstructed long-term trend features. The expression is as follows:

[0057] in, and This represents the learnable parameters.

[0058] S2: Input the long-term trend implicit representation into the periodic trend feature extractor and output the implicit representation of the periodic trend.

[0059] like Figure 2 As shown in (b), this step aims to extract periodic trend features from long-term traffic flow data to enhance the model's ability to perceive long-term regular changes. Traffic flow data typically exhibits significant periodic characteristics; for example, traffic flow patterns on the same workdays each week show similar regularities, and different dates often show similar travel characteristics during the same time period (e.g., morning rush hour 8:00–9:00). Therefore, capturing such periodic changes is crucial for improving the accuracy of future traffic flow predictions. In this step, weekly and daily periodic features are first extracted from the implicit representation of long-term trend features obtained in step 1 using the Periodic Trend Extractor (PTE). Simultaneously, temporal and spatial features are extracted from the traffic characteristics of the current time period to reflect the current road conditions and their positional relationship within the spatial network. Subsequently, the obtained periodic features and the current spatiotemporal features are combined through a feature fusion module to form a comprehensive feature representation, providing input for subsequent prediction modules.

[0060] The specific process is as follows: Implicit representation of long-term trend characteristics Input the periodic trend extractor PTE, output periodic features and daily cycle characteristics The expression is as follows:

[0061] Among them, the periodic characteristics Used to reflect long-term patterns across weeks, such as traffic fluctuation trends on the same workday over several consecutive weeks; daily cycle characteristics. Used to depict traffic patterns at different times of the same day, such as the recurring changes during morning and evening rush hours.

[0062] In this embodiment, the periodic trend extractor PTE is a standard Dynamic Graph Convolution (DynamicGraphConv) module.

[0063] Implicit representation of long-term trend characteristics Extracting current time features and current spatial characteristics This is used to reflect the current road status and its positional relationship in the spatial network, and its expression is as follows:

[0064] in, Implicit representation of long-term trend characteristics Get the last slice block from the middle; Periodic characteristics Daily cycle characteristics Current time characteristics and current spatial characteristics Input feature fusion module, output comprehensive features; By inputting the comprehensive features into the multilayer perceptron module, an implicit representation of the periodic trend is obtained. The expression is as follows:

[0065] MLP stands for Multilayer Perceptron Module.

[0066] S3: Input the implicit representation of the cyclical trend into the short-term trend feature extractor and output the traffic flow prediction results.

[0067] like Figure 2 As shown in (c), with the implicit temporal representation already obtained... This includes cyclical trend characteristics derived from long-term trend features. and and the latest traffic flow characteristics at the current moment. and To further improve the accuracy of predicting short-term traffic flow changes, this step introduces a Short-Term Trend Feature Extraction (STE) module into the model. This module is mainly used to capture sudden or localized traffic flow fluctuations in the near term, thereby enhancing the model's responsiveness to real-time changes.

[0068] More specifically, the short-term trend feature extractor is a spatiotemporal prediction network structure based on graph convolution and wavelet mechanisms. The implicit representation of the periodic trend is input into the short-term trend feature extractor, and the traffic flow prediction result is output. The expression is as follows:

[0069] Where t represents the current time step, This indicates the time step for traffic flow forecasting; STE stands for Short-Term Trend Feature Extractor. Indicates current time characteristics and current spatial characteristics .

[0070] To verify the effectiveness of this invention, the predictive performance of the model was evaluated on four classic traffic flow prediction datasets: PEMS04, PEMS08, PEMS-BAY, and METR-LA. Three commonly used error metrics were employed: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE). Lower values ​​for these three metrics indicate better model performance. The STMHTNet model was compared with seven representative spatiotemporal prediction models, covering two mainstream paradigms: graph-based methods and Transformer-based methods. Graph-based models (DAGN, Graph WaveNet, STFGNN, DDSTGCN) utilize graph neural networks to capture spatiotemporal correlations, modeling spatial dependencies through graph structures and learning temporal dynamics through recurrent or convolutional operations. Transformer-based models (Trafformer, LLGformer, DSTET) employ a self-attention mechanism to adaptively capture long-range dependencies in both spatial and temporal dimensions. Experimental results are shown below: Table 1

[0071] Table 2

[0072] As shown in Tables 1 and 2, this patent demonstrates excellent overall performance on four classic traffic flow prediction datasets: PEMS04, PEMS08, PEMS-BAY, and METR-LA. Compared to previous traffic flow prediction models, this patent achieves the best results on the PEMS04, PEMS08, and METR-LA datasets, and is second only to the other two transformer-based models on the PEMS-BAY dataset. Considering all four datasets, this patent exhibits the best overall performance.

[0073] In summary, this invention addresses the problems of insufficient spatiotemporal dependency characterization, inadequate utilization of long-term and periodic information, and limited global spatial correlation modeling capabilities in traffic flow prediction. It proposes a traffic flow prediction scheme based on spatiotemporal decoupling and multi-range feature extraction. On one hand, by introducing a long-term trend feature extractor, employing a temporal mask hourglass Transformer structure in the time dimension and a spatial mask Transformer structure in the spatial dimension, it extracts temporal and spatial dependencies respectively, thereby independently characterizing the complex dynamic correlations in traffic flow. On the other hand, it constructs a multi-range feature extraction module that uses long-term traffic data as input to extract long-term trend features, periodic trend features, and short-term trend features, effectively compensating for the insufficient modeling of long-term patterns when relying solely on short-term historical input. Simultaneously, the STMHTNet model of this invention, based on a Transformer encoder-decoder architecture, can fully utilize the global spatial dependencies in the traffic network to model the interactions between distant nodes. Experimental results show that this invention outperforms existing graph-based prediction models in terms of traffic flow prediction accuracy and robustness, demonstrating significant substantive features and substantial progress.

[0074] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network, characterized in that, Includes the following steps: Spatiotemporal traffic flow data is input into a long-term trend feature extractor with a spatiotemporal mask hourglass for spatiotemporal decoupling and feature extraction, and the output is an implicit representation of the long-term trend. Input the long-term trend implicit representation into the periodic trend feature extractor and output the periodic trend implicit representation. The implicit representation of the cyclical trend is input into the short-term trend feature extractor, and the traffic flow prediction results are output.

2. The traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network according to claim 1, characterized in that, The long-term trend feature extractor includes an encoder and a decoder. Spatiotemporal traffic flow data is input into the long-term trend feature extractor, which outputs an implicit representation of the long-term trend. This includes the following steps: spatiotemporal traffic flow data X long slicing and converting into time-length equilong data blocks X patched ; X patched input encoder, output implicit representation H encoder ; H encoder input decoder, outputting an implicit representation H of long-term trends long .

3. The traffic flow prediction method based on the space-time mask hourglass Transformer network according to claim 2, characterized in that, Inputting data blocks of equal duration into the encoder and outputting an implicit representation involves the following steps: By mapping data blocks of equal duration to a high-dimensional embedding space, a vector representation is obtained. The vector representation is input into the two-dimensional position encoding module, and the position encoding is superimposed on the vector representation to obtain the vector representation with position encoding; The vector representation with position encoding is input into the temporal mask hourglass Transformer encoder and the spatial mask Transformer encoder respectively, and the temporal dimension feature representation and the spatial dimension feature representation are output respectively. The implicit representation is obtained by concatenating the time dimension feature representation and the spatial dimension feature representation.

4. The traffic flow prediction method based on the space-time mask hourglass Transformer network according to claim 3, characterized in that, The input temporal mask hourglass Transformer encoder is represented by a vector with positional encoding, and the output is a temporal dimension feature representation. This includes the following steps: The time-step traffic flow data represented by the input vector is masked using a random masking strategy with a preset masking ratio to obtain masked data; The mask data is input into a Transformer encoding layer with a sandglass network structure, and time dimension features are output The expression is as follows: in, (·) represents a masking operation in the time dimension. Indicates the location encoding in the time dimension. This represents the input vector representation. This represents a pre-trained Transformer encoder with an hourglass network structure.

5. The traffic flow prediction method of claim 4, wherein, The Transformer encoder with an hourglass network structure includes a first downsampling layer, a second downsampling layer, a bottleneck encoder, a first upsampling layer, and a second upsampling layer. Mask data is input into the Transformer encoder with the hourglass network structure, and the output is time-dimensional features, including the following steps: masked data X input input a first down-sampling layer, and output first down-sampling features skip1; Input skip1 into the second downsampling layer and output the second downsampling feature skip2; Input skip2 into the bottleneck encoder and output the first encoded feature X1; Input X1 and skip2 (which is connected to skip) into the first upsampling layer to output the second encoded feature X2. Input X2 and the skip1 connection into the second upsampling layer to output the time dimension features.

6. The traffic flow prediction method based on a spatiotemporal mask hourglass Transformer network according to claim 3, characterized in that, The Transformer encoder, which uses position-encoded vectors as input to a spatial mask, outputs a spatial dimension feature representation, including the following steps: The spatial node traffic flow data represented by the input vector is masked using a random masking strategy with a preset masking ratio to obtain masked data. inputting the mask data into a pre-trained Transformer encoder to output spatial dimensional features , as shown in the following expression: in, (·) represents a masking operation for spatial dimensions. Indicates spatial dimension location encoding, SMTE represents the input vector representation and represents the pre-trained transformer encoder.

7. The traffic flow prediction method based on the space-time mask hourglass Transformer network according to claim 2, characterized in that, The implicit representation is input into the decoder, and the output is an implicit representation of the long-term trend, including the following steps: The implicit representation H encoder The input linear layer performs spatial mapping, keeping the feature dimension unchanged, and outputs the implicit representation of the adaptive decoder The expression is as follows: wherein, and both represent learnable parameters; Position encoding processing is performed on the input sequence X , and for the unmasked part, an implicit representation is added to the corresponding position information to obtain the time feature representation T unmasked and the space feature representation S unmasked of the unmasked part; for the masked part, the learned mask vector is filled in the mask position, and the mask vector is added to the corresponding mask position information to obtain the time feature representation T masked and the space feature representation S masked reconstructed by the mask part. T masked、 T unmasked and S unmasked , S masked are merged to obtain a complete temporal and spatial representation with temporal and spatial order information; inputting the complete time and space representation into a transformer decoder performing a denoising reconstruction to obtain a reconstructed feature representation , as follows: wherein, and respectively represent the time and spatial feature representation of the unmasked part, and respectively represent the reconstructed time and spatial representation of the masked part; reconstructing the feature representation input linear output layer, outputting an implicit representation of the final complete reconstructed long-term trend feature , as shown in the following expression: wherein, and denote learnable parameters.

8. The traffic flow prediction method based on the space-time mask hourglass Transformer network according to claim 1, characterized in that, The implicit representation of the long-term trend is input into the periodic trend feature extractor, and the implicit representation of the periodic trend is output. This includes the following steps: implicit representation of long-term trend features input periodic trend extractor PTE, output weekly periodic features and daily periodic features , expressed as follows: extracting current time features from long-term trend characteristics extracting current spatial features and current spatial features , as follows: wherein, represents an implicit representation from long-term trend features the last slice chunk is obtained from the middle a weekly periodicity feature , a daily periodicity feature , a current time feature , and a current space feature to the feature fusion module, and output a comprehensive feature; The comprehensive features are input into a multi-layer perception module to obtain an implicit representation of the periodic trend The expression is as follows: MLP stands for Multilayer Perceptron Module.

9. The traffic flow prediction method based on the space-time mask hourglass Transformer network according to claim 8, characterized in that, The periodic trend extractor is a dynamic graph convolution module.

10. The traffic flow prediction method based on the space-time mask hourglass Transformer network according to claim 1, characterized in that, The short-term trend feature extractor is a space-time prediction network structure based on graph convolution and wavelet mechanism, and the implicit representation of the periodic trend is input into the short-term trend feature extractor, and a traffic flow prediction result is output , and the expression is as follows: where t denotes the current time step, denotes the time step of the traffic flow prediction, STE denotes the short-term trend feature extractor, denotes the current time feature and the current spatial feature .

Images