A traffic flow prediction method based on spatio-temporal decoupling mask self-encoding pre-training

Abstract

The application discloses a traffic flow prediction method based on space-time decoupling mask self-encoding pre-training, reconstructs a time encoder and a space encoder based on an LSTM unit and an STAtt module, introduces feature fusion and a downstream prediction model by a gating screening unit and an adaptive adjacency matrix to improve long-range information utilization, and obtains a traffic flow prediction result. Static and dynamic space structures and multi-scale time features are fused to improve global and local space-time dependence description capability, and high-precision and high-generalization traffic flow prediction is realized.

Description

Technical Field

[0001] This invention relates to the field of traffic flow spatiotemporal data modeling and prediction technology, specifically to a traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training. Background Technology

[0002] Traffic flow prediction is a crucial part of the deployment of intelligent transportation systems. Traffic data typically exhibits a time-series structure, and early research often employed statistical analysis methods, such as the seasonal autoregressive integral moving average model. While these methods have been successfully applied to traffic flow prediction to some extent, their predictive performance is inherently limited due to the difficulty in effectively extracting the complex features inherent in the traffic data. Traditional machine learning methods have been introduced into the field of traffic flow prediction, but they heavily rely on complex mathematical modeling and prior knowledge to obtain features.

[0003] In existing technologies, spatial modeling methods based on graph neural networks have become the mainstream approach for traffic flow prediction. By capturing complex spatiotemporal features and analyzing coupling relationships, they aim to predict traffic flow more accurately. However, traffic flow prediction models based on conventional spatiotemporal encoders focus more on static spatial dependencies or only model dynamic spatial correlations, lacking an effective fusion of the two. This makes it difficult to comprehensively depict the complex changes in traffic flow across different time scales and spatial ranges, thus limiting the model's predictive performance and generalization ability to some extent. While long-range information can further improve model predictive performance, it also raises the barrier to deployment. Therefore, there is an urgent need for a traffic flow prediction method that can further decouple spatiotemporal features, combine dynamic and static spaces, and efficiently apply long-range information, providing a new feasible solution for intelligent transportation deployment. Summary of the Invention

[0004] Purpose of the invention: To address the problems mentioned in the background art, this invention discloses a traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training. It replaces DWT with LSTM for decoupling, designs and introduces STATt module to construct a spatiotemporal encoder, and improves the utilization of long-range information by introducing feature fusion and downstream prediction model through gating filtering unit and adaptive adjacency matrix construction to obtain traffic flow prediction results.

[0005] Technical solution:

[0006] This invention discloses a traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training, the method comprising the following steps:

[0007] S1 acquires and preprocesses spatiotemporal sequence data of traffic flow based on multi-source sensors;

[0008] S2 Constructs a time mask autoencoder, using LSTM units to replace the original DWT for decoupling, and the time mask autoencoder obtains time feature representations;

[0009] S3 designs and introduces an improved spatiotemporal graph convolution STATt module to construct a spatial mask autoencoder, which acquires spatial feature representations;

[0010] S4 introduces a gating and filtering unit to fuse spatiotemporal features. These spatiotemporal features are input into the downstream prediction model and the prediction results are output.

[0011] Furthermore, S1 preprocessing includes taking highway traffic flow as the research object, setting statistical intervals, recording traffic flow data into a database, and filtering out redundant, outlier, and missing values ​​in the collected data to obtain traffic flow data.

[0012] Furthermore, the time mask autoencoder structure described in S2 is as follows:

[0013] The temporal mask autoencoder uses LSTM units instead of the original DWT to decouple and learn temporal features, and uses PatchEmbedding to divide the preprocessed long-flow sequence into segments. For non-overlapping patches of a given length, calculate the position code of the specified patch. ;

[0014] The masking strategy follows Bernoulli random sampling, randomly masking the sequence information at time step T. The input to the LSTM layer can be obtained after applying a mask. ,in For batch size, For time step, For the number of nodes, As the feature dimension, the LSTM layer progressively reads the input sequence and outputs a hidden temporal feature representation of the same shape. Used for subsequent downstream feature fusion prediction; the LSTM unit adopts a basic Cell structure.

[0015] Furthermore, the spatial mask autoencoder Patch Embedding and position encoding method described in S3 are the same as those of the temporal mask autoencoder. The improved spatiotemporal graph convolution STATt module consists of a temporal attention layer, a spatial attention layer, and a GCN layer. The temporal attention layer captures temporal dependencies and ensures causality, and its output is then processed by the GCN layer to extract topological information. The spatial attention layer is responsible for modeling the dynamic interactions between road network nodes.

[0016] Furthermore, the improved spatiotemporal graph convolution STATt module's operational structure is as follows:

[0017] Input preprocessed flow sequence tensor spatiotemporal embedded information The location and time information are embedded into the STE and added element-wise to the original features to inject spatiotemporal information. Multi-head causal self-attention is then applied to the length-T sequence of each node to obtain the temporal attention result. ,Will Flattened by time dimension Graph convolutional networks combined with graph structure support matrices { Extract topological information and restore the time-series graph convolution result. : Representing the same intermediate At each time step, multi-head spatial self-attention is performed on N nodes to obtain... Introducing a gated fusion unit to process the convolutional results of the time series graph Results of Spatial Attention The spatiotemporal fusion result is obtained by weighting and fusing the learning scores. The hidden space state output is obtained through residual connections. .

[0018] Furthermore, the temporal attention layer operation structure of the improved spatiotemporal graph convolution STATt module includes:

[0019] Input Element-wise addition with the optional spatiotemporal embedding STE(X) to inject position and time information; respectively through linear mapping layers , , The D-dimensional features are projected onto the query Q, key K, and value V space. The projection result is split into K heads along the node dimension, each subspace having dimension d, to support parallel multi-head computation. For each head, a scaled dot product attention score is calculated: a lower triangular causal mask is applied to ensure that only information not exceeding t can be accessed at time t. Softmax is then performed on the masked score to obtain attention weights, which are multiplied by the corresponding value vector v to generate a weighted output. Finally, the outputs of all heads are concatenated back to dimension D in their original order and passed through an output mapping layer. By fusing information from multiple sources, the final [B, T, N, D]-dimensional feature representation is obtained.

[0020] Furthermore, the spatial attention layer operation structure of the improved spatiotemporal graph convolution STATt module includes:

[0021] Input Element-wise addition with the optional spatiotemporal embedding STE(X) through a linear mapping layer , , The D-dimensional features are projected onto the query Q, key K, and value V space. The projection result is split into K heads along the node dimension, each subspace with dimension d, and reshaped into shape [B×T×K, N, d]. For each attention head, a scaled dot product attention score is calculated, and the score is normalized along the node dimension using Softmax to obtain the attention weight, which is then multiplied with the corresponding value vector 𝑉 to generate a weighted output. The outputs of all heads are concatenated in their original order and reshaped back to [B, T, N, D], and then passed through an output mapping layer. The STATt module output is obtained.

[0022] Furthermore, the spatiotemporal feature fusion process described in S4 is as follows:

[0023] Given a long-range input sequence The corresponding feature representations are obtained by pre-training a temporal mask autoencoder and a spatial mask autoencoder, respectively. , Downstream prediction models will use a short range of inputs from the test set. Input parameters are Prediction Head Module Extract the hidden representation of its last layer. : for and Shape alignment, retain only , The end Each dimension and flattened into Enhanced features are formed by mapping and fusing through gating and filtering units:

[0024] .

[0025] in, Feature vectors after spatiotemporal fusion , They are respectively and at last The feature vector after flattening each dimension This is the hidden representation vector of the last layer of the prediction head.

[0026] Furthermore, the gated filtering unit maps two long sequence features to the same dimension D' as the downstream hidden state through a two-layer isomorphic multilayer perceptron (MLP). Specifically, MLP downsampling maps the upstream long window representation to the same time scale as the downstream short window. Its internal 1×1 convolution and layer normalization map the channels to the residual dimension while injecting weights into the joint representation learning of upstream and downstream features, thereby achieving dynamic selective fusion of upstream information.

[0027] Furthermore, the downstream prediction model described in S4 operates as follows:

[0028] Accept hidden state features Then, a 1×1 convolution is used to map the channels to the residual dimension:

[0029]

[0030] Stack B dilated and gated convolutional blocks of WaveNet, each containing L layers of the same structure: (1) For residual features Two one-dimensional dilated convolutions are applied in parallel: and ;

[0031] Gated fusion generates native layer output: And it is accumulated to the jump branch through a 1×1 convolution: ,Will After graph convolution:

[0032]

[0033] Adaptive adjacency The residual connection adds the clipped features from the previous layer: The data is then normalized using BatchNorm and used as input for the next layer. After completing all B×L layers, the skip features are accumulated. The external spacetime hidden state is mapped and added through two fully connected channels: The predicted tensor is output after two layers of 1×1 convolution. .

[0034] Furthermore, a conditional adaptive adjacency matrix is ​​constructed during the spatiotemporal feature fusion process. Based on traditional learnable adjacency, upstream spatial representation is introduced as a conditional input, enabling the adjacency matrix to simultaneously reflect the prior topology and the latent spatial correlation learned upstream. This conditional adjacency is used for graph convolution operations in the downstream dilated convolutional block. Graph convolution is used for both local topology injection in skip branches and global topology correction in the main path. The dilation rate of the dilated causal convolution is adaptively adjusted according to the scale of the upstream temporal representation, synergistically expanding the receptive field in the temporal dimension. The upstream long-range representation and conditional adjacency can be pre-computed and cached offline, with only lightweight projection, convolution, and graph convolution performed online.

[0035] Beneficial effects:

[0036] 1. This invention constructs a time mask autoencoder, which uses LSTM units to replace DWT for decoupling through time feature extraction, effectively solving the bottleneck of long sequence modeling, improving the gradient vanishing problem when decoupling long-range features, and further improving the accuracy of subsequent traffic prediction.

[0037] 2. This invention constructs a spatial mask autoencoder, designs and introduces the STATt module (including a temporal attention layer, a spatial attention layer, and a GCN) to achieve dynamic interaction. By reconstructing three operations instead of two independent spatial self-attention modules, it takes into account both temporal dependency modeling and spatial topology modeling, avoiding the computational and overfitting risks caused by directly calculating global attention on a high-dimensional spatiotemporal tensor.

[0038] 3. The feature fusion mechanism of this invention employs a gated screening unit that injects weights based on the joint representation learning of upstream and downstream features, thereby achieving dynamic selective fusion of upstream information and avoiding noise injection. In the downstream prediction model, a conditional adaptive adjacency matrix is ​​constructed. Building upon traditional learnable adjacency, upstream spatial representation is introduced as a conditional input, enabling the adjacency matrix to simultaneously reflect the prior topology and the latent spatial correlation learned from the upstream. During real-time inference, the downstream prediction model only needs to perform lightweight projection and additive fusion on the low-dimensional representation. This preserves the performance improvement brought by long-range information while avoiding additional large-scale spatiotemporal computational overhead. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the overall method of the present invention;

[0040] Figure 2 This is a schematic diagram of the LSTM structure of the present invention;

[0041] Figure 3 This is a structural diagram of the STATt of the present invention;

[0042] Figure 4 This is a structural diagram of the time attention layer of the present invention;

[0043] Figure 5 This is a diagram of the spatial attention layer structure of the present invention;

[0044] Figure 6 This is a diagram showing the ablation experiment results of an embodiment of the present invention; Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] like Figure 1 As shown, this invention discloses a traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training. The method steps are as follows:

[0047] Step 1: Using highway traffic flow as the research object, this embodiment records the traffic flow data to a database file at 5-minute intervals. Redundant, outlier, and missing values ​​in the collected data are filtered and removed to obtain the final traffic flow data;

[0048] Step 2: Establish a pre-training module for temporal and spatial decoupled mask autoencoders, acquire spatial and temporal feature representations respectively, and perform feature fusion. The pre-trained model consists of a temporal mask autoencoder and a spatial mask autoencoder.

[0049] The time mask autoencoder uses LSTM units to learn temporal features. The LSTM structure diagram is shown below. Figure 2 As shown, firstly, Patch Embedding is used to divide the original long input sequence into... Non-overlapping patches of length. Position encoding for a specified patch. The calculation formula is as follows:

[0050]

[0051] Where t is the index of the time step, n is the index of the spatial node, D is the total dimension of the embedding vector, i is the sub-dimension index of the time part, and j is the sub-dimension index of the spatial part.

[0052] The masking strategy follows Bernoulli random sampling, randomly masking the sequence information at time step T. The input can then be obtained. .

[0053]

[0054] The LSTM layer uses a basic Cell structure, and its calculation process is as follows:

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061] in For the Gate of Oblivion For the improved input gate, For memory cells, For output gate, , , It is a weight matrix. and For activation function, for Input at any time , , , For bias.

[0062] The spatial mask autoencoder uses the same patch embedding and positional encoding methods as the temporal mask autoencoder. It mainly consists of STATt modules with temporal and spatial attention layers. The STATt module structure diagram is shown below. Figure 3 As shown, the structures of the temporal attention layer and the spatial attention layer are as follows: Figure 4 and Figure 5 As shown.

[0063] In the STATt module, input tensors , First, the location and time information embedded in the STE is element-wise added to the original features to inject spatiotemporal information. Then, multi-head causal self-attention is applied to the length-T sequence of each node, resulting in:

[0064]

[0065] Each attention head restricts access to the future using a lower triangular mask. Then, Flattened by time dimension Graph convolutional networks combined with graph structure support matrices { Extract the topology information and restore it as follows:

[0066]

[0067] At the same time, for the same intermediate representation Perform multi-head spatial self-attention (without causality masking) on ​​N nodes at each time step to obtain... :

[0068]

[0069] Finally, a gated fusion unit is introduced to combine the temporal convolution results. Results of Spatial Attention Weighted fusion based on learning scores:

[0070]

[0071] The hidden space state output is obtained through residual connection:

[0072]

[0073] In the STATt module, the temporal attention layer and spatial attention layer do not run independently and in parallel, but interact through an explicit data flow and gating fusion mechanism. The temporal attention layer applies multi-head causal self-attention to a sequence of length T for each node, ensuring causality and capturing dependencies within the sequence; its output maintains the same shape as the input for subsequent topological information extraction. This intermediate representation is then flattened along the temporal dimension and input into a graph convolutional network (GCN). The GCN combines the graph's support matrix... The topological correlations between nodes are extracted and reconstructed into a temporal node structure. At each time step, a spatial attention layer performs multi-head self-attention on N nodes to model the dynamic interactions between them. Finally, a gated fusion unit performs weighted fusion of the temporal path (the temporal convolution result after GCN) and the spatial path (the spatial attention result) based on learned weights. This introduces topology-aware spatial information while maintaining temporal causality, and the final output module result is connected via residual connections. This design balances temporal dependency modeling and spatial topology modeling, avoiding the computational and overfitting risks associated with directly calculating global attention on a high-dimensional spatiotemporal tensor.

[0074] The temporal attention layer is responsible for capturing the sequential dependencies within the sequence over time, and its output features of the same shape are used in subsequent graph convolutions and spatial attention layers. First, the input... The element-wise addition with the optional spatiotemporal embedding STE(X) injects positional and temporal information; subsequently, it is processed through a linear mapping layer. , , The D-dimensional features are projected onto the query Q, key K, and value V space; then, the projection result is split into K heads along the node dimension, each subspace having dimension d, to support parallel multi-head computation. For each head, a scaled dot product attention score is calculated:

[0075]

[0076] Then, a lower triangular causal mask is applied to ensure that only information not exceeding t can be accessed at time t. Softmax is then applied to the masked scores to obtain attention weights, which are multiplied by the corresponding value vector v to generate a weighted output. Finally, the outputs of all heads are concatenated back to dimension D in their original order and passed through an output mapping layer. By fusing information from multiple sources, the final [B, T, N, D]-dimensional feature representation is obtained.

[0077] In the spatial attention layer, the input is first... Element-wise addition with the optional spatiotemporal embedding STE(X) through a linear mapping layer , , The D-dimensional features are projected onto the query Q, key K, and value V space. The projection result is split into K heads along the node dimension, each subspace with dimension d, and reshaped into shape [B×T×K, N, d]. For each attention head, a scaled dot product attention score is calculated. The score is then Softmax normalized along the node dimension to obtain the attention weights, which are then compared with the corresponding value vectors.

[0078] The values ​​are multiplied to generate a weighted output; finally, the outputs of all the heads are concatenated in their original order and reconstructed back to [B, T, N, D], and then passed through the output mapping layer. Obtain the module output;

[0079] Step 3: Fusing spatiotemporal features. First, for a given long-range input sequence... The corresponding feature representations are obtained by pre-training a temporal mask autoencoder and a spatial mask autoencoder, respectively. , Secondly, the downstream forecast head will input a short range. Input parameters are Prediction Head Module Extract the hidden representation of its last layer:

[0080]

[0081] Then, in order to... Shape alignment, retain only , The end Each dimension and flattened into Then, these two long sequence features are mapped to the same dimension as the downstream hidden state using two isomorphic multilayer perceptrons (MLPs). Specifically, MLP downsampling is used to map the upstream long window representation to the same time scale as the downstream short window; its internal 1×1 convolution and layer normalization map the channels to the residual dimension while injecting weights into the joint representation learning of upstream and downstream features, thereby achieving dynamic selective fusion of upstream information and avoiding direct injection of noise information.

[0082]

[0083] Each MLP can be written as:

[0084]

[0085] h is the dimension between hidden layers, which is an adjustable hyperparameter.

[0086] Finally, the three sets of feature representations are added together to form the enhanced feature:

[0087]

[0088] To ensure consistency in temporal scale and channel semantics between the long-range spatiotemporal representation obtained from upstream pre-training and the downstream GraphWaveNet residual path, this embodiment first performs temporal alignment projection on the upstream long-sequence representation when receiving the upstream fused hidden state. Specifically, learnable downsampling is used to map the upstream long-window representation to the same temporal scale as the downstream short-window representation; then, 1×1 convolution and layer normalization are used to map the channels to the residual dimension. To avoid directly injecting noise information, a gating filter unit is used to learn the injection weights based on the joint representation of upstream and downstream features, thereby achieving dynamic selective fusion of upstream information.

[0089] Step 4: In this embodiment, GraphWaveNet is used as the downstream prediction head module. The downstream prediction model flow structure is as follows:

[0090] In accepting the hidden state tensor from the previous stage Then, a 1×1 convolution is used to map the channels to the residual dimension:

[0091]

[0092] Subsequently, stack B expanded and gated convolutional blocks of WaveNet, each containing L layers of the same structure: (1) for residual features Two one-dimensional dilated convolutions are applied in parallel:

[0093]

[0094]

[0095] Gated fusion generates native layer output:

[0096]

[0097] And accumulate to the jump branch through a 1×1 convolution:

[0098]

[0099] This embodiment proposes a method for constructing a conditional adaptive adjacency matrix: based on traditional learnable adjacency, an upstream spatial representation is introduced as a conditional input, enabling the adjacency matrix to simultaneously reflect the prior topology and the latent spatial correlation learned upstream. This conditional adjacency is used for graph convolution operations in the downstream dilated convolutional block. Graph convolution is used for both local topology injection in skip branches and global topology correction in the main path. The dilation rate of the dilated causal convolution can be adaptively adjusted according to the scale of the upstream temporal representation, thereby synergistically expanding the receptive field in the temporal dimension with the upstream representation. The upstream long-range representation and conditional adjacency can be pre-computed and cached offline, while only lightweight projection, convolution, and graph convolution are performed online, balancing performance and efficiency.

[0100] Will After graph convolution:

[0101]

[0102] Adaptive adjacency .

[0103] Residual connections add together the clipped features from the previous layer:

[0104]

[0105] The data is then normalized using BatchNorm and used as input for the next layer. After completing all B×L layers, the skip features are accumulated. The external spacetime hidden state is mapped and added through two fully connected channels:

[0106]

[0107] Finally, the predicted tensor is output after two layers of 1×1 convolution. :

[0108]

[0109] The mean squared error is used as the model loss function, and the mean absolute error and mean absolute percentage error are used as the evaluation indicators for the final prediction results. The formulas are as follows:

[0110]

[0111]

[0112]

[0113] To verify the effectiveness of each part of the method of the present invention, an ablation experiment was conducted in this embodiment, and the experimental results were analyzed. The following three variants of MAP-ST-TFF were used for the ablation experiment: (1) Spatial dimension encoding was performed using STATt. (2) : Encoding with improved temporal dimension. (3) MAP-ST-TFF: A complete model that improves temporal and spatial encoding by using STATt and LSTM simultaneously. The results are shown in Table 1. Figure 6 This is a graph showing the model's prediction results.

[0114] Table 1

[0115]

[0116] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training, characterized in that, The method includes the following steps: S1 acquires and preprocesses spatiotemporal sequence data of traffic flow based on multi-source sensors; S2 Constructs a time mask autoencoder, using LSTM units to replace the original DWT for decoupling, and the time mask autoencoder obtains time feature representations; S3 designs and introduces an improved spatiotemporal graph convolution STATt module to construct a spatial mask autoencoder, which acquires spatial feature representations; S4 introduces a gating and filtering unit to fuse spatiotemporal features. These spatiotemporal features are input into the downstream prediction model and the prediction results are output.

2. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training as described in claim 1, characterized in that, The time mask autoencoder structure described in S2 is as follows: The temporal mask autoencoder uses LSTM units to replace the original DWT for decoupling and learning temporal features, and uses Patch Embedding to divide the preprocessed long-flow sequence into segments. For non-overlapping patches of a given length, calculate the position code of the specified patch. ; The masking strategy follows Bernoulli random sampling, randomly masking the sequence information at time step T. The input to the LSTM layer can be obtained after applying a mask. ,in For batch size, For time step, For the number of nodes, As the feature dimension, the LSTM layer progressively reads the input sequence and outputs a hidden temporal feature representation of the same shape. Used for subsequent downstream feature fusion prediction; The LSTM unit adopts a basic Cell structure.

3. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training as described in claim 2, characterized in that, The spatial mask autoencoder Patch Embedding and position encoding method described in S3 are the same as those of the temporal mask autoencoder. The improved spatiotemporal graph convolution STATt module consists of a temporal attention layer, a spatial attention layer, and a GCN layer. The temporal attention layer captures temporal dependencies and ensures causality, and its output is then processed by the GCN layer to extract topological information. The spatial attention layer is responsible for modeling the dynamic interactions between road network nodes.

4. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training as described in claim 3, characterized in that, The operational structure of the improved spatiotemporal graph convolution STATt module is as follows: Input preprocessed flow sequence tensor spatiotemporal embedded information The location and time information are embedded into the STE and added element-wise to the original features to inject spatiotemporal information. Multi-head causal self-attention is then applied to the length-T sequence of each node to obtain the temporal attention result. ,Will Flattened by time dimension Graph convolutional networks combined with graph structure support matrices { Extract topological information and restore the time-series graph convolution result. : Representing the same intermediate At each time step, multi-head spatial self-attention is performed on N nodes to obtain... Introducing a gated fusion unit to process the convolutional results of the time series graph Results of Spatial Attention The spatiotemporal fusion result is obtained by weighting and fusing the learning scores. The hidden space state output is obtained through residual connections. .

5. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training as described in claim 4, characterized in that, The temporal attention layer operation structure of the improved spatiotemporal graph convolution STATt module includes: Input Element-wise addition with the optional spatiotemporal embedding STE(X) to inject position and time information; respectively through linear mapping layers , , The D-dimensional features are projected onto the query Q, key K, and value V space. The projection result is split into K heads along the node dimension, each subspace having dimension d to support parallel multi-head computation. For each head, a scaled dot product attention score is calculated: a lower triangular causal mask is applied to ensure that only information not exceeding t can be accessed at time t. Softmax is then performed on the masked score to obtain attention weights, which are multiplied by the corresponding value vector v to generate a weighted output. Finally, the outputs of all heads are concatenated back to dimension D in their original order and passed through an output mapping layer. By fusing information from multiple sources, the final [B, T, N, D]-dimensional feature representation is obtained.

6. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training as described in claim 4, characterized in that, The spatial attention layer operation structure of the improved spatiotemporal graph convolution STATt module includes: Input Element-wise addition with the optional spatiotemporal embedding STE(X) through a linear mapping layer , , The D-dimensional features are projected onto the query Q, key K, and value V space. The projection result is split into K heads along the node dimension, each subspace having dimension d, and reshaped into shape [B×T×K, N, d]. For each attention head, a scaled dot product attention score is calculated, and the score is normalized along the node dimension using Softmax to obtain the attention weight. This weight is then multiplied by the corresponding value vector 𝑉 to generate a weighted output. The outputs of all heads are concatenated in their original order and reshaped back to [B, T, N, D], and then passed through an output mapping layer. The STATt module output is obtained.

7. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training as described in claim 1, characterized in that, The spatiotemporal feature fusion process described in S4 is as follows: Given a long-range input sequence The corresponding feature representations are obtained by pre-training a temporal mask autoencoder and a spatial mask autoencoder, respectively. , Downstream prediction models will use a short range of inputs from the test set. Input parameters are Prediction Head Module Extract the hidden representation of its last layer. : for and Shape alignment, retain only , The end Each dimension and flattened into Enhanced features are formed by mapping and fusing through gating and filtering units: ； in, The feature vector after spatiotemporal fusion. , They are respectively and at last The feature vector after flattening each dimension This is the hidden representation vector of the last layer of the prediction head.

8. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training according to claim 7, characterized in that, The gated filtering unit maps two long sequence features to the same dimension D' as the downstream hidden state through a two-layer isomorphic multilayer perceptron (MLP). Specifically, it uses MLP downsampling to map the upstream long window representation to the same time scale as the downstream short window. Its internal 1×1 convolution and layer normalization map the channels to the residual dimension while injecting weights into the joint representation learning of upstream and downstream features, thereby achieving dynamic selective fusion of upstream information.

9. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training as described in claim 1, characterized in that, The downstream prediction model described in S4 operates as follows: Accept hidden state features Then, a 1×1 convolution is used to map the channels to the residual dimension: ； Stack B dilated and gated convolutional blocks of WaveNet, each containing L layers of the same structure: (1) For residual features Two one-dimensional dilated convolutions are applied in parallel: and ; Gated fusion generates native layer output: And it is accumulated to the jump branch through a 1×1 convolution: ,Will After graph convolution: ； Adaptive adjacency The residual connection adds the clipped features from the previous layer: After being normalized by BatchNorm and used as input for the next layer, the skip features are accumulated after completing all B×L layers. The external spacetime hidden state is mapped and added through two fully connected channels: The predicted tensor is output after two layers of 1×1 convolution. .

10. The traffic flow prediction method based on spatiotemporal decoupling mask autoencoder pre-training according to claim 9, characterized in that, In the spatiotemporal feature fusion process, a conditional adaptive adjacency matrix is ​​constructed. Based on the traditional learnable adjacency, an upstream spatial representation is introduced as a conditional input, so that the adjacency matrix simultaneously reflects the prior topology and the latent spatial correlation learned upstream. This conditional adjacency is used for graph convolution operations in the downstream dilated convolution block. Graph convolution is used for both local topology injection of skip branches and global topology correction of the main path. The dilation rate of the dilated causal convolution is adaptively adjusted according to the scale of the upstream temporal representation, and it expands the receptive field in collaboration with the upstream representation in the temporal dimension. The upstream long-range representation and the conditional adjacency can be pre-computed and cached in the offline stage, and only lightweight projection, convolution and graph convolution are performed online.

Images