A multimodal spatiotemporal traffic flow modeling method supporting real-time prediction of large-scale road networks

Abstract

The present invention belongs to the field of intelligent transportation systems and relates to a multimodal spatiotemporal traffic flow modeling method that supports real-time prediction of large-scale road networks. The method first designs a spatiotemporal prediction framework for dynamic traffic networks and then performs training to obtain the final prediction model. The spatiotemporal prediction framework includes a data embedding layer, a spatiotemporal coding module, and a deep modeling and output module based on an expert mixture mechanism. The data embedding layer includes two parallel channels, namely time embedding and spectral-domain spatial embedding, and a spatiotemporal data fusion module. The spatiotemporal coding module includes a parallel block-level sparse temporal attention module and a spatial attention-message passing module and a weighted fusion layer. The deep modeling and output module based on the expert mixture mechanism includes a MoE dynamic expert modeling module, a fully connected mapping module, a skip connection layer, and an output layer. This design improves the model's response speed, prediction accuracy, and cross-regional adaptability in highly heterogeneous scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation systems, and in particular relates to a multi-mode spatiotemporal traffic flow modeling method that supports real-time prediction of large-scale road networks. Background Art

[0002] The core of intelligent transportation systems lies in the accurate prediction of traffic flow dynamics, which directly impacts key functions such as traffic signal optimization, congestion warning, and emergency dispatch. Traditional prediction models based on static spatiotemporal assumptions face severe performance degradation in scenarios with high spatiotemporal heterogeneity, such as sudden crowd gatherings (such as holiday visitor surges at scenic spots), instantaneous high-density crowd concentrations (such as after sporting events), and the coupling of multiple flow sources (such as during peak transfer times at transportation hubs).

[0003] Traditional traffic flow prediction technologies often rely on pre-set graph structures and fixed time series modeling frameworks. For example, traffic flow prediction models based on graph convolutional neural networks have been widely used (e.g., Chinese patent CN117037491A). This method extracts spatial features from a predefined road adjacency matrix and combines it with a time series embedding mechanism to model the evolution of traffic states. The model proposed in this patent combines graph convolution, temporal attention, and external factor modeling, improving prediction accuracy to a certain extent. However, its graph structure construction still relies on static embedding methods such as Node2vec, which cannot dynamically reflect the spatial dependency adjustments caused by event disturbances or changes in traffic density during actual operation.

[0004] Self-attention mechanisms for global modeling have become a mainstream research area in spatiotemporal sequence prediction. For example, the multi-head self-attention mechanism proposed in Chinese patent CN117037483A can be used to capture long-range dependencies. However, this type of mechanism suffers from the problem of discrete distribution of attention weights when faced with non-uniform sampling, multi-period perturbations, or discontinuous event sequences, making it difficult to effectively model the short-term impact of local mutations. Furthermore, because the computational complexity of this attention mechanism in the time dimension is quadratic (O(T²)), it is prone to computational bottlenecks when applied to long sequences or high-frequency traffic data, making it particularly challenging to deploy in real time on edge devices.

[0005] In terms of spatial modeling, existing technologies often use graph convolutional frameworks driven by static adjacency graphs (such as Chinese patent CN117037491A), which fail to fully account for the dynamic topological changes of traffic networks. This is especially true during rush hour or traffic accident scenarios, where traffic paths and inter-node connections can rapidly change. Static graph models lack adaptability, which can easily lead to distorted depictions of linkages between upstream and downstream nodes, leading to cumulative amplification of prediction errors. Although some studies have attempted to introduce dynamic graph structure learning mechanisms, these are often accompanied by problems such as increased computational complexity and decreased network interpretability. For example, Chinese patent CN119443156A uses a fully connected attention structure for spatial modeling. However, its O(N²) message passing mechanism, with its complexity, is unable to support the performance requirements of online reasoning when faced with a road network with tens of thousands of nodes.

[0006] Regarding spatiotemporal cascade modeling of traffic conditions, Chinese patent CN119107799A proposes a graph convolutional recurrent neural network (GCNN) that accounts for data loss, emphasizing the impact of data integrity on traffic prediction accuracy. While this approach enhances the model's robustness to anomalous inputs by introducing a Gaussian mixture mechanism, it is insufficient for modeling the effects of propagation delays between multiple nodes. In real road networks, congestion often exhibits cascading diffusion characteristics, particularly in compact urban cores, where its propagation paths exhibit complex patterns such as nonlinear superposition and multi-scale time lags. Existing local models based on geometric neighborhoods or sliding windows (such as those in Chinese patent CN119107799A) lack a detailed description of congestion evolution paths due to their coarse detection area divisions. This makes it difficult to accurately capture the spatiotemporal dependencies and dynamic changes in the propagation process, leading to error amplification. In particular, when faced with non-stationary disturbances such as holiday passenger traffic surges and transient congestion caused by large-scale events, the predictive performance of these sliding window-based models significantly degrades, failing to meet the precise response requirements for emergencies in real-world applications.

[0007] Some research attempts to improve model generalization by incorporating multi-source information. For example, Chinese patent CN119541217A proposes integrating external factors such as weather and holidays through feature splicing. However, these approaches often employ static fusion strategies and lack a learnable dynamic weighting mechanism. This can lead to reduced model prediction stability when external environmental data is noisy or data sources are missing. In particular, when sensors are sparsely deployed or experiencing offline failures, the indiscriminate injection of redundant features can further interfere with model training and decision-making.

[0008] In summary, current traffic flow prediction technology still has the following significant shortcomings in dealing with complex traffic behaviors and sudden disturbances in real urban environments: First, the static graph structure and decoupled spatiotemporal modeling methods make it difficult to characterize the evolution process of the topological structure and the time delay characteristics of path propagation in dynamic traffic networks; second, the temporal modeling layer does not sufficiently explore the cross-nested relationship between short-term disturbances and periodic laws, resulting in unstable model performance in non-stationary traffic scenarios; third, the overall modeling framework has obvious bottlenecks in computational efficiency, feature fusion robustness, and cross-regional generalization capabilities, making it difficult to meet the real-time prediction and deployment requirements in large-scale road networks. Summary of the Invention

[0009] In order to solve the technical bottlenecks of existing traffic flow prediction models in spatiotemporal coupling modeling, dynamic topology perception, computational efficiency and scenario adaptability, the present invention provides a multimodal spatiotemporal traffic flow modeling method that supports real-time prediction of large-scale road networks. This method designs a spatiotemporal prediction framework for dynamic traffic networks, which can support multimodal travel behavior modeling, and has dynamic structure perception, efficient reasoning ability and elastic feature fusion mechanism. It significantly improves the response speed, prediction accuracy and cross-regional adaptability of the model in highly heterogeneous scenarios, and meets the actual application needs of intelligent transportation systems in emergency response, traffic scheduling optimization and long-term flow control.

[0010] To achieve the above object, the present invention adopts the following technical solutions:

[0011] A multimodal spatiotemporal traffic flow modeling method that supports real-time prediction of large-scale road networks. The method first designs a spatiotemporal prediction framework for dynamic traffic networks and then trains the designed spatiotemporal prediction framework to obtain a final prediction model. The designed spatiotemporal prediction framework includes a data embedding layer, a spatiotemporal encoding module, and a deep modeling and output module based on a mixture of experts mechanism.

[0012] The data embedding layer is used to convert the original spatiotemporal data into a high-dimensional dense representation. The data embedding layer includes two parallel channels, time embedding and spectral-domain spatial embedding, and a spatiotemporal data fusion module. The time embedding channel is used to extract temporal semantic features based on local temporal features and periodic position coding, and the spectral-domain spatial channel obtains spatial-frequency domain structural features through graph Laplacian spectrum decomposition. The spatiotemporal data fusion module fuses temporal semantic features and spatial-frequency domain structural features through element-by-element addition operations to obtain input data for the spatiotemporal coding module.

[0013] The spatiotemporal encoding module is used to perform temporal modeling and spatial modeling on the preliminary fused feature tensor obtained by the data embedding layer, and includes a parallel block-level sparse temporal attention module, a spatial attention-message passing module and a weighted fusion layer; the block-level sparse temporal attention module adopts a self-attention mechanism based on time blocks and introduces a dynamic mask mechanism, which slices the input data according to nodes, extracts the time series corresponding to each node from the input data, and performs local modeling and scale alignment on the temporal features; the spatial attention-message passing module integrates the potential dynamic graph structure, the kernel function approximation dynamic graph attention mechanism and the adaptive dynamic time warping driven spatiotemporal mask mechanism, which slices the input data according to time steps to model the dynamic spatial dependency relationship between nodes in the traffic network; the weighted fusion layer is used to perform feature alignment on the outputs of the block-level sparse temporal attention module and the spatial attention-message passing module, and realizes weighted fusion through learnable weight coefficients, and outputs a unified spatiotemporal feature representation after fusion;

[0014] The deep modeling and output module based on the expert mixture mechanism is used to model the unified spatiotemporal feature tensor and generate prediction output; it includes an MoE dynamic expert modeling module, a fully connected mapping module, a jump connection layer, and an output layer; the MoE dynamic expert modeling module includes multiple independent expert sub-networks and a gating network, and the gating network is used to dynamically calculate the weight coefficients of the expert sub-networks to achieve weighted fusion of multiple expert outputs; the fully connected mapping module is used to perform fully connected mapping on the features output by the MoE dynamic expert modeling module to obtain preliminary prediction results; the jump connection layer is used to transfer the spatiotemporal fusion features to the linear mapping layer to retain low-order feature information; the output layer introduces a learnable fusion weight coefficient to balance the outputs of the jump connection and fully connected mapping modules to generate the final prediction results.

[0015] As a preferred embodiment of the present invention, the time embedding channel constructs a fine-grained time series representation of traffic flow by fusing local time series features with explicit periodic semantics. The specific data processing flow is as follows:

[0016] Step A. Input the original tensor Perform unified dimension projection and use a fully connected layer (i.e. ) is mapped to the fixed-dimensional embedding space to obtain the standard feature embedding, which is expressed as: ;

[0017] Step B. Set up two learnable embedding dictionaries, namely the weekday embedding dictionary and intraday timestamp embedding dictionary , respectively construct the original tensor The embedding tensor of the week , intraday timestamp embedding tensor ;

[0018] Step C. Concatenate the standard feature embedding, week embedding, and intraday timestamp embedding on the feature dimension. Input the concatenated time embedding into a linear learnable projection layer to obtain high-order temporal semantic features. ;

[0019] Step D. Define the position encoding matrix , by element-by-element addition to fuse the periodic semantics and position structure; finally, the output of the temporal embedding channel .

[0020] As a preferred embodiment of the present invention, the data processing flow of the spectral domain spatial channel is as follows:

[0021] Step A. constructing a symmetric normalized graph Laplacian matrix based on the self-loop-free adjacency matrix and the correspondence matrix of the traffic network;

[0022] Step B. Perform eigenvalue decomposition on the graph Laplace matrix, remove the eigenvectors corresponding to zero eigenvalues, start from the eigenvector corresponding to the second smallest eigenvalue, and select the first feature vectors to construct low-dimensional spectral graph embedding;

[0023] Step C. Use a learnable linear function to embed the low-dimensional spectrogram into a high-dimensional space to obtain spatial embedding features, i.e., spatial frequency domain structural features. .

[0024] As a preferred embodiment of the present invention, the data processing flow of the spatiotemporal data fusion module is as follows:

[0025] Step A. Perform element-wise addition operation on the output of the time embedding channel and the output of the spectral domain spatial channel to generate a preliminary spatiotemporal embedding tensor ;

[0026] Step B. Introducing a learnable positional encoding matrix , and add it to the preliminary spatiotemporal embedding tensor element by element to obtain the final fused spatiotemporal encoding module input feature tensor .

[0027] As a preferred embodiment of the present invention, the data processing flow of the block-level sparse temporal attention module is as follows:

[0028] Step A: Slice at the node level and extract the corresponding time series from the input data for each node. Then, use a sliding window strategy to divide the time series corresponding to the node into multiple time blocks with overlapping areas along the time axis.

[0029] Step B. Flatten the two-dimensional features in each time block into a one-dimensional vector to obtain a flattened time block sequence;

[0030] Step C. The flattened time block sequence is input into a learnable linear transformation module to obtain the embedding tensor ;

[0031] Step D. Introduce a multi-head self-attention mechanism. For each attention head, based on the obtained embedding tensor , respectively generate query, key and value vectors through learnable linear mapping;

[0032] Step E. For each attention head, according to The query vector of the time block and the The key vector of the time block is calculated Time block for the The attention scores of time blocks are obtained to obtain the attention score matrix;

[0033] Step F. In each query time block Only the attention score before The key vector position index is used to perform softmax normalization on the reserved index position, and the attention scores of the remaining positions are set to zero to achieve sparse connection and obtain the sparse attention matrix ;

[0034] Step G. Introduce dynamic channel mask mechanism and construct mask matrix , each element of the matrix obeys Bernoulli distribution sampling, which is used to indicate whether to retain the first The channel in Output on time blocks;

[0035] Step H. Multiply the sparse attention matrix by the dynamic mask matrix element by element to obtain the attention output with suppression effect ;

[0036] Step I. Concatenate the outputs of all attention heads along the channel dimension and fuse them through a learnable linear layer to obtain the intermediate output of the block-level sparse temporal attention module. ;

[0037] Step J: The intermediate output is normalized by the layer to obtain the final normalized output;

[0038] Step K. Introduce a one-dimensional transposed convolution operation to restore the final normalized output to its original time length. Then all nodes output tensors after passing through the block-level sparse time attention module. .

[0039] As a preferred embodiment of the present invention, the data processing flow of the spatial attention-message transmission module is as follows:

[0040] Step A: Slice the input data in time steps and construct a T-frame time series graph, where each frame contains all nodes. Layer, and The characteristics of the nodes are and ;Introduce three learnable linear mapping matrices to construct query vector, key vector and value vector respectively;

[0041] Step B. Replace the dot product form of attention similarity calculation by the approximate method of the positive kernel function. , It is a low-dimensional kernel feature mapping function, which is constructed based on random Fourier features; Representing the The query vector of the node and the The key vector of the nodes, Indicates the index of the central node currently being updated. Represents the candidate source node index;

[0042] Step C: Introduce the spatiotemporal masking mechanism driven by adaptive dynamic time warping, node With node The mask coefficients corresponding to the spatiotemporal mask matrix for:

[0043] ;

[0044] in, Representation node and There is a direct geographical connection in the transportation network, otherwise ; Indicates that the node pair The connection strength under the influence of external constraints between ; is the temperature factor, which is used to adjust the influence of DTW similarity on mask weight. Representative Node The original time series characteristics DTW distance, Same definition as input time step length;

[0045] Step D. Use the spatiotemporal mask matrix as a priori weights to update the graph neural network Nodes in the layer Features, nodes The features at the current layer are constructed from the query, key, and value vectors of the previous layer features, and are obtained by aggregating neighbor information in a weighted manner using a kernel function. The calculation of the attention weight and the feature update process are as follows:

[0046] ;

[0047] Among them, the index represents all nodes used for normalization, represents the corresponding key vector, Representation node The value vector of , the elements in the mask matrix The construction logic and Consistent, it is a node With node For the mask coefficients corresponding to the spatiotemporal mask matrix, the kernel mapping operation in the formula can share the calculation results to avoid redundancy, thereby optimizing its computational complexity.

[0048] As a preferred embodiment of the present invention, a layer normalization mechanism and a residual connection structure are introduced into the weighted fusion layer, that is, the fused feature tensor is layer normalized and the residual is added to the initial input feature as the final output.

[0049] As a further preferred embodiment of the present invention, when the time embedding channel processes the data, for each time step , the corresponding even-numbered dimensions use the sine function to calculate the position encoding matrix , the corresponding odd dimensions use the cosine function to calculate the position encoding matrix .

[0050] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0051] (1) In view of the limitation of the existing spatiotemporal embedding methods in modeling temporal patterns and spatial structures separately, the present invention proposes a fused spatiotemporal spectrum embedding mechanism to achieve a unified, high-dimensional structured expression of traffic flow data. The mechanism captures the intra-day periodicity and short-term local fluctuation characteristics in the traffic sequence by combining local temporal features with periodic semantic coding in the time channel. In the spatial channel, graph Laplacian spectral decomposition and clustering strategies are simultaneously introduced to extract the low-frequency principal components of the road network structure, thereby achieving a compact expression (compressed expression) of the spatial structure and feature de-redundancy. In addition, a learnable position code is introduced as a structural prior, and through explicit combination with the fusion feature, the implicit spatiotemporal structure information in the input sequence is effectively perceived. Finally, a unified time-frequency joint representation space is constructed, which comprehensively enhances the model's coverage and modeling expression of spatiotemporal coupling features in complex traffic scenarios. Compared with the traditional single-modal embedding method, it has stronger spatiotemporal coupling modeling capability and generalization performance.

[0052] (2) To address the high computational complexity and insufficient capture of local patterns in the attention mechanism in long-term time series prediction, this paper innovatively designs a time series modeling mechanism based on block compression. This mechanism divides the initial time series sequence into multiple sliding time segments, performs local attention modeling in each time block, and improves the ability to capture short-term fluctuations and periodic signals. To further reduce the computational burden, a block-level sparse time attention structure is constructed based on the Top-k strategy under the time block unit, which not only effectively captures fine-grained local dynamics and periodic fluctuations, but also reduces the attention computational complexity to a sublinear level, improving the computational throughput while maintaining the accuracy of time series correlation. In addition, an adaptive masking mechanism is introduced to dynamically adjust the attention weights through Bernoulli sampling in the feature dimension, realizing feature-level perturbation injection and generalization enhancement. This mechanism improves the robustness and cross-scenario prediction capability of the model in non-stationary time series change scenarios (such as holidays, emergencies, frequent jumps, etc.).

[0053] (3) Aiming at the problems of solidified spatial dependency structure of traffic network, difficulty in adapting to dynamic evolution and high computational overhead restricting real-time prediction, this paper proposes an efficient spatial attention mechanism based on kernel function approximation and dynamic regularization. This mechanism is based on kernel function mapping and effectively compresses the similarity calculation and information propagation process between nodes by learning the potential graph structure and the kernel approximation method based on random Fourier features, thus reducing the computational complexity of spatial attention from 100% to 100%. Optimize to While maintaining the ability to model potential topological relationships, it significantly improves the scalability and real-time processing capabilities on traffic maps with a scale of tens of thousands of nodes (supports real-time reasoning on a scale of tens of thousands of nodes), meeting the deployment requirements in edge computing scenarios.

[0054] (4) In order to solve the problem of spatial modeling deviation caused by asynchronous traffic evolution between nodes and irregular sudden events, this paper introduces for the first time a spatiotemporal masking mechanism driven by adaptive dynamic time warping. This mechanism uses the DTW algorithm to measure the temporal consistency of historical traffic sequences between nodes, and dynamically adjusts the attention weights in combination with external constraints such as geographical connectivity and road control. This masking mechanism works in conjunction with kernelized spatial attention to guide the model to focus on node pairs with similar path evolution trends and meeting actual road connectivity, thereby improving the ability to express propagation delay effects, enhancing the model's stability and prediction accuracy during peak hours and in heterogeneous traffic environments, as well as its adaptability and robustness to sudden abnormal scenarios.

[0055] (5) To address the dimensionality mismatch and semantic gap problems in multimodal feature fusion, the present invention proposes a dynamic weighted spatiotemporal joint optimization framework; dynamic weighted coupling of spatiotemporal attention outputs is achieved through a differentiable gating mechanism, and a mixture of experts network is combined to replace the traditional feedforward network to achieve weighted fusion of multi-expert outputs. At the same time, combined with jump connections and adaptive fusion strategies, the complementarity between low-order spatiotemporal features and deep semantic features is retained, and the model's ability to jointly model traffic flow mutation patterns and long-range evolution laws is enhanced, thereby improving the model's expressiveness and generalization performance in heterogeneous traffic behavior scenarios and enhancing its adaptability to complex urban traffic management needs.

[0056] (6) The model provided by the present invention demonstrates excellent prediction accuracy and structural robustness on typical datasets such as PeMS, NYCTaxi, T-Drive, and CHIBike, and demonstrates strong robustness and cross-scenario generalization capabilities in dense urban areas, main road corridors, and multi-station travel networks, meeting the practical application needs in emergency response, traffic scheduling optimization, and long-term flow control. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of the spatiotemporal prediction framework for dynamic transportation networks constructed by the present invention;

[0058] Figure 2 A data processing flow chart for the spatiotemporal prediction framework for dynamic transportation networks constructed by the present invention;

[0059] Figure 3 Processing flow chart of the block-wise temporal self-attention module based on dynamic sparse masking constructed for the present invention;

[0060] Figure 4 Processing flow chart of the kernel-based spatial attention-message passing module constructed for the present invention;

[0061] Figure 5 is the loss curve of the training and validation process;

[0062] Figure 6 Visualize the PeMSD8 dataset for data prediction results. DETAILED DESCRIPTION

[0063] In order to enable those skilled in the art to better understand the technical solutions and advantages of the present invention, the present application is described in detail below with reference to the accompanying drawings, but this is not intended to limit the scope of protection of the present invention.

[0064] like Figures 1 to 4As shown, the present invention provides a multi-modal spatiotemporal traffic flow modeling method that supports real-time prediction of large-scale road networks. The method first designs a spatiotemporal prediction framework for dynamic traffic networks, and then trains the designed spatiotemporal prediction framework to obtain a final prediction model. The designed spatiotemporal prediction framework consists of a data embedding layer, a spatiotemporal encoding module, and a deep modeling and output module based on an expert mixture mechanism.

[0065] The data embedding layer is used to perform spatiotemporal feature mapping on the raw traffic flow input data to construct a high-dimensional dense representation. The data embedding layer includes two parallel channels: time embedding and spectrum-domain spatial embedding, and a spatiotemporal data fusion module. The model first constructs a unified high-dimensional dense representation through time embedding and spectrum-domain spatial embedding (graph embedding), which is then input into the spatiotemporal encoding module for joint modeling.

[0066] The temporal modeling component (block-level sparse temporal attention module) of the spatiotemporal encoding module divides the embedded data into time blocks using a sliding window, extracts local and global dependencies using a sparse multi-head attention mechanism based on a Top-k strategy, and improves the model's robustness to perturbations by combining query-adaptive masking. The spatial modeling component (spatial attention-message passing module) uses a dynamic graph attention mechanism with kernel function approximation to improve the efficiency of modeling dynamic adjacency relationships by constructing a latent graph structure. It further introduces a masking mechanism based on dynamic time warping (DTW) that considers external constraints to guide spatial attention to focus on temporal consistency between nodes. Finally, the spatiotemporal data is fused through a spatiotemporal data fusion module, and the fused spatiotemporal features are input into a deep modeling and output module based on a mixture of experts mechanism.

[0067] The deep modeling and output module based on the expert mixture mechanism is used to model the unified spatiotemporal feature tensor and generate prediction output; it includes an MoE dynamic expert modeling module, a fully connected mapping module, a jump connection layer, and an output layer; the MoE dynamic expert modeling module includes multiple independent expert sub-networks and a gating network, and the gating network is used to dynamically calculate the weight coefficients of the expert sub-networks to achieve weighted fusion of multiple expert outputs; the fully connected mapping module is used to perform fully connected mapping on the features output by the MoE dynamic expert modeling module to obtain preliminary prediction results; the jump connection layer is used to transfer the spatiotemporal fusion features to the linear mapping layer to retain low-order feature information; the output layer introduces a learnable fusion weight coefficient to balance the outputs of the jump connection and fully connected mapping modules to generate the final prediction results.

[0068] In the traffic flow prediction task, we define Indicates that at each time step At this time, the road network The traffic flow of nodes, Represents the dimension of traffic flow, which is the original tensor after combining the traffic flow on all time slices :

[0069] ;

[0070] The goal of traffic flow prediction is to predict the traffic flow of a traffic system at a future time based on given historical observation data. Formally, let the traffic flow tensor observed in the traffic system be , this embodiment aims to learn a mapping function , so that it can be based on the input Traffic flow observations at time steps, combined with the traffic network graph structure , predicting the future The traffic flow tensor of time steps is:

[0071] ;

[0072] To achieve the above prediction goals and effectively capture the complex spatiotemporal dynamic characteristics of the traffic system, this embodiment proposes a data embedding layer that aims to convert the original spatiotemporal data into a high-dimensional dense representation, thereby providing sufficient feature support for subsequent modules.

[0073] In this embodiment, the data embedding layer includes two parallel channels of time embedding and spectral domain spatial embedding and a spatiotemporal data fusion module. The two parallel channels respectively extract the temporal semantic features based on local temporal features and periodic position coding, and the spatial frequency domain structural features obtained by graph Laplacian spectrum decomposition; then, the key prior knowledge is integrated into the representation learning process through spatiotemporal data fusion to obtain the preliminary fusion features. ,in The unified feature dimension input to the encoder (space-time encoding module) serves as the unified representation space processed by the subsequent space-time encoding module.

[0074] Specifically, the expressions of the two parallel channels included in the data embedding layer are:

[0075]

[0076] in, Representing time-embedded channels to model periodic patterns and short-term disturbances in traffic flow sequences; It represents the spectral domain spatial embedding channel, which combines graph structure spectral decomposition and uses the eigenvectors of the road network Laplacian matrix to extract the implicit patterns of traffic data in the frequency domain, thereby further revealing the intrinsic connection between spectral characteristics and road network structure.

[0077] In this embodiment, the time embedding channel constructs a fine-grained time series representation of traffic flow by fusing local time series features with explicit periodic semantics. In order to preserve the intrinsic information in the original data as much as possible, the original tensor of the input is first Perform unified dimension projection and use a fully connected layer (i.e. ) is mapped to a fixed-dimensional embedding space to obtain standard feature embedding , expressed as: ,in, Indicates the hidden dimension set by the network structure (a global fixed value is set in this embodiment).

[0078] Secondly, in order to introduce explicit periodic time semantic information, this embodiment sets two learnable time embedding dictionaries, namely week embedding dictionary and intraday timestamp embedding dictionary .in, Indicates the seven natural days of the week. Indicates the number of time periods in a day with a granularity of 5 minutes. In the example, set the week index and intraday timestamp index , extract the corresponding embedding representation from the above embedding dictionary and construct the week embedding tensor respectively Embedding tensors with timestamps . Then, to keep the three embedding tensor dimensions consistent, and The copies are broadcasted across the node dimension N and compared with the standard feature embedding tensor Perform concatenation on the feature dimension to obtain the fused time series representation tensor: , where the symbol Represents the concatenation operation on the feature dimension (i.e. the last dimension). Finally, the concatenated tensor Mapped to a unified high-order representation space, achieving temporal semantic fusion and feature dimension compression, we obtain .

[0079] To further enhance the sequential information in the time series features, this embodiment adopts the sine-cosine position encoding method, which can effectively capture the sequential structure in the sequence and has translation invariance and scalability. and each dimension , define the position encoding matrix The calculation rules are as follows:

[0080] ;

[0081] For each time step , the corresponding even dimensions ( dimensional) are encoded using a sine function, and the corresponding odd dimensions ( Finally, the position encoding matrix is broadcasted using a cosine function. Extension to temporal semantic tensors The same dimension, get the broadcasted position encoding tensor , and finally perform feature integration by element-wise addition to obtain the final output tensor of the time embedding channel:

[0082] .

[0083] In this embodiment, a spectral domain spatial embedding channel is set up, and a spectral decomposition method of a graph Laplacian matrix is adopted to model the spatial structural characteristics of the transportation network.

[0084] Given a traffic network graph structure The self-loop-free adjacency matrix and the correspondence matrix ,in N Indicates the number of nodes in the graph. In the elements Representation node With node Is there an edge between them? If there is an edge, then ,otherwise ,in , Represents a node set. Since the present invention considers a graph without self-loops, it satisfies The corresponding graph degree matrix is a diagonal matrix, whose diagonal elements Representation node The degree of is defined as: Based on the above definition, the present invention further constructs a symmetric normalized graph Laplacian matrix: ,in represents the identity matrix, since is a symmetric semi-positive definite matrix, and its eigenvalue reflects the structural characteristics of the graph. In order to further extract the frequency domain feature information of the spectrum, the present invention Performing eigenvalue decomposition (i.e. spectral decomposition) yields: ,in, is the diagonal eigenvalue matrix, is the eigenvector matrix, the superscript Represents transpose.

[0085] In order to extract the low-frequency global information of the graph, the present invention excludes the eigenvectors corresponding to zero eigenvalues. , starting from the eigenvector corresponding to the second smallest eigenvalue, select the first non-trivial eigenvectors (in this embodiment ), construct low-dimensional spectral graph embedding:

[0086] ;

[0087] Finally, in order to obtain a unified feature format and facilitate subsequent spatiotemporal feature fusion, this embodiment embeds the spectrogram into Projection to the preset feature space , and at this stage along the time dimension Perform broadcast copy to get the spatial embedding tensor .

[0088] In this embodiment, after the extraction of temporal embedding features and spatial embedding features is completed, the spatiotemporal data fusion module integrates the two types of features by element-wise addition to generate a spatiotemporal feature tensor in a unified format for use in subsequent modules.

[0089] Specifically, the temporal embedding and spatial embedding are element-wise added to generate a fused tensor in a unified format (preliminary temporal and spatial embedding tensor). ;

[0090] In addition, in order to enhance the model's ability to model the implicit spatiotemporal structural information in the input sequence, a learnable position encoding matrix is introduced in the data embedding layer. As a structural prior. The position encoding matrix Along the time dimension through the broadcast mechanism Copy the extension so that its dimensions are the same as Keep it consistent and add element by element to get the final fused spatiotemporal input feature tensor .

[0091] like Figure 3 As shown, in this embodiment, a block-level sparse temporal attention module is constructed, and the block-level sparse temporal attention module is used to perform local modeling and scale alignment processing on temporal features.

[0092] Traffic flow data is highly nonlinear and dynamic, making it difficult for traditional point-by-point modeling methods to effectively capture its temporal dependencies. To address this, we design a patch-based temporal self-attention (PTSA) mechanism to extract short- and long-term dependencies in time series. We also integrate an adaptive masking mechanism to enhance model robustness and prevent over-reliance on a single time point.

[0093] Specifically, in this embodiment, the block-level sparse temporal attention module adopts a self-attention mechanism based on time blocks and introduces a dynamic mask mechanism to form a block-wise temporal self-attention mechanism based on dynamic sparse masks. The data processing flow is as follows: first, the embedded time series is divided into overlapping time blocks, and the contextual dependencies between time blocks are modeled based on the Top-k sparse multi-head self-attention mechanism. A query adaptive masking strategy based on Bernoulli distribution is introduced to achieve dynamic selection of feature dimensions and enhanced robustness to disturbances. Afterwards, layer normalization is combined to improve nonlinear modeling capabilities, and one-dimensional transposed convolution is used to upsample and restore the downsampled temporal features to provide a unified input for subsequent spatiotemporal fusion.

[0094] Specifically, the data processing flow of the block-level sparse temporal attention module is as follows:

[0095] Step A. Overlapping time block generation and high-order timing modeling:

[0096] The present invention adopts a sliding window strategy to divide the multidimensional feature tensor after spatiotemporal encoding into multiple time blocks with overlapping areas according to the time axis, so as to enhance the model's ability to model local dynamic changes.

[0097] Assume the input feature tensor is ,in is the batch size, represents the number of nodes in the traffic network, is the time step length, The unified feature dimension of the encoder (spatiotemporal encoding module).

[0098] First, slice according to the node level. For each node , from the input feature tensor Extract the corresponding time series: ; For each node, its corresponding time series is divided into lengths of , the step length is The total number of time blocks is:

[0099] ;

[0100] Among them, the symbol Represents a floor operation, and the time block of each node constitutes a four-dimensional tensor: ; In order to capture the global semantic structure in each time block, the two-dimensional features in each time block (i.e., the size is ) is flattened into a one-dimensional vector to obtain the flattened time block sequence: , the flattened time block sequence is a three-dimensional tensor;

[0101] Then, the flattened time block sequence is input into a learnable linear transformation module (linear) to map it to a unified high-order time series embedding space. Let the embedding dimension after mapping be ,in is the number of heads of the multi-head time attention set in step B (in this embodiment, is 4), is the subspace dimension of each head. The mapping result is a node The corresponding time block embedding tensor: .

[0102] Step B. Multi-head temporal attention and dynamic mask mechanism:

[0103] In order to model the high-order temporal dependencies between overlapping time blocks, this paper introduces a multi-head self-attention mechanism. , the present invention is based on the node time block embedding tensor constructed in step A , generate query, key, and value vectors respectively through the following linear mapping 、 、 :

[0104] ;

[0105] in, Respectively The attention heads are used to construct a learnable projection matrix of query, key, and value vectors, ensuring that each head learns temporal patterns independently in the subspace.

[0106] In order to further improve the computational efficiency and structural sparsity of multi-head temporal attention, this paper introduces a block-level Top-k sparse attention mechanism and a dynamic channel mask mechanism. Specifically, for each attention head , set up the The query vector for a time block is , No. The key vector of a time block is , then the scaled dot product attention score between the two is defined as: . Stack all query vectors and key vectors into a matrix, where Indicates the Time block for the The attention score of each time block is obtained, and the attention score matrix is: .

[0107] In order to improve efficiency, the present invention adopts On the top, use Top- The selection strategy is to only keep the previous The key vector position index, build the collection . Softmax normalization is performed only on these index positions, and the attention scores of the remaining positions are set to zero to achieve sparse connections, so the sparse attention weight is defined and the sparse attention matrix The calculation is as follows:

[0108] ;

[0109] ;

[0110] In order to enhance the robustness of the model and the ability to select feature channels, this paper introduces a dynamic channel mask mechanism after the multi-head temporal attention output. This mechanism is achieved by masking the channel dimension (i.e. the last dimension) in the output of each attention head. ) Dynamic retention strategy to achieve the suppression of feature-level noise and the enhancement of effective information. Specifically, a dynamic mask matrix is constructed for each attention head. , among which The retention probability of a channel is , dynamically generated using a linear increasing strategy, namely:

[0111] ;

[0112] in, and are hyperparameters, representing the minimum and maximum retention probabilities, respectively, satisfying , to achieve the incremental design of channel retention probability. Each element of the matrix Follows Bernoulli distribution:

[0113] ;

[0114] The Bernoulli distribution function Indicates that the sampling value is The probability of , which means that the channel is in the time block The mask matrix obtained by sampling is After broadcasting and sparse attention output tensor The dimensions are consistent, and finally the attention output with suppression effect is obtained by element-by-element multiplication of the sparse attention matrix and the dynamic mask matrix: .in, Represents the Hadamard product, or element-wise multiplication. This mechanism maintains the expressive power of attention while introducing dynamic selection and suppression at the channel level, effectively improving the model's robustness and feature selection capabilities.

[0115] Subsequently, the outputs of all attention heads are concatenated along the channel dimension and fused through a learnable linear layer, where the learnable linear mapping matrix is , and obtain the intermediate output of the block-level sparse temporal attention module :

[0116] ;

[0117] In order to alleviate feature distribution drift and improve training stability, after obtaining the intermediate output, the layer normalization (LayerNorm) operation is introduced. This normalization process calculates the mean and standard deviation of each sample in units of feature dimensions, and maps them through standardization and learnable scaling and bias parameters to finally obtain a normalized representation. .

[0118] Step C. Time dimension restoration and spatiotemporal alignment:

[0119] Since the attention input adopts a sliding window structure, the output dimension is In order to keep the temporal dimension consistent with spatial attention, the present invention introduces a one-dimensional transposed convolution (i.e. TransposedConv1D ( )) Operation restores the original time length ;

[0120] ;

[0121] In summary, the above time modeling process is executed independently for all nodes in the graph, and the complete time feature tensor can be obtained as follows: This tensor is consistent with the output of the spatial attention module in terms of time axis and channel dimension, providing structurally unified and semantically aligned input for the subsequent spatiotemporal fusion module.

[0122] like Figure 4 As shown, in this embodiment, a kernel function-based spatial attention-message passing module is constructed to model the dynamic spatial dependency between nodes in a transportation network.

[0123] In traditional graph neural networks (GNNs), graph structures are typically predefined and static; however, in dynamic traffic networks, relationships between nodes exhibit significant time-varying properties. To more effectively model spatial dependencies, this paper designs a spatial attention-message passing module that integrates a latent graph structure, multi-head graph attention, and a spatiotemporal masking mechanism. The data processing pipeline is as follows: A latent dynamic graph structure is constructed using node features, and a kernel function approximation mechanism is introduced to achieve efficient spatial message propagation without explicitly calculating the attention matrix, significantly reducing computational complexity. An adaptive dynamic time warping-driven spatiotemporal masking mechanism is then introduced. Combined with external structural constraints such as the dynamic alignment similarity of historical traffic sequences between nodes, geographic adjacency, and road access restrictions, an attention prior is constructed that integrates behavioral consistency and structural accessibility. This guides spatial attention to key node pairs with consistent temporal evolution trends and the possibility of physical connectivity, significantly improving the model's expressiveness and predictive robustness in complex traffic scenarios.

[0124] Specifically, the data processing flow of the spatial attention-message passing module is as follows:

[0125] Step A. Constructing the latent dynamic graph structure:

[0126] The present invention first uses time steps as units to input feature tensors Slice and construct Frame timing diagram, each frame contains Node, The initial node features of the frame graph (i.e., the input of the 0th layer of the graph neural network) are defined as:

[0127] ;

[0128] This paper adopts the idea of message propagation layer by layer. To elaborate on the calculation process of the spatial attention-message passing module, the batch dimension is not considered below, and only the feature interaction relationship between any node pairs in a single frame graph is focused. Layer ( , this embodiment sets ), No. The feature representation of all nodes in the frame graph constitutes a tensor , all node pairs The potential graph connection weights of are expressed as: .in, Consistent with the node set in the spatial feature embedding step, Represents nodes respectively and nodes In the The feature vector of the layer, Representation node To Node The potential graph connection weights of Indicates the target node index to be updated, Represents the candidate source node index. To construct the spatial attention relationship between nodes, this paper introduces three learnable linear mapping matrices , respectively used to construct query vectors , key vector Sum vector , which is calculated as:

[0129] ;

[0130] Based on the above, we further define a full-graph attention network, which estimates the potential interactions between instance nodes and enables corresponding dense connection information transmission. The calculation of its attention weight and feature update process are as follows:

[0131] ;

[0132] Among them, the index Traverse all nodes in the graph, Representation node The corresponding spatial attention key vector. This module does not adopt the traditional multi-head attention structure, but the design retains the core idea of weighted aggregation of adjacent nodes in the graph attention mechanism; at the same time, in order to maintain output consistency with the block-level sparse temporal attention module, the output dimension of this module is set to .

[0133] Step B. Kernel function approximation attention acceleration;

[0134] In order to effectively reduce the traditional attention mechanism in the graph structure To reduce computational complexity, this paper introduces an efficient message passing mechanism based on kernel functions. This mechanism replaces the dot-product form of attention similarity calculation with an approximate kernel function method, thereby achieving scalable approximate attention propagation.

[0135] Specifically, the present invention adopts a positive definite kernel function of the following form: .in, is a kernel function that satisfies symmetry and positive definiteness. represents a low-dimensional kernel feature mapping function, which is constructed based on random Fourier features (RFF) in this embodiment and is used to approximate the above-mentioned kernel inner product operation in Euclidean space to simulate the common translation-invariant kernel function.

[0136] by For the construction example, the corresponding kernel feature mapping function is defined as follows:

[0137] ;

[0138] in, , is the frequency vector obtained by independent sampling from the Gaussian distribution corresponding to the kernel function, which is used to construct the Fourier feature; is a phase offset sampled from a uniform distribution to make the mapping result smoothly distributed within the cosine period. Represents the mapping dimension (the mapping dimension in the above equation Adjustable, in this embodiment, set Through this mapping, the attention mechanism of dot product followed by exponential can be transformed into a linear inner product operation in the vector space, which significantly reduces the computational overhead.

[0139] Step C. Introduce the spatiotemporal masking mechanism driven by adaptive dynamic time warping:

[0140] To enhance the spatial attention-message passing module's ability to perceive the heterogeneity of temporal evolution characteristics in traffic networks, this paper designs a spatiotemporal masking mechanism based on adaptive dynamic time warping (DTW) to improve the model's ability to model node temporal heterogeneity. This mechanism comprehensively considers the dynamic alignment of historical traffic sequences between nodes to measure their temporal consistency. It also incorporates external structural constraints such as geographic connectivity and road regulations to construct a spatiotemporal mask.

[0141] Specifically, for the original tensor Process and extract any two nodes and The original time series characteristics , its DTW distance is defined as: ,in represents the set of legal alignment paths that satisfy boundary conditions, monotonicity and continuity constraints, For nodes In the The features of the time step, For nodes In the Features of the time step.

[0142] In order to consider the spatial accessibility and external constraints between nodes and other structural prior information, the present invention constructs a physical adjacency matrix and the external constraint matrix ;in Representation node and There is a direct geographical connection in the transportation network, otherwise . Indicates that the node pair The connection strength under the influence of external constraints between , such as the attenuation weight caused by human factors such as construction restrictions. Combining the above physical connections, external constraints and DTW similarity, the final node is defined With node The mask coefficients corresponding to the spatiotemporal mask matrix for:

[0143] ;

[0144] In the above formula is the temperature factor, which is used to adjust the influence of DTW similarity on mask weight. The time series distance can be mapped to similarity weight and normalized by time length Control the scale and improve the consistency and stability of similarity measurement at different lengths.

[0145] Furthermore, the present invention uses the mask matrix as the prior weight in the kernel function attention, and combines it with the decomposability of the random Fourier feature kernel mapping in step B to effectively simplify the calculation process; thus, the potential graph connection weight is redefined as the following matrix form:

[0146] ;

[0147] In this embodiment, the decomposability of the kernel mapping avoids the need to explicitly construct a fully connected attention matrix. , while the node's representation in the current layer It is the feature of the previous layer node The constructed query, key and value vectors are normalized after aggregating neighbor information in a weighted manner using kernel functions, thereby reducing the computational complexity from Significantly reduced to , which greatly improves the modeling efficiency and scalability on large-scale graph data.

[0148] The final spatial attention-message passing module outputs all node features in the Lth layer (final layer) as .

[0149] In this embodiment, a weighted fusion layer is constructed to align the features of the outputs of the block-level sparse temporal attention module and the spatial attention-message passing module, and weighted fusion is achieved through learnable weight coefficients to output a unified spatiotemporal feature representation after fusion.

[0150] In order to improve the stability and expressiveness of the fusion process, a layer normalization mechanism and a residual connection structure are further introduced in the fusion layer. That is, the feature results after weighted fusion are normalized and residuals are added to the original input features to enhance feature propagation and optimize training effects.

[0151] Temporal attention mechanism shares the same input tensor as spatial attention mechanism To ensure the consistency of temporal and spatial attention outputs, the temporal attention and spatial attention output dimensions are set to , from the above we can see that the temporal attention output is , the spatial attention output is .

[0152] Specifically, in this embodiment, the data processing flow of the weighted fusion layer is as follows:

[0153] Step A. Feature dimension alignment:

[0154] In order to achieve the effective fusion of temporal modeling output and spatial modeling output, the present invention first needs to map both to the hidden dimension of the unified network structure. Specifically, the present invention designs two independent fully connected layers (i.e., one-dimensional linear transformation) for the time branch and the space branch to adjust the channel dimension. , Represent the fully connected mapping functions acting on the time branch and the space branch respectively, and define the spatiotemporal feature representation after dimension alignment as :

[0155] ;

[0156] Step B. Weighted fusion:

[0157] After completing the dimensional alignment of temporal and spatial features, the present invention adaptively fuses the two by introducing learnable fusion weights to fully integrate temporal and spatial perception information.

[0158] Specifically, the present invention sets a learnable fusion coefficient , the final fused spatiotemporal feature tensor is expressed as:

[0159] ;

[0160] Step C. Normalization and residual connection:

[0161] In order to further improve the expression stability of fusion features and the convergence efficiency of training, after obtaining the fusion feature tensor Finally, the present invention introduces the layer normalization mechanism and residual connection structure. First, the fusion result Perform layer normalization to obtain .

[0162] Then, in order to maintain information continuity and enhance the gradient propagation effect, the spatiotemporal embedding feature tensor Obtain the residual feature tensor through a learnable linear mapping function , the normalized fusion features and residual features are added element by element to realize residual connection, and the final output fusion features are:

[0163] ;

[0164] This structure not only achieves the deep fusion of temporal and spatial perception information, but also improves the expressiveness and training stability of the model through layer normalization and residual connection mechanism, providing more robust spatiotemporal feature representation for downstream modeling tasks.

[0165] In this embodiment, a deep modeling and output module (MoE-FFN-OUT) based on the expert mixture mechanism is constructed to further perform nonlinear modeling and output generation on the fused high-dimensional spatiotemporal features.

[0166] To achieve more expressive and generalizable deep modeling and output prediction of fused high-dimensional spatiotemporal features, this embodiment proposes a comprehensive modeling module that integrates a Mixture of Experts (MoE) mechanism, a fully connected mapping structure, skip connection paths, and an output layer design. This module not only replaces the single feedforward network layer in the traditional Transformer but also enhances the model's ability to model multi-scale spatiotemporal dependencies through dynamic expert selection and skip connection mechanisms, and maps the final deep representation to the output space. The module primarily consists of four components: a MoE dynamic expert modeling module, a fully connected mapping module, a skip connection layer, and an output layer. This overall structure effectively improves the model's expressiveness, stability, and predictive accuracy.

[0167] Specifically, in this embodiment, the data processing flow of the deep modeling and output module based on the expert mixture mechanism is as follows:

[0168] Step A. Introduce the MoE dynamic expert modeling module to implement dynamic expert modeling:

[0169] In this embodiment, the improved feedforward network layer uses a MoE dynamic expert modeling module, which includes multiple independent expert sub-networks and a gating network. The gating network is used to dynamically calculate the weight coefficients of the expert sub-networks based on the input features, thereby adapting to the multi-modal spatiotemporal dependencies in traffic flow and enhancing the model's expression flexibility and generalization ability.

[0170] With (in this embodiment =4) Independent expert sub-network, each expert sub-network Corresponding to a feedforward submodule , define the input data as , which comes from the spatiotemporal fusion feature tensor output by the spatiotemporal encoding module: , for each position in the tensor, all expert subnetworks process the input in parallel to produce an expert output: , where serial number .

[0171] In this embodiment, in order to realize the soft selection mechanism of multiple expert outputs, a gating network is introduced to adaptively generate the expert weight vector of each position according to the input features. , calculated as follows:

[0172] ;

[0173] in, is a learnable parameter of the gating network, and the softmax function ensures that the weights are normalized across all experts to satisfy: Finally, the MoE dynamic expert modeling module performs a weighted combination of all expert sub-network outputs to form the final feedforward output :

[0174] ;

[0175] This mechanism not only improves the nonlinear expression ability of the model, but also has stronger expert selectivity, which helps to capture diverse spatiotemporal dependency structures.

[0176] Step B. Fully connected mapping and skip path establishment:

[0177] In this embodiment, in order to convert the fused spatiotemporal features into the final prediction results, the high-order semantic features output by the MoE dynamic expert modeling module in step A are first fully connected and mapped to obtain preliminary prediction results. The mapping process is as follows:

[0178] ;

[0179] in, represents the fully connected mapping function, is the target variable dimension (consistent with the input). At the same time, in order to retain the direct information in the low-order spatiotemporal features, this embodiment designs a jump connection path, namely the jump connection layer, to combine the spatiotemporal fusion features Directly fed into an independent linear mapping layer; where Representing the linear projection function in the skip connection path, we get:

[0180] ;

[0181] Step C. Output layer design:

[0182] In order to take into account both deep semantic features and initial fusion representation, this embodiment introduces a learnable fusion weight coefficient , used to balance the output of the two paths; the final prediction result The expression is as follows:

[0183] ;

[0184] This design effectively enhances the adaptability and robustness of the model in multi-time prediction scenarios by combining high-order semantic representations with low-order spatiotemporal features, and enhances the model's ability to model traffic flow mutation patterns and long-term evolution trends.

[0185] In this embodiment, in order to verify the effectiveness and generalization ability of the proposed block-based temporal self-attention prediction method, a unified experimental environment is constructed and a scientific and reasonable performance evaluation mechanism is set up.

[0186] Experimental Environment Configuration: Given that traffic prediction tasks involve large-scale graph data and long sequence modeling, experiments were conducted on a high-performance computing platform. Hardware used included an NVIDIA GeForce RTX 4090 graphics card (24GB of GDDR6X video memory), a 12th-generation Intel Core i9-12900 processor (24 threads), and 128GB of DDR4 memory to meet the computational and memory requirements of large-scale graph neural network training. Software used included Ubuntu 18.04 LTS, the PyTorch 2.0.0 deep learning framework, and CUDA version 11.8 to balance operator optimization and system compatibility. Model training used the Adam optimizer with an initial learning rate of 0.001 and a cosine annealing scheduler for dynamic learning rate adjustment. The batch size was set to 256, the maximum number of training epochs was 200, and early stopping (patience = 10) was enabled to monitor the validation set MAE performance and prevent overfitting. All experiments used a fixed random seed (seed=42) and employed five-fold random initialization cross-validation to enhance the stability of the results. In terms of data processing, traffic flow data was Z-score normalized, and the graph structure was constructed using an adjacency matrix generation strategy with a threshold distance of 2 km. Dataset partitioning followed common standards: the PEMS series datasets used a training / validation / testing ratio of 7:2:1, while the Grid-based format datasets used a 7:1:2 split.

[0187] Experimental dataset selection:

[0188] To fully validate the model's generalization capabilities, this paper selected six representative spatiotemporal datasets covering multiple scenarios, including highway traffic flow and urban traffic demand. The structured road network data comes from the PeMSD4 and PeMSD8 datasets released by the Caltrans Performance Measurement System (PeMS), covering the San Francisco Bay Area and the Los Angeles and San Bernardino areas, respectively. These data are collected by approximately 39,000 sensors deployed on major metropolitan freeways in California. Each data point contains three features: traffic flow, average speed, and average occupancy. This paper focuses on traffic flow, and the spatial adjacency matrix for each dataset is constructed based on geographic distances within the actual road network. Urban traffic demand data includes the following representative datasets: the New York City taxi dataset (NYCTaxi) (published by the New York City Taxi and Limousine Commission), the Chicago bike-sharing dataset (CHIBike), which contains ride records from multiple stations, and the Beijing T-Drive taxi trajectory dataset released by Microsoft Research Asia. These datasets constitute an experimental environment for urban traffic prediction, encompassing multiple timescales and travel modes.

[0189] Performance evaluation mechanism design:

[0190] In order to comprehensively evaluate the performance of the proposed model in the traffic flow prediction task, a multi-dimensional performance evaluation system was constructed, using the following statistical indicators: 1) Mean absolute error (MAE): measures the average absolute value of the difference between the predicted value and the true value, used to evaluate the overall level of error; 2) Root mean square error (RMSE): based on MAE, it emphasizes the impact of large prediction deviations, takes the weighted average of the squared errors and then takes the square root, which is suitable for application scenarios that are sensitive to outliers; 3) Mean absolute percentage error (MAPE): quantifies the proportion of prediction error in the true value in percentage form, which facilitates intuitive comparison of the relative prediction errors of the model on data of different magnitudes.

[0191] Baseline model selection:

[0192] To comprehensively evaluate the performance of the proposed model in traffic flow prediction tasks, we compared it with current mainstream deep learning models on multiple standard datasets, covering graph neural networks, sequence modeling structures, and various paradigms based on attention mechanisms. We selected eight representative models for comparison: STGCN, DCRNN, Graph WaveNet, STSGCN, STFGNN, STGODE, Transformer, and GMAN. These models cover different approaches, including static graph modeling, dynamic graph modeling, and spatiotemporal joint modeling.

[0193] Experimental results analysis:

[0194] The experimental results are shown in Tables 1 and 2 below. On the PeMSD4 and PeMSD8 highway datasets, the proposed model achieved optimal performance across all evaluation metrics. For example, on PeMSD4, the proposed model achieved a MAE of 18.89, an RMSE of 29.71, and a MAPE of 12.84%, significantly outperforming Graph WaveNet by approximately 24.1%, 25.1%, and 4.5%, respectively. On PeMSD8, the proposed model also achieved a MAE of 13.89, an RMSE of 23.32, and a MAPE of 9.08%, significantly outperforming strong baseline models such as GMAN, demonstrating excellent modeling capabilities and cross-regional generalization. In complex urban traffic scenarios, the proposed model achieved optimal performance in both in-flow and out-flow directions on the NYCTaxi dataset. In the in-flow direction, the model achieved a MAE of 13.83, an RMSE of 22.72, and a MAPE of 13.60%; in the out-flow direction, it further achieved a MAE of 12.11, an RMSE of 19.12, and a MAPE of 13.43%, comprehensively outperforming models such as GMAN, Transformer, and STFGNN, demonstrating strong adaptability and stability to irregular urban travel patterns.

[0195] Table 1: Performance comparison of different models on PeMSD4 and PeMSD8 datasets

[0196]

[0197] Table 2: Performance comparison of different models on the NYCTaxi dataset

[0198]

[0199] Figure 5The training and validation loss curves of this model in four typical experiments are shown. Figure 5 (a) and (b) correspond to the training loss and validation loss of the PeMSD4 dataset, (c) and (d) to the PeMSD8 dataset, (e) and (f) to the CHIBike data, and (g) and (h) to the NYCTaxi data. Analysis of the training and validation loss curves for four different experiments reveals that the training loss curves for all experiments continue to decrease with increasing iterations, indicating that the model's ability to fit the training data is continuously improving. Furthermore, most validation losses also show a trend of first decreasing and then stabilizing, demonstrating that the model has good generalization performance and convergence across multiple traffic scenarios.

[0200] Figure 6 The figure shows a visualization of the traffic flow prediction results of the proposed model on the PeMSD8 dataset, comparing them with existing SOTA models and real values. The horizontal axis represents the time step, and the vertical axis represents the traffic flow value, covering the entire time series range of the day. The visual comparison of the difference between the model prediction and the actual traffic flow shows that the proposed model closely matches the real traffic flow data curve in the vast majority of time steps, significantly outperforming the existing SOTA model and demonstrating higher prediction accuracy and robustness. To further demonstrate the model's advantages, three representative local areas are specifically selected for zoomed-in analysis in the figure. The results are as follows: During the early morning rush hour fluctuation period (time steps of approximately 20-60): The proposed model has a stronger ability to fit sudden fluctuations in traffic flow and can accurately reflect small, rapid increases and decreases. In comparison, the SOTA model's prediction results show significant lags and errors. During the trough transition phase (time steps of approximately 120-160), the model of the present invention maintains good prediction accuracy in low traffic volume ranges and accurately captures the recovery changes after the trough. The SOTA model exhibits large deviations and insufficient capture of rebound trends during this phase. During the evening peak oscillation phase (time steps of approximately 180-220), the model of the present invention achieves a more refined and stable prediction effect for the rapid fluctuations in traffic flow during the evening peak period, reducing error accumulation. The SOTA model exhibits increased prediction jitter and directional errors at multiple oscillations, and exhibits poor robustness.

[0201] To further verify the applicability and effectiveness of the traffic flow prediction model of the present invention in a real, complex urban road network environment, a city-level heat map was created using the traffic flow prediction results generated by the model of the present invention, in conjunction with the 2008 Beijing taxi trajectory dataset published by Microsoft Corporation in Beijing. In this embodiment, the heat map is constructed by spatially aggregating the traffic flow intensity predicted by the model within a given time window, rasterizing it in combination with geographic location coordinates, and ultimately mapping it onto a geographic base map of Beijing's central urban area. The heat map uses changes in color intensity to represent the density of traffic flow. The heat map generated based on the prediction results of the model of the present invention accurately depicts the peak-hour traffic concentration phenomenon on Beijing's major traffic arteries and hubs (such as Chang'an Avenue, Zhongguancun area, and the traffic corridor between the East Second Ring Road and the East Fourth Ring Road). It is highly consistent with the historical statistics of actual taxi trajectories, demonstrating that the model of the present invention has strong spatial generalization capabilities and the ability to identify regional congestion patterns.

[0202] To more accurately understand the contribution and importance of each core component in this model, we designed a series of ablation experiments. These experiments gradually removed key components from the model to reveal the specific impact of each component on the overall performance of the model. The specific ablation experiment settings and results are as follows:

[0203] The block-based input mechanism was removed, and the traditional multi-head temporal attention structure was compared with this model design. In this ablation experiment, the present invention removed the local temporal series sparse modeling mechanism (i.e., the block-based temporal self-attention mechanism based on dynamic sparse masking) from the model and replaced it with a traditional multi-head temporal attention mechanism. The multi-head temporal attention mechanism performs global attention calculations based on the entire sequence. Although it has certain advantages in modeling long-term dependencies, it ignores local structure, resulting in a large number of parameters and high computational overhead, especially in long sequences or multi-head parallel settings. Experimental results show that in short-term prediction tasks, the MAPE of the traditional mechanism is significantly higher than that of the PTSA module with patch introduction, indicating that it is difficult to effectively capture short-term fluctuations and emergencies in traffic flow. In addition, because this structure is highly dependent on complete, fixed-length input sequences, it has poor adaptability to variable time periods or missing data, and exhibits insufficient generalization ability.

[0204] The dynamic spatial attention mechanism was replaced with traditional graph convolution operations. Specifically, the spatial attention-message passing module based on the latent graph structure and kernel function in the original model was replaced with the traditional static graph convolution operation. After this replacement, node feature updates relied solely on a predefined static adjacency matrix. When the dynamic spatial attention mechanism was replaced with the traditional GCN (graph convolutional neural network), the MAE increased by approximately 18%, and the RMSE and MAPE also increased significantly, indicating that static graph convolutions are unable to effectively capture the time-varying correlations between nodes. Furthermore, because the dynamic spatial attention mechanism enhances the modeling capability of spatial dependency changes by introducing the latent graph structure and kernel function, the fixed graph structure of the traditional GCN limits the model's ability to respond to the dynamics of real traffic networks.

[0205] The DTW-based spatiotemporal masking mechanism was removed. After removing the DTW mask, model performance decreased slightly, with MAE and RMSE increasing by approximately 7.3% and 6.0%, respectively, and MAPE increasing by approximately 0.26%. While the DTW mask contributes slightly less to performance than the latent graph structure and kernelized attention mechanism, as a pluggable spatiotemporal alignment module, it improves the model's ability to represent complex traffic scenarios. While the DTW mask is not a core structure, it supplements temporal structure information in capturing spatiotemporal alignment features. This design is more adaptable to situations such as abnormal traffic flow, holiday impacts, and sudden changes in local traffic.

[0206] The above description of the present invention using specific examples is intended only to facilitate understanding of the present invention and is not intended to limit the present invention. A person skilled in the art of the present invention may make several simple deductions, modifications, or substitutions based on the principles of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A multi-modal spatiotemporal traffic flow modeling method that supports real-time prediction of large-scale road networks, characterized by: The method first designs a spatiotemporal prediction framework and then trains it to obtain the final prediction model; the spatiotemporal prediction framework includes a data embedding layer, a spatiotemporal encoding module, and a deep modeling and output module based on an expert mixture mechanism; The data embedding layer is used to convert the original spatiotemporal data into a high-dimensional dense representation. It includes two parallel channels, time embedding and spectral-spatial embedding, and a spatiotemporal data fusion module. The time embedding channel is used to extract the temporal semantic features of local temporal features and periodic position encoding, while the spectral-spatial channel obtains spatial-frequency domain structural features through graph Laplacian spectrum decomposition. The spatiotemporal data fusion module is used to perform feature fusion. In the traffic flow prediction task, define χ t ∈R N×F represents the traffic flow of N nodes in the road network at each time step t∈{0,1…,T-1}, where F represents the dimension of the traffic flow. The traffic flow at all time slices is combined and recorded as the original tensor χ: x=(x0,x1,…,x T-1 )∈R T×N×F ; The spatiotemporal encoding module is used to perform temporal and spatial modeling on the input tensor obtained by the data embedding layer, including a parallel block-level sparse temporal attention module, a spatial attention-message passing module, and a weighted fusion layer; the block-level sparse temporal attention module adopts a self-attention mechanism based on time blocks and introduces a dynamic mask mechanism, which slices the input data according to nodes, extracts the time series corresponding to each node from the input data, and performs local modeling and scale alignment on the temporal features; the spatial attention-message passing module integrates the potential dynamic graph structure, the dynamic graph attention mechanism of kernel function approximation, and the spatiotemporal mask mechanism driven by adaptive dynamic time warping, which slices the input data according to time steps and constructs a T-frame temporal graph. Each frame contains all nodes. In the lth layer of the graph neural network, the features of the uth and vth nodes are respectively and The mask coefficients corresponding to nodes u and v in the spatiotemporal mask matrix for: in, Indicates that nodes u and v have a direct geographical connection in the transportation network, otherwise it is 0; represents the connection strength under the influence of road control conditions between node pairs (u, v), ξ uv ∈[0,1]; γ is the temperature factor, which is used to adjust the influence of DTW similarity on mask weight. DTW(χ u ,χ v ) represents the original time series feature χ of node u,v u ,χ v The DTW distance, T is defined the same as the input time step length; Using the spatiotemporal mask matrix as a priori weights, the features of node u in the l+1th layer of the graph neural network are updated. The features of node u in the current layer are constructed by the query, key, and value vectors constructed from the features of the previous layer, and the neighbor information is aggregated in a weighted manner using the kernel function. This is used to model the dynamic spatial dependency relationship between nodes in the transportation network. The weighted fusion layer is used to align features and achieve weighted fusion. The deep modeling and output module based on the expert mixture mechanism is used to model the unified spatiotemporal feature tensor and generate prediction outputs, including the MoE dynamic expert modeling module, the fully connected mapping module, the skip connection layer, and the output layer; the MoE dynamic expert modeling module includes multiple independent expert sub-networks and a gating network; the fully connected mapping module is used for fully connected mapping; the skip connection layer is used to transfer the spatiotemporal fusion features to the linear mapping layer; the output layer is used to balance the outputs of the skip connection and fully connected mapping modules to generate the final prediction results.

2. The multi-modal spatiotemporal traffic flow modeling method supporting real-time prediction of large-scale road networks according to claim 1 is characterized in that: The time embedding channel constructs a fine-grained time series representation of traffic flow by fusing local time series features with explicit periodic semantics. The specific data processing flow is as follows: Step A: Perform a uniform dimension projection on the input original tensor χ and use a fully connected layer to map it into a fixed-dimensional embedding space to obtain a standard feature embedding. Step B. Set two learnable embedding dictionaries, namely the week embedding dictionary T week and intraday timestamp embedding dictionary T day , respectively construct the week embedding tensor E of the original tensor χ week , intraday timestamp embedding tensor E dat ; Step C. Concatenate the standard feature embedding, week embedding, and intraday timestamp embedding on the feature dimension. Input the concatenated time embedding into a learnable linear projection layer to obtain the high-order temporal semantic feature Z time ; Step D. Define the position encoding matrix Z pos , by element-by-element addition to fuse the periodic semantics and position structure; finally, the output of the time embedding channel X time =Z time +Z pos .

3. The multi-modal spatiotemporal traffic flow modeling method supporting real-time prediction of large-scale road networks according to claim 1 is characterized in that: The data processing flow of the spectral domain spatial channel is as follows: Step A. constructing a symmetric normalized graph Laplacian matrix based on the self-loop-free adjacency matrix and the correspondence matrix of the traffic network; Step B. Perform eigenvalue decomposition on the graph Laplace matrix, remove the eigenvectors corresponding to zero eigenvalues, and start from the eigenvector corresponding to the second smallest eigenvalue, and select the first d spec feature vectors to construct low-dimensional spectral graph embedding; Step C. Use a learnable linear mapping function to embed the low-dimensional spectrogram into a high-dimensional space to obtain the spatial embedding feature, i.e., the spatial frequency domain structure feature X spec .

4. The multi-modal spatiotemporal traffic flow modeling method supporting real-time prediction of large-scale road networks according to claim 1 is characterized in that: The data processing flow of the spatiotemporal data fusion module is as follows: Step A. Perform element-wise addition operation on the output of the time embedding channel and the output of the spectral domain spatial channel to generate the preliminary spatiotemporal embedding tensor X st ; Step B. Introduce the learnable position encoding matrix E struct , and add it to the preliminary spatiotemporal embedding tensor element by element to obtain the final fused spatiotemporal encoding module input feature tensor X.

5. The multi-modal spatiotemporal traffic flow modeling method supporting real-time prediction of large-scale road networks according to claim 1 is characterized in that: The data processing flow of the block-level sparse temporal attention module is as follows: Step A: Slice at the node level and extract the corresponding time series from the input data for each node. Then, use a sliding window strategy to divide the time series corresponding to the node into multiple time blocks with overlapping areas along the time axis. Step B. Flatten the two-dimensional features in each time block into a one-dimensional vector to obtain a flattened time block sequence; Step C. The flattened time block sequence is input into a learnable linear transformation module to obtain the embedding tensor Step D. Introduce a multi-head self-attention mechanism. For each attention head, based on the obtained embedding tensor Generate query, key, and value vectors through linear mapping of learnable systems; Step E. For each attention head, calculate the attention score of the i-th time block to the j-th time block according to the query vector of the i-th time block and the key vector of the j-th time block, and obtain the attention score matrix; Step F. At each query time block i, only retain the top r attention scores top The key vector position index is used to perform softmax normalization on the reserved index position, and the attention scores of the remaining positions are set to zero to achieve sparse connection and obtain the sparse attention matrix O h (T) ; Step G. Introduce dynamic channel mask mechanism and construct mask matrix Each element of this matrix follows the Bernoulli distribution sampling, which is used to indicate whether to retain the output of the c-th channel in the i-th time block; Step H. Multiply the sparse attention matrix by the dynamic mask matrix element by element to obtain the attention output with suppression effect Step I. Concatenate the outputs of all attention heads along the channel dimension and fuse them through a learnable linear layer to obtain the intermediate output of the block-level sparse temporal attention module. Step J: The intermediate output is normalized by the layer to obtain the final normalized output; Step K. Introduce a one-dimensional transposed convolution operation to restore the final normalized output to its original time length. Then all nodes output tensor O after passing through the block-level sparse time attention module. time .

6. The multi-modal spatiotemporal traffic flow modeling method supporting real-time prediction of large-scale road networks according to claim 1, characterized in that: The data processing flow of the spatial attention-message passing module is as follows: Step A: Slice the input data in time steps and construct a T-frame time series graph, where each frame contains all nodes. In the lth layer of the graph neural network, the features of the uth and vth nodes are and Introduce three learnable linear mapping matrices to construct query vector, key vector and value vector respectively; Step B. Replace the dot product form of attention similarity calculation by the approximate method of the positive kernel function, the positive kernel function κ(q u (S) ,k v (S) )=φ(q y (S) ) T φ(k v (s) ), φ(·) is a low-dimensional kernel feature mapping function, which is constructed based on random Fourier features; q u (S) ,k v (S) Represent the query vector of the u-th node and the key vector of the v-th node respectively, where u represents the index of the currently updated central node and v represents the index of the candidate source node; Step C. Introduce the spatiotemporal masking mechanism driven by adaptive dynamic time warping, and the mask coefficients corresponding to nodes u and v in the spatiotemporal mask matrix are for: in, Indicates that nodes u and v have a direct geographical connection in the transportation network, otherwise it is 0; represents the connection strength under the influence of road control conditions between node pairs (u, v), ξ uv ∈[0,1]; γ is the temperature factor, which is used to adjust the influence of DTW similarity on mask weight. DTW(χ u ,χ v ) represents the original time series feature χ of node u,v u ,χ v The DTW distance, T is defined the same as the input time step length; Step D. Use the spatiotemporal mask matrix as a priori weights to update the features of node u in the l+1th layer of the graph neural network. The features of node u in the current layer are constructed by the query, key, and value vectors constructed from the features of the previous layer, and are obtained by aggregating neighbor information in a weighted manner using the kernel function. The calculation process of its attention weight and feature update is as follows: where index τ represents all nodes used for normalization, Represents the corresponding key vector, v v (S) represents the value vector of node v, is the mask coefficient corresponding to node u and node τ in the spatiotemporal mask matrix.

7. The multi-modal spatiotemporal traffic flow modeling method supporting real-time prediction of large-scale road networks according to claim 1 is characterized in that: The weighted fusion layer introduces a layer normalization mechanism and a residual connection structure, that is, the fused feature tensor is subjected to layer normalization processing, and the residual is added to the initial input feature as the final output.

8. The multi-modal spatiotemporal traffic flow modeling method supporting real-time prediction of large-scale road networks according to claim 2, characterized in that: When the time embedding channel processes data, for each time step t, the corresponding even dimension uses the sine function to calculate the position encoding matrix Z pos , the corresponding odd dimensions use the cosine function to calculate the position encoding matrix Z pos .

Images