Traffic flow prediction method based on mask auto-encoding clustering and adaptive hybrid experts

Abstract

The application provides a traffic flow prediction method based on mask self-encoding clustering and adaptive hybrid experts, and relates to the technical field of traffic flow prediction.The method first extracts deep space-time features of traffic data through a mask self-encoder, and explicitly decomposes nodes into two subsets of large flow and small flow by using K-Means clustering; then a double-channel adaptive hybrid expert network is constructed, a multi-graph dynamic graph convolution network that fuses a mode library and self-similarity is used for the large flow expert to capture complex spatial dependence, a simplified spatial module is used for the small flow expert, and the two experts share a time feature encoder based on a Transformer and time embedding; finally, the prediction results of the two experts are fused and output.The application realizes end-to-end collaborative optimization of data explicit decomposition and prediction tasks, can effectively capture long-term and dynamic spatial correlation of a traffic network, and significantly improves traffic flow prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of traffic flow prediction technology, specifically to a traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert. Background Technology

[0002] In recent years, with the continuous increase in urban motor vehicle ownership and the growing complexity of road networks, traffic flow prediction has become a key foundational task in the construction of intelligent transportation systems and smart cities. Accurate short-term traffic flow prediction can provide core data support for higher-level applications such as adaptive traffic light control, dynamic route guidance, congestion warning, and travel planning, and has significant practical implications for alleviating traffic congestion and improving road network operational efficiency. However, urban transportation systems are essentially complex spatiotemporal dynamic systems composed of a large number of heterogeneous nodes (such as intersections, road segments, and toll stations). Their traffic flow data exhibits significant nonlinearity, nonstationarity, strong spatiotemporal coupling, and highly dynamic evolution, posing a severe challenge to high-precision and robust traffic flow prediction.

[0003] To address these challenges, existing research mainly focuses on three directions. First, prediction methods based on spatiotemporal feature extraction attempt to simultaneously capture the temporal dependence and spatial correlation of traffic data. Early temporal modeling methods were represented by statistical models such as the Autoregressive Integrated Moving Average (ARIMA), but these methods, based on linear and stationary assumptions, struggle to adapt to the strong nonlinear and nonstationary characteristics of traffic flow. Subsequently, machine learning methods such as Support Vector Regression (SVR) and Random Forest (RF) modeled temporal variation patterns through artificial feature engineering, but still struggled to autonomously uncover deep nonlinear correlations within the data. The rise of deep learning has driven the widespread application of Recurrent Neural Networks (RNNs) and their variants, Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRUs). These models effectively mitigate the vanishing gradient problem through gating mechanisms and can capture, to some extent, the periodic patterns and long-term trends in traffic flow. However, these models only demonstrate strong capabilities in the temporal dimension and struggle to effectively model the complex spatial dependencies between traffic network nodes. For spatial feature extraction, methods based on Graph Neural Networks (GNNs) aggregate spatial neighborhood information through predefined graph structures and combine them with recurrent neural networks or convolutional neural networks to learn temporal features. However, these methods heavily rely on static topological graphs and struggle to reflect the dynamic evolution of spatial relationships in real-world traffic networks. To overcome the limitations of static graphs, researchers have further proposed adaptive graph and dynamic graph learning methods. Adaptive graph learning automatically discovers potential spatial relationships from node embeddings using a data-driven approach, while dynamic graph learning attempts to generate topological structures that evolve over time to characterize differentiated spatial relationships across different time periods. Nevertheless, existing methods still have two shortcomings: firstly, adaptive or dynamic graphs typically rely only on local time-period input information to reconstruct the graph structure, making it difficult to capture long-term stable spatial relationships across time periods; secondly, existing message passing mechanisms often employ a uniform spatiotemporal aggregation strategy, treating the heterogeneous spatiotemporal features of nodes in different functional areas such as residential, commercial, and industrial zones as equal, resulting in a mixture of high-frequency fluctuations, periodic patterns, and long-term trends in traffic patterns, making it difficult to reflect the unique traffic evolution patterns formed by differences in point of interest distribution and road type among nodes. In recent years, the Transformer architecture has been introduced into spatiotemporal feature learning, capturing long-term spatiotemporal dependencies through a self-attention mechanism. However, its global computational characteristics make it difficult to focus on local dynamic changes, limiting its application efficiency in real-time prediction scenarios for large-scale road networks.

[0004] Secondly, traffic flow prediction methods based on data decoupling attempt to improve prediction accuracy by separating subsequences with different patterns and modeling them one by one. Early methods mainly relied on classic time series decomposition techniques such as seasonal decomposition and wavelet decomposition. However, these methods are based on linear assumptions and fixed basis functions, and their decomposition effect is limited when facing nonlinear and non-stationary scenarios such as sudden traffic flow, extreme weather interference, and holiday anomalies. To overcome this limitation, researchers have introduced Empirical Mode Decomposition (EMD) and its improved algorithms, as well as Variational Mode Decomposition (VMD) and robust seasonal trend decomposition methods. However, the above-mentioned mode decomposition methods only decompose single-node time series, completely ignoring the spatial relationships between nodes, resulting in a significant lack of spatial awareness. Although adaptive decoupling frameworks such as Spatiotemporal-Decoupled Interactive Learning (STDIL) and Decoupled Graph Spatial-Temporal Transformer Networks (DSTTN) have emerged, these models still generally lack explicit decoupling mechanisms. That is, there is an architectural disconnect between the decoupling module and the downstream prediction task, making it difficult for decoupling and prediction to work together, resulting in semantic transfer loss.

[0005] Third, traffic flow prediction methods based on expert systems have evolved from traditional rule-driven expert systems to data-driven Mixture of Experts (MoE) systems. Early traditional expert systems were centered on symbolic reasoning and rule-driven approaches, using fuzzy logic reasoning and expert experience to characterize traffic patterns. However, model performance was highly dependent on manually constructed rule bases, lacking autonomous learning and adaptive adjustment capabilities, and difficult to deeply integrate with deep learning models to achieve end-to-end modeling. In recent years, hybrid expert systems have been introduced into traffic flow prediction research. Through divide-and-conquer, expert specialization, and gating fusion, complex spatiotemporal sequences are decomposed into multiple sub-tasks, modeled separately by different expert models, and then the results are dynamically weighted and fused through gating mechanisms. Based on the homogeneity of expert structures, hybrid expert systems can be divided into homogeneous hybrid expert systems and heterogeneous hybrid expert systems. The former uses the same model structure to handle different sub-tasks, while the latter relies on the complementary advantages of different modeling structures to overcome the limitations of a single model. Nevertheless, most existing research on hybrid expert systems has not explicitly separated traffic flow data at the end-to-end level and organically integrated it with the hybrid expert system modeling. Although some studies use adaptive routing to select different experts, they still directly input the full traffic flow data into the hybrid expert system and rely on gating mechanisms for fusion. They do not pre-decompose the traffic flow based on the differences in spatiotemporal characteristics, and then adaptively assign the traffic flow of different modes after decomposition to appropriate experts to achieve more refined predictive modeling.

[0006] In summary, although existing methods have made significant progress in extracting complex spatiotemporal features, they still generally face three core problems: First, traditional data decoupling methods have rigid decomposition mechanisms, relying on signal decomposition techniques based on fixed mathematical principles. They cannot adaptively adjust according to the inherent semantics of the data, and the decomposition process completely ignores the spatial relationships between nodes. Furthermore, the two-stage pipeline architecture of decomposition followed by prediction leads to inconsistencies between the decomposition and prediction objectives. Second, most existing graph convolutional networks rely on static adjacency matrices, making it difficult to adapt to the dynamically changing spatial relationships in traffic flow networks. While adaptive graph or dynamic graph methods can model time-varying relationships to some extent, they cannot simultaneously capture long-term stable spatial relationships. Moreover, the unified message passing mechanism treats heterogeneous spatiotemporal features equally, failing to reflect the unique traffic patterns of nodes in different regions. Third, existing hybrid expert systems fail to achieve end-to-end adaptive fusion of explicit decoupling of traffic flow data and multi-expert modeling, limiting the model's adaptive modeling capabilities in complex scenarios.

[0007] In view of the above, this application is hereby submitted. Summary of the Invention

[0008] This invention provides a traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert, which can at least partially improve the above-mentioned problems.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] A traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert, comprising: The original data is acquired, and the original data is randomly masked to obtain masked data. The masked data is then subjected to spatiotemporal extraction processing using a preset encoder to obtain a low-dimensional feature representation. The encoder consists of a CNN layer and a GRU layer. Based on low-dimensional feature representation, the original data is reconstructed using a decoder to obtain reconstructed data. According to the loss function, the reconstructed data and the original data, a low-dimensional feature representation that retains the original information to the greatest extent is obtained. K-Means clustering is then performed on the low-dimensional feature representation that retains the original information to the greatest extent to obtain clusters with large means and clusters with small means. A high-flow expert network is used to process clusters with large means to obtain a high-flow output feature representation, and a low-flow expert network is used to process clusters with small means to obtain a low-flow output feature representation. Feature extraction is performed on the high-flow output feature representation and the low-flow output feature representation respectively to generate high-flow expert prediction results and low-flow expert prediction results. The high-flow expert prediction results and low-flow expert prediction results are then spliced ​​and fused to obtain the final prediction result.

[0011] In summary, this method is mainly divided into two sub-modules: a clustering module and a prediction module. The prediction module further includes a spatial feature extraction module and a temporal feature extraction module. First, the original data is randomly masked to obtain masked data. This masked data is then input into the encoder of the autoencoder to extract spatiotemporal features. The extracted features are then fed into K-Means clustering to divide the original data into two subsets (i.e., a cluster with a large mean and a cluster with a small mean), completing explicit decomposition—the core function of the clustering module. The decoder of the autoencoder is responsible for reverse mapping of the features. After reconstructing the data, loss calculation is performed using the original and reconstructed data to further improve the quality of feature representation and ultimately enhance the K-Means clustering effect. Subsequently, two subsets are introduced, and appropriate expert networks are designed to model the different spatiotemporal features reflected by these two subsets. Both expert networks include a Diffusion Graph Convolutional Network (DGCN), temporal embedding, and a Transformer network, differing only in their internal hyperparameter settings and structural adjustments to certain modules.

[0012] Compared with existing technologies, the advantages of this method are as follows: by fusing masked autoencoders and K-Means clustering, it achieves end-to-end collaborative optimization of explicit decomposition and prediction tasks of traffic flow data, overcoming the shortcomings of traditional decomposition methods such as lack of spatial awareness and task disconnection; at the same time, it constructs a multi-graph fusion dynamic graph convolutional network based on a pattern library to capture long-term and dynamic spatial correlations, solving the problem that existing graph models cannot distinguish heterogeneous features; and it designs a dual-channel adaptive hybrid expert network to differentiate the spatiotemporal patterns of large and small flow nodes. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert provided in an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of the traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert provided in an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0016] refer to Figure 1 , Figure 2 As shown, the first embodiment of the present invention discloses a traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert, which can be executed by a traffic flow prediction device based on masked autoencoder clustering and adaptive hybrid expert (hereinafter referred to as prediction device), specifically, by one or more processors within the prediction device, to implement the following method: S1. Obtain the original data, perform random masking on the original data to obtain masked data, and use a preset encoder to perform spatiotemporal extraction processing on the masked data to obtain a low-dimensional feature representation. The encoder consists of a CNN layer and a GRU layer. The CNN consists of a series of convolutional layers, which are used to extract the spatial features of the masked data and reduce the time step to achieve dimensionality reduction. Specifically, step S1 further includes: obtaining raw data X According to the original data X Generate a data that is consistent with the original data. X A random tensor of the same shape is used to generate a boolean mask based on the random tensor and a preset mask ratio mask_ratio; Replace the positions in the original data X that are masked as False with NaN, and create a new data set with a random mask. ， The formula is: , For the set of real numbers, T The total number of historical time steps. N The number of road network nodes. d For each node's feature dimensions, This indicates that the original data should be preserved at this location and should not be masked. This process not only helps train the model to be robust to incomplete data, but also reduces computational cost.

[0017] The masked data is processed using the CNN layer of the encoder. Extraction processing is performed to obtain spatial features. Its formula is: , The final dimension after dimensionality reduction via the CNN layers. for ReLU Activation function ， For batch normalization, This is a one-dimensional convolution operation; specifically, each convolutional layer uses a different kernel size and stride to extract features, and batch normalization layers are used to accelerate training and stabilize the model. Finally, ... ReLU The activation function introduces nonlinearity, transforming it into spatial features.

[0018] Spatial features are processed using the GRU layer of the encoder. Temporal feature extraction is performed to obtain low-dimensional feature representations. The specific steps are as follows: spatial features Update the door by updating and resetting the door control information flow. The formula is: Reset the door The formula is: , Among them, the update door This determines how much past information needs to be retained and resets the gate. It determines how much of the past information needs to be forgotten. for Sigmoid Activation function To update the gate weight matrix, For time steps t -1 is a hidden state. For raw data X At time step t Input data, To reset the weight matrix of the gate; Adjusting the time step using the reset door t -1 hidden state Compared with the original data X At time step t Input data The combined information is used to generate a time step. t Candidate hidden state , The hyperbolic tangent activation function is used. Let be the weight matrix of the candidate hidden states. For element-wise multiplication; Using update gates in combination with time steps t -1 hidden state With time step t Candidate hidden state , obtain time step t Hidden state In this way, GRU can effectively capture temporal dependencies in sequence data while mitigating long-term dependency issues.

[0019] By concatenating the dimensions of the hidden states at each time step, a low-dimensional feature representation is obtained. .

[0020] In this embodiment, the autoencoder is an unsupervised learning model based on neural networks, whose core structure includes an encoder and a decoder. The encoder compresses the input data into a low-dimensional representation, while the decoder attempts to reconstruct the input data from the low-dimensional representation. The encoder part of the autoencoder adopts a combined structure of 1D CNN and GRU. The 1D CNN extracts features and reduces dimensionality of the input data through convolution operations, effectively capturing local features and the local structure of the time series. GRU can handle long-short-term dependencies in time series data, further extracting features in the time dimension. This combined structure fully utilizes the local feature extraction capability of CNN and the time series modeling capability of GRU, and can better handle the complex structure of time series data. The decoder then gradually maps the low-dimensional representation back to the original data space through a multi-layer fully connected network, ultimately achieving the reconstruction of the input data.

[0021] The primary reason for using autoencoders is their ability to automatically learn low-dimensional feature representations of data without requiring manual data labeling, making them suitable for feature extraction and dimensionality reduction of unsupervised data. Furthermore, autoencoders optimize model parameters by reconstructing errors, effectively capturing important features in the data while removing noise and redundant information. In this invention, combining autoencoders with a masking mechanism further enhances the model's robustness to data and its feature learning capabilities.

[0022] S2, based on low-dimensional feature representation, uses a decoder to reconstruct the original data to obtain reconstructed data. According to the loss function, the reconstructed data and the original data, a low-dimensional feature representation that retains the original information to the greatest extent is obtained. K-Means clustering is then performed on the low-dimensional feature representation that retains the original information to the greatest extent to obtain clusters with large means and clusters with small means. Specifically, step S2 further includes: based on low-dimensional feature representation Use a decoder to process the raw data X Reconstruct the data to obtain the reconstructed data. ; Use the loss function to calculate the reconstructed data. Compared with the original data X The differences between them are used to update the encoder parameters so that, during computation, the low-dimensional feature representation can retain the original information to the greatest extent, where the loss function... The calculation formula is: , Mean absolute error; Low-dimensional feature representation that retains the original information to the greatest extent possible The clustering objective is calculated using the data as input to K-Means clustering, and the formula for the clustering objective is: , N For the sample size, K The number of clusters. This is an indicator function (which is a Boolean judgment function that outputs 0 or 1, mainly to determine whether the current sample belongs to the current cluster; if it belongs, it outputs 1, and if it does not, it outputs 0). For the first i Cluster labels for each sample, For the first i Data points of each sample, For the first j The cluster center of the nth cluster. Only when the nth cluster... i The summation is only counted when the cluster label of a sample matches the label of the current j.

[0023] Low-dimensional feature representation that retains the original information to the greatest extent possible Clustering is performed using the data as input to K-Means, specifically as follows: Multiple initial cluster centers are randomly selected. For each data point, it is assigned to the nearest cluster based on its distance from all cluster centers until all data points have been assigned. Then, the center of each cluster is recalculated. Repeat the above steps until the change in cluster centers is less than the preset value or the preset number of iterations is reached; this means that the clustering process has converged, thus completing the clustering of traffic flow data.

[0024] Considering that in practical applications, the core differences in traffic flow data mainly lie in the binary distribution characteristics of high-flow and low-flow nodes, and that these two types of nodes exhibit significantly different evolutionary patterns in spatiotemporal dynamics, the nodes are divided into two clusters—high-flow and low-flow—while also taking into account the contour coefficient method. The intra-cluster mean of each cluster is calculated and compared, and the data is adaptively assigned to different experts for spatiotemporal feature extraction of specific tasks. The cluster with the largest mean is defined as... Clusters representing high-traffic nodes are handled by high-traffic experts; clusters with low mean are defined as... This represents a cluster of low-traffic nodes, which should be handled by low-traffic specialists. , For high-traffic nodes, This refers to the number of nodes with low traffic.

[0025] In this embodiment, K-Means is a classic clustering algorithm whose core idea is to divide the data into a predefined number of clusters. K The algorithm minimizes the sum of squared distances within clusters by iteratively optimizing the allocation of cluster centers and data points. It is simple, efficient, easy to implement, and has low computational complexity, making it suitable for clustering tasks on large-scale datasets. K-Means is used for traffic flow spatiotemporal feature clustering because it can quickly divide the low-dimensional feature vectors extracted by the autoencoder into different clusters, thus achieving unsupervised classification of the data. Clustering can uncover natural grouping structures in the data, providing valuable insights for subsequent analysis. Furthermore, the scalability and efficiency of K-Means allow it to adapt to datasets of different sizes, meeting the requirements for clustering speed and effectiveness in practical applications. In this invention, based on the K-Means method, explicit decomposition of the entire traffic flow data is achieved through clustering.

[0026] S3. A high-flow expert network is used to process clusters with large means to obtain high-flow output feature representations, and a low-flow expert network is used to process clusters with small means to obtain low-flow output feature representations. Specifically, step S3 further includes: processing clusters with large means through a fully connected layer. A high-dimensional hidden representation is obtained by performing a nonlinear mapping. , , For the process FC Dimensional changes, It is a fully connected layer; Hiding the high dimension As query vectors, they are respectively compared with the preset traffic pattern database. It performs spatial similarity calculations on itself to measure the correlation between nodes, generating two dynamic adjacency matrices. and Its formula is: , , It is an exponential function. For row index, For column indexes, For transpose, For the first The high-dimensional hidden representation corresponding to the row, For the first Transpose of the traffic pattern database data corresponding to the column. For the first The transpose of the high-dimensional hidden representation corresponding to the column; The two have similar structures, the difference being that the former calculates... The relevance to the pattern library, the latter being calculated Similarity to itself. Although the pattern library and While there is no direct correlation, through training, the pattern library can adaptively adjust and store the unique traffic patterns generated by each node due to spatial heterogeneity, thus effectively overcoming the limitations of reconstructing dynamic graphs solely based on local periodic inputs. Therefore, It can capture potential long-term stable spatial correlations between nodes, and The spatial correlation of short-term fluctuations between nodes within the current time period is captured by the inner product of features.

[0027] Dynamic adjacency matrix and By splicing and merging with a fully connected layer, the final dynamic adjacency matrix is ​​obtained. , , For splicing operations; With the final dynamic adjacency matrix As supplementary information, a diffusion-based GCN graph convolutional network is used to capture dynamic spatial correlations and generate high-throughput output feature representations. Its formula is: , For high-volume time steps, For GCN graph convolutional networks, d For diffusion steps, o Index of diffusion steps, Weights for high-volume output feature representation. b This is the bias value for the high-volume output feature representation. This GCN treats the dynamic changes in the traffic network as a diffusion process, and realizes feature updates by aggregating the diffusion information of nodes. The representation of the target node is jointly determined by the diffusion information of its surrounding nodes.

[0028] Clusters with small mean are processed through a fully connected layer. A nonlinear mapping is performed to obtain a low-dimensional hidden representation. , ; Hidden low-dimensional representation As query vectors, they are respectively compared with the preset traffic pattern database. It also performs spatial similarity calculations on itself to measure the correlation between nodes, generating a dynamic adjacency matrix. Its formula is: , For the first The low-dimensional hidden representation corresponding to the row, For the first The transpose of the low-dimensional hidden representation corresponding to the column; With dynamic adjacency matrix As supplementary information, a diffusion-based GCN graph convolutional network is used to capture dynamic spatial correlations and generate low-flow output feature representations. Its formula is: , For small flow time steps, Weights for outputting feature representations for low flow rates. The bias value is used to represent the characteristics of small flow output.

[0029] In this embodiment, after dividing the dataset into two clusters using K-Means, a unified modeling approach would struggle to effectively capture the inherent characteristics of both traffic patterns simultaneously. Furthermore, after clustering, different experts can learn specialized feature representations and prediction strategies for specific traffic patterns. Specifically, experts specializing in high-traffic patterns focus on complex spatial dependencies such as congestion propagation, while experts specializing in low-traffic patterns excel at capturing random fluctuations and periodic trends, effectively avoiding the problem of insufficient fitting of a single model in complex spatiotemporal scenarios. Therefore, a hybrid expert model is used instead of a unified modeling method, integrating it with self-supervised clustering based on spatiotemporal features for traffic flow prediction.

[0030] When constructing a hybrid expert model, the core difference between high and low traffic flow patterns lies primarily in the spatial dimension. In high-traffic scenarios, the traffic network exhibits significant congestion propagation and strong coupling between road segments, requiring deeper dynamic graph convolutional networks to effectively capture complex spatial dependencies. Conversely, in low-traffic scenarios, the mutual influence between road segments is weaker, and spatial relationships are relatively simple; a lightweight spatial feature extraction module can ensure the effectiveness of spatial feature extraction. Secondly, the temporal dependence of traffic flow has strong universality. Regardless of whether the traffic flow is high or low, traffic data exhibits similar daily periodicity, weekly periodicity, and overall trends. These temporal features can be effectively captured through a unified architecture such as the shared Transformer, eliminating the need for separate design for different traffic flow patterns. Therefore, the main difference between the two expert systems proposed in this invention lies in the spatial feature extraction part.

[0031] Specifically, for high-traffic experts, a weight-shared DGCN module is used as the spatial feature extraction module to dynamically capture spatial correlations. DGCN primarily implements two functions: dynamic graph construction and spatial correlation modeling. To achieve these functions, DGCN includes an adaptive graph generator and a graph convolutional network based on a diffusion mechanism. The adaptive graph generator uses a data-driven approach to construct the dynamic graph, adaptively adjusting the graph structure according to the traffic patterns of each node and the dynamic relationships between nodes. Specifically, the adaptive graph generator defines a trainable traffic pattern library, which adaptively stores the specific traffic patterns generated by each node during training due to spatial heterogeneity. In the adaptive graph generator, the traffic pattern library and hidden representations are used as input to construct a fused adjacency matrix. The fused adjacency matrix can dynamically evolve over time to characterize the dynamic relationships between nodes caused by changes in traffic states, enabling the reconstructed graph to accurately reflect the dynamic spatial correlations of the traffic network.

[0032] For low-traffic expert networks, the spatial extraction module is basically the same as that for high-traffic expert networks, with the main difference being a simplified method for generating the adaptive graph. Considering the weak short-term fluctuations and relatively simple spatial relationships in low-traffic scenarios, low-traffic expert networks only retain... Related to the pattern library, removed Calculations related to itself.

[0033] S4 performs feature extraction processing on the high-flow output feature representation and the low-flow output feature representation respectively, generates high-flow expert prediction results and low-flow expert prediction results, and splices and merges the high-flow expert prediction results and low-flow expert prediction results to obtain the final prediction result.

[0034] Specifically, step S4 further includes: In order to effectively capture the inherent periodic patterns in traffic flow data, two common time embedding methods are introduced, namely daily embedding within a week. and timestamp embedding throughout the day , Indicates the number of days in a week. This indicates the total number of timestamps per day, spaced 5 minutes apart. For the embedded dimension; Embedded within a week and timestamp embedding throughout the day Perform concatenation and broadcasting to generate a first periodic embedding that matches the traffic time series dimension. and ; For the first periodic embedding , and high flow rate output characteristics By splicing the data, a representation containing rich spatiotemporal information is constructed. , ; Using linear transformation, Z Mapped to query matrices respectively Query Key matrix Key Sum matrix Value Its formula is ： , , , To query the weight matrix of the matrix, The weight matrix is ​​the key matrix. The weight matrix is ​​the value matrix; According to the first query matrix Query First key matrix Ke y and the first value matrix Value The first attention score is calculated using the following formula: , for softmax Activation function Let be the dimension of the key matrix. It is a scaling factor used to prevent the dot product result from becoming too large. softmax The function exhibits the gradient vanishing problem; To further improve the model's ability to capture multi-view temporal dependencies, a multi-head attention mechanism is employed. O Each independent self-attention calculation is performed, and the results are concatenated and fused using linear projection to obtain a high-volume fusion result. The formula is as follows: , For concatenation functions, This is the first self-attention output. For the firstO Self-attention output, This is the weight matrix used to fuse multi-head outputs back to the original dimension; The high-volume fusion results are input into the Transformer time feature extraction module to obtain high-volume expert prediction results.

[0035] Embedded within a week and timestamp embedding throughout the day Perform concatenation and broadcasting to generate a second periodic embedding that matches the traffic time series dimension. and ; For the second periodic embedding , and small flow output characteristic representation By splicing the data, a representation containing rich spatiotemporal information is constructed. , ; Using linear transformation, Mapped to the second query matrix respectively Second bond matrix Second-value matrix Its formula is: , , ; According to the second query matrix Second bond matrix Second-value matrix The second attention score is calculated using the following formula: ; Employ a multi-head attention mechanism to execute O Each independent self-attention calculation is performed, and the results are concatenated and fused using linear projection to obtain the low-flow fusion result, the formula of which is: ; Input the small flow fusion result into the Transformer time feature extraction module to obtain the small flow expert prediction result; The prediction results from high-volume and low-volume experts are combined and merged to obtain the final prediction result. .

[0036] In this embodiment, the large and small flow expert networks adopt the same architecture for time feature extraction, and the large flow expert network is mainly used as an example for detailed explanation.

[0037] Furthermore, DGCN is a graph convolutional network that incorporates a diffusion process, aiming to enhance the feature extraction capabilities of graph-structured data by simulating the diffusion of information within a graph. The core of DGCN lies in its diffusion mechanism, which iteratively aggregates information from neighboring nodes, simulating the propagation of information in the graph. This allows it to capture a broader range of inter-node relationships, not just directly connected neighbors. This diffusion mechanism enables DGCN to better handle long-distance dependencies in graphs while maintaining sensitivity to local structures. In this invention, DGCN can effectively process graph data with complex topologies and dynamic changes. Through the diffusion process, it effectively integrates local and global information of nodes, generating more representative node feature representations, thus providing richer feature information for downstream tasks. Moreover, the flexibility and efficiency of DGCN allow it to adapt to graph data of varying sizes and complexities, making it suitable for a variety of graph-related tasks.

[0038] The Transformer is a deep learning architecture based on self-attention mechanisms. Originally used for natural language processing tasks, it has been widely applied in various fields due to its powerful feature extraction capabilities and parallel computation efficiency. Its core is a multi-head self-attention mechanism, which can simultaneously capture long-range dependencies and local features in input data from multiple perspectives without relying on traditional recurrent structures. The parallelization of the Transformer gives it a significant efficiency advantage when processing large-scale data, while its encoder-decoder architecture can flexibly handle sequence-to-sequence tasks. In this invention, the Transformer is used to process time-series data, capturing complex dependencies in the temporal dimension through self-attention, while combining it with DGCN to process graph structure information in the spatial dimension, thereby achieving efficient modeling of spatiotemporal data. This combination enables the model to handle complex interactions in both time and space dimensions, improving the predictive performance of traffic flow.

[0039] Specifically, in this embodiment, to further verify the effectiveness of the proposed method, four datasets—PEMS03, PEMS04, PEMS07, and PEMS08—were selected as the most widely used public datasets in the field of traffic prediction. These datasets all originate from the California Department of Transportation's PeMS system and contain traffic data collected by loop detectors on highways in different regions. All data are processed at a uniform 5-minute time granularity, recording three core indicators: flow rate, speed, and occupancy. PEMS03, from the Los Angeles area, contains 358 sensors and spans from September 1, 2018 to November 30, 2018, primarily covering core urban highway sections. PEMS04, from the San Francisco Bay Area, contains 307 sensors and spans from January 1, 2018 to February 28, 2018, mainly covering various road sections such as cross-sea bridges and urban expressways. PEMS07 covers a large road network in California's 7th Region, containing 883 sensors and spanning from May 1, 2017 to August 31, 2017. It is the largest and most complex dataset. PEMS08 comes from the San Bernardino area, containing 170 sensors and spanning from July 1, 2016 to August 31, 2016. Traffic flow in this area exhibits obvious tidal characteristics. Detailed statistical information for these datasets is shown in Table 1.

[0040] Table 1 provides detailed information about the dataset:

[0041] The experimental results used three evaluation metrics: Mean Square Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE). The specific definitions of these metrics are shown in the formulas below. , , As shown, N For observation samples ， For the first The prediction results for each element, For the first The true values ​​of each element. The smaller the values ​​of MAE, RMSE, and MAPE, the better the model's performance.

[0042] In this embodiment, 14 baseline models are introduced to verify the effectiveness of the proposed Masked Autoencoder Clustering and Adaptive Mixture of Experts (MAC-AMoE) model, which can be divided into three categories: (1) Traditional deep learning models, including Historical Average (HA) and ARIMA. (2) Graph Neural Network-based models. This includes Diffusion Convolutional Recurrent Neural Network (DCRNN), Spatio-Temporal Graph Convolutional Network (STGCN), Attention-Based Spatio-Temporal Graph Convolutional Network (ASTGCN), Spatio-Temporal Synchronous Graph Convolutional Network (STSGCN), Multivariate Time-series Graph Neural Network (MTGNN), Graph Dynamic Convolutional Network (GDGCN), GraphWaveNet (GWN), and Multi-Scale Temporal Graph Convolutional Network (MSTGCN). (3) Transformer-based joint model. These include Graph Multi-Attention Network (GMAN), Spatial-Temporal Transformer Network (STNN), Periodic Dynamic Transformer (PDformer), and Spatial-Temporal Adaptive Embedding Transformer (STAEformer).

[0043] The comparison results between this method and other models are shown in Table 2. It can be seen that MAC-AMoE achieves the best performance on most metrics of the PEMS04, PEMS07, and PEMS08 datasets. Although it does not achieve the best performance on PEMS03, it still shows excellent prediction results overall.

[0044] Table 2 Experimental Results:

[0045] In summary, compared with the prior art, the present invention has the following advantages: 1. To address the inherent shortcomings of traditional decoupling methods, this paper proposes an explicit traffic flow decomposition scheme that uses masked self-supervised feature learning, autoencoders, and K-Means clustering to perform masked autoencoder clustering on spatiotemporal features. This method introduces a random masking mechanism, enabling MAC-AMoE to extract robust deep representations from raw traffic flow data. Subsequently, an encoder composed of CNN and GRU is used to extract spatial and temporal features of the data, effectively solving the problem of missing spatial awareness in the original decoupling method. Finally, by embedding K-Means clustering into the training process, an explicit traffic flow data decomposition method that co-optimizes with the prediction objective is constructed.

[0046] 2. A local and global spatial feature extraction scheme based on pattern library-based multi-graph fusion is proposed. Addressing the shortcomings of existing GCNs (Graph Convergence Networks) that rely on static adjacency matrices and cannot adapt to the dynamic evolution of traffic flow, as well as the topological abrupt changes caused by temporal fragmentation and the heterogeneous spatiotemporal feature mixture due to multi-scale homogenization, this invention proposes a local and global spatial feature extraction method based on pattern library-based multi-graph fusion. This method mines deep spatiotemporal dependencies using DGCN based on a pattern library and channel attention constraints without relying on a fixed prior topology. Specifically, the pattern library adaptively stores long-term traffic patterns formed by each node due to spatial heterogeneity (such as functional differences between residential and commercial areas), overcoming the temporal fragmentation defects caused by the hourly or daily graph construction of existing dynamic graphs. Furthermore, this method fuses the pattern library associations with the adjacency matrix of dynamic associations calculated based on input self-similarity, realizing the modeling of local and global spatial features of inherent node patterns and real-time traffic states, significantly enhancing the model's ability to extract spatial features from complex traffic networks.

[0047] 3. A dual-channel adaptive hybrid expert network based on Transformer and DGCN was constructed. Addressing the problem that a single model architecture forces all nodes to share parameters and cannot adapt to spatial heterogeneity, this invention designs a dual-channel hybrid expert network. This network can adaptively allocate expert networks for large and small traffic flows based on the traffic volume scale, achieving a seamless process from explicit decomposition of raw traffic flow data to prediction. This effectively solves the problem of separating traffic flow decomposition from prediction objectives, providing a new modeling approach for dynamic modeling of complex traffic systems. For temporal feature extraction, both expert networks uniformly employ Transformer and temporal embedding methods to ensure consistent capture of periodic and trend-based temporal features. For spatial feature extraction, a dual-expert parallel network is designed. For high-volume traffic patterns, two adaptive adjacency matrices based on pattern library correlation and self-similarity are combined to model strong spatial coupling and long-range dependencies; for low-volume traffic patterns, long-term dependencies are learned solely based on the dynamic adjacency matrix of the pattern library. This spatiotemporal collaborative modeling method of "unified temporal feature extraction and differentiated spatial feature extraction" enables differentiated processing of spatial patterns at different time scales, effectively improving the model's accuracy in predicting large-scale network-level traffic flows. Finally, by integrating masked autoencoder clustering and adaptive hybrid experts, an innovative end-to-end traffic flow prediction method with explicit decomposition and predictive co-optimization was constructed.

[0048] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert, characterized in that, include: The original data is acquired, and the original data is randomly masked to obtain masked data. The masked data is then subjected to spatiotemporal extraction processing using a preset encoder to obtain a low-dimensional feature representation. The encoder consists of a CNN layer and a GRU layer. Based on low-dimensional feature representation, the original data is reconstructed using a decoder to obtain reconstructed data. According to the loss function, the reconstructed data and the original data, a low-dimensional feature representation that retains the original information to the greatest extent is obtained. K-Means clustering is then performed on the low-dimensional feature representation that retains the original information to the greatest extent to obtain clusters with large means and clusters with small means. A high-flow expert network is used to process clusters with large means to obtain a high-flow output feature representation, and a low-flow expert network is used to process clusters with small means to obtain a low-flow output feature representation. Feature extraction is performed on the feature representations of high-volume output and low-volume output respectively to generate expert prediction results for high-volume output and low-volume output. The expert prediction results for high-volume output and low-volume output are then spliced ​​and fused to obtain the final prediction result. The high-traffic expert network employs a weight-shared DGCN module as a spatial feature extraction module. The DGCN includes an adaptive graph generator and a graph convolutional network based on a diffusion mechanism. The adaptive graph generator uses a data-driven approach to construct a dynamic graph, adaptively adjusting the graph structure according to the traffic patterns of each node and the dynamic relationships between nodes. The adaptive graph generator defines a trainable traffic pattern library, which can adaptively store the specific traffic patterns generated by each node during training due to spatial heterogeneity. In the adaptive graph generator, the traffic pattern library and hidden representation are used as inputs to construct a fused adjacency matrix. The fused adjacency matrix can dynamically evolve over time to characterize the dynamic relationships between nodes caused by changes in traffic state, so that the reconstructed graph can accurately reflect the dynamic spatial correlation of the traffic network. The adaptive graph generator of the low-flow expert network generates a dynamic adjacency matrix based solely on low-dimensional hidden representations and a pattern library.

2. The traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert as described in claim 1, characterized in that, Obtain the original data, and then perform random masking on the original data to obtain the masked data, specifically as follows: Obtain raw data X According to the original data X Generate a data that is consistent with the original data. X A random tensor of the same shape is used to generate a boolean mask based on the random tensor and a preset mask ratio mask_ratio; Original data X The positions where the mask is False are replaced with NaN, and a new data set with a random mask is created. Its formula is: , For the set of real numbers, T The total number of historical time steps. N The number of road network nodes. d For each node's feature dimensions, This indicates that the original data should be preserved at this location and does not need to be masked.

3. The traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert as described in claim 2, characterized in that, The masked data is spatiotemporally extracted using a pre-defined encoder to obtain a low-dimensional feature representation, specifically: The masked data is processed using the CNN layer of the encoder. Extraction processing is performed to obtain spatial features. Its formula is: , The final dimension after dimensionality reduction via the CNN layers. for ReLU Activation function For batch normalization, This is a one-dimensional convolution operation; Spatial features are processed using the GRU layer of the encoder. Temporal feature extraction is performed to obtain low-dimensional feature representations. The specific steps are as follows: spatial features Update the door by updating and resetting the door control information flow. The formula is: Reset the door The formula is: , Among them, the update door This determines how much past information needs to be retained and resets the gate. It determines how much of the past information needs to be forgotten. for Sigmoid Activation function To update the gate weight matrix, For time step t -1 is a hidden state. For raw data At time step t Input data, To reset the weight matrix of the gate; Adjusting the time step using the reset door t -1 hidden state Compared with the original data X At time step t Input data The combined information is used to generate a time step. t candidate hidden state state , The hyperbolic tangent activation function is used. Let be the weight matrix of the candidate hidden states. For element-wise multiplication; Using update gates in combination with time steps t -1 hidden state With time step t Candidate hidden state , obtain time step t Hidden state ; By concatenating the dimensions of the hidden states at each time step, a low-dimensional feature representation is obtained. .

4. The traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert as described in claim 3, characterized in that, Based on low-dimensional feature representation, a decoder is used to reconstruct the original data, resulting in reconstructed data. According to the loss function, the reconstructed data, and the original data, a low-dimensional feature representation that retains the most original information is obtained. K-Means clustering is then applied to this low-dimensional feature representation to obtain clusters with high and low means. Specifically: Based on low-dimensional feature representation Use a decoder to process the raw data X Reconstruct the data to obtain the reconstructed data. ; Use the loss function to calculate the reconstructed data. Compared with the original data X The differences between them are used to update the encoder parameters so that, during computation, the low-dimensional feature representation can retain the original information to the greatest extent, where the loss function... The calculation formula is: , Mean absolute error; Low-dimensional feature representation that retains the original information to the greatest extent possible Clustering is performed using the data as input to K-Means, with the clustering target being... The formula is: ，N Number of samples ，K Number of clusters ， For indicator functions, For the first i Cluster labels for each sample, For the first i Data points of each sample, For the first j The cluster center of each cluster.

5. The traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert as described in claim 4, characterized in that, Low-dimensional feature representation that retains the original information to the greatest extent possible Clustering is performed using the data as input to K-Means, specifically as follows: Multiple initial cluster centers are randomly selected. For each data point, it is assigned to the nearest cluster based on its distance from all cluster centers until all data points have been assigned. Then, the center of each cluster is recalculated. Repeat the above steps until the change in the cluster center is less than the preset value or the preset number of iterations is reached; Calculate and compare the within-cluster means of each cluster, and define the cluster with the largest mean as... Clusters representing high-traffic nodes are handled by high-traffic experts; clusters with low mean are defined as... This represents a cluster of low-traffic nodes, which should be handled by low-traffic specialists. , For high-traffic nodes, This refers to the number of nodes with low traffic.

6. The traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert as described in claim 5, characterized in that, A high-flow expert network is used to process clusters with large means to obtain a high-flow output feature representation, specifically: Clusters with large mean are processed through a fully connected layer. A high-dimensional hidden representation is obtained by performing a nonlinear mapping. , , For the process FC Dimensional changes, It is a fully connected layer; Hiding the high dimension As query vectors, they are respectively compared with the preset traffic pattern database. It performs spatial similarity calculations on itself to measure the correlation between nodes, generating two dynamic adjacency matrices. and Its formula is: , , It is an exponential function. For row index, For column indexes, For transpose, For the first The high-dimensional hidden representation corresponding to the row, For the first Transpose of the traffic pattern database data corresponding to the column. For the first The transpose of the high-dimensional hidden representation corresponding to the column; Dynamic adjacency matrix and By splicing and merging with a fully connected layer, the final dynamic adjacency matrix is ​​obtained. , , For splicing operations; With the final dynamic adjacency matrix As supplementary information, a diffusion-based GCN graph convolutional network is used to capture dynamic spatial correlations and generate high-throughput output feature representations. Its formula is: , For high-volume time steps, For GCN graph convolutional networks, d For the number of diffusion steps, o Index of diffusion steps, Weights for high-volume output feature representation. b The bias value is used to represent the characteristics of high-volume output.

7. The traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert as described in claim 6, characterized in that, Using a low-flow expert network to process clusters with low means, we obtain a low-flow output feature representation, specifically: Clusters with small mean are processed through a fully connected layer. A nonlinear mapping is performed to obtain a low-dimensional hidden representation. , ; Hidden low-dimensional representation As query vectors, they are respectively compared with the preset traffic pattern database. It also performs spatial similarity calculations on itself to measure the correlation between nodes, generating a dynamic adjacency matrix. Its formula is: , For the first The low-dimensional hidden representation corresponding to the row, For the first The transpose of the low-dimensional hidden representation corresponding to the column; With dynamic adjacency matrix As supplementary information, a diffusion-based GCN graph convolutional network is used to capture dynamic spatial correlations and generate low-flow output feature representations. Its formula is: , For small flow time steps, Weights for outputting feature representations for low flow rates. The bias value is used to represent the characteristics of small flow output.

8. The traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert as described in claim 7, characterized in that, Feature extraction is performed on the feature representation of high-volume output to generate high-volume expert prediction results, specifically: Two general time embedding methods are introduced: daily embedding within a week. and timestamp embedding throughout the day , , , For the embedded dimension; Embedded within a week and timestamp embedding throughout the day Perform concatenation and broadcasting to generate a first periodic embedding that matches the traffic time series dimension. and ; For the first periodic embedding , and high flow rate output characteristics By splicing the data, a representation containing rich spatiotemporal information is constructed. , ; Using linear transformation, Z Mapped to query matrices respectively Query Key matrix Key Sum matrix Value Its formula is: , , , To query the weight matrix of the matrix, The weight matrix is ​​the key matrix. The weight matrix is ​​the value matrix; According to the first query matrix Query First key matrix Key and the first value matrix Value The first attention score is calculated using the following formula: , for softmax Activation function Let be the dimension of the key matrix; Employ a multi-head attention mechanism to execute O Each independent self-attention calculation is performed, and the results are concatenated and fused using linear projection to obtain a high-volume fusion result. The formula is as follows: , For concatenation functions, This is the first self-attention output. For the first O Self-attention output, This is the weight matrix used to fuse multi-head outputs back to the original dimension; The high-volume fusion results are input into the Transformer time feature extraction module to obtain high-volume expert prediction results.

9. The traffic flow prediction method based on masked autoencoder clustering and adaptive hybrid expert as described in claim 8, characterized in that, Feature extraction is performed on the low-flow output feature representation to generate low-flow expert prediction results. The high-flow and low-flow expert prediction results are then concatenated and fused to obtain the final prediction result, as follows: Embedded within a week and timestamp embedding throughout the day Perform concatenation and broadcasting to generate a second periodic embedding that matches the traffic time series dimension. and ; For the second periodic embedding , and small flow output characteristic representation By splicing the data, a representation containing rich spatiotemporal information is constructed. , ; Using linear transformation, Mapped to the second query matrix respectively Second bond matrix Second-value matrix Its formula is: , , ; According to the second query matrix Second bond matrix Second-value matrix The second attention score is calculated using the following formula: ; Employ a multi-head attention mechanism to execute O Each independent self-attention calculation is performed, and the results are concatenated and fused using linear projection to obtain the low-flow fusion result, the formula of which is: ; Input the small flow fusion result into the Transformer time feature extraction module to obtain the small flow expert prediction result; The prediction results from high-volume and low-volume experts are combined and merged to obtain the final prediction result. .

Images