Traffic flow data completion method and system based on frequency domain diffusion model

Abstract

The invention provides a traffic flow data completion method and system based on a frequency domain diffusion model, and the method comprises the steps: carrying out the singular spectrum analysis of traffic flow data, obtaining an accumulated feature value contribution rate and an accumulated contribution rate threshold value, and obtaining a global reserved component number according to the accumulated contribution rate threshold value and the accumulated feature value contribution rate; performing masking processing on missing values in the traffic flow data, performing singular spectrum analysis on the masked traffic flow data according to the number of global reserved components to obtain frequency domain data, performing gradual noise adding and gradual noise reduction on the frequency domain data according to spatial association of the traffic flow data, and generating frequency domain recovery data by using reverse singular spectrum analysis; and fusing the frequency domain recovery data and the time domain recovery data after time sequence diffusion processing, and performing feature extraction on a fusion result through a graph deep learning model to obtain a final traffic flow data completion result. According to the technical scheme, the traffic flow data complementation precision and robustness can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to a traffic flow data completion method and system based on a frequency domain diffusion model. BACKGROUND

[0002] The integrity and quality of traffic flow data are the core basis for the effective operation of intelligent transportation systems, directly affecting the accuracy of traffic management decisions and the reliability of service output. In the actual data collection process, sensors often produce various forms of data missing due to sudden failure or the influence of external harsh environment, which destroys the continuity and correlation of the original data in the time and space dimensions. If not effectively repaired, such missing data will seriously restrict the performance of downstream tasks such as traffic flow prediction and signal control optimization, hindering the overall development of intelligent transportation.

[0003] Existing traffic data completion methods mainly include traditional statistical methods, numerical analysis and tensor methods, deep learning methods, and diffusion model methods. Traditional statistical methods such as moving average, historical mean interpolation, etc. are based on intuitive statistical principles, with high computational efficiency and easy implementation; methods based on low-rank matrix / tensor completion treat traffic data as matrices or high-order tensors for processing, repairing by mining the low-rank nature and global structure of the data; deep learning methods, especially graph neural networks, can learn complex nonlinear spatiotemporal dependencies in traffic data end-to-end, explicitly modeling road network topology; diffusion model methods, as a new emerging deep generation paradigm, achieve data reconstruction through a gradual noise adding and denoising process, with advantages such as stable training and good sample diversity.

[0004] However, existing methods still have obvious limitations: traditional tensor completion methods usually rely on fixed low-rank assumptions or hand-designed thresholds, making it difficult to adapt to the complex spatiotemporal heterogeneity and multi-scale periodic patterns in traffic data; graph neural network-based models are prone to over-smoothing in spatial dependency modeling, making it difficult to capture rapid spatial changes; existing diffusion model methods often process time or frequency domain information independently, lacking effective multi-domain collaboration mechanisms, resulting in a loss of modeling accuracy. These limitations seriously restrict the effectiveness and reliability of traffic data completion technology in practical applications. SUMMARY

[0005] In view of this, the present application proposes a traffic flow data completion method and system based on a frequency domain diffusion model to solve the above-mentioned problems existing in the prior art.

[0006] To achieve the above-mentioned purpose, the present application proposes a traffic flow data completion method and system based on a frequency domain diffusion model, comprising: Traffic flow data is acquired, singular spectrum analysis is performed on the traffic flow data, cumulative eigenvalue contribution rate and spectral entropy are obtained based on the singular spectrum analysis results, cumulative contribution rate threshold is calculated based on the spectral entropy, and the number of globally retained components is obtained based on the cumulative contribution rate threshold and cumulative eigenvalue contribution rate. Missing values ​​in the traffic flow data are masked. Singular spectrum analysis is performed on the masked traffic flow data based on the number of globally retained components to obtain frequency domain data. The frequency domain data is then progressively denoised to obtain noisy frequency domain data. Progressive denoising and inverse singular spectrum analysis are then performed on the noisy frequency domain data to obtain frequency domain reconstructed data. Temporal diffusion is then performed on the masked traffic flow data to obtain temporal domain reconstructed data. Progressive denoising and denoising are performed based on the spatial correlation of the traffic flow data. Frequency domain recovery data and time domain recovery data are fused, and features are extracted from the fusion result using a graph deep learning model to obtain the final traffic flow data completion result.

[0007] Optionally, the process for obtaining the cumulative contribution rate threshold is as follows:

[0008] in, This represents the cumulative contribution rate threshold. Based on the threshold, To compensate for parameters, Here, j represents the normalized spectral entropy value, and j denotes the label of each sensor node in the traffic flow.

[0009] Optionally, the process of obtaining the number of globally retained components includes: The cumulative feature value contribution rate of each node in the traffic flow under different component numbers is determined by a cumulative contribution rate threshold. When the cumulative feature value contribution rate is greater than the cumulative contribution rate threshold, the number of components corresponding to each node is recorded. The minimum value of the recorded component number of each node is searched to obtain the minimum value component number of each node. The minimum value component data of the recorded minimum value is calculated by mean calculation and rounding down to obtain the number of globally retained components.

[0010] Optionally, the process of progressively adding noise to the frequency domain data includes: Construct distributed noise, wherein the distributed noise is related to and independently and identically distributed with respect to the graph topology corresponding to the traffic flow data; obtain the graph structure information of the graph structure corresponding to the traffic flow data. The graph convolutional neural network generates structured noise that conforms to the road network topology based on the distributed noise and graph structure information; and the structured noise is gradually added to the frequency domain data through a weighted sum.

[0011] Optionally, the process of progressively denoising the noisy frequency domain data includes: Obtain the graph structure information corresponding to the traffic flow data, and use a denoising network to predict the predicted noise at the current time step based on the graph structure information, the denoised data at the current time step, and the index at the current time step. The noise in the frequency domain data after adding noise is gradually removed at different time steps to obtain the noise-reduced frequency domain data.

[0012] Optionally, frequency domain recovery data and time domain recovery data can be fused using deep learning.

[0013] Optionally, the process of feature extraction from the fusion result includes: Obtain the adjacency matrix of the road network structure corresponding to the traffic flow data; Traffic flow data completion results are obtained by processing the adjacency matrix and fusion results through a graph convolutional network and a graph attention network.

[0014] On the other hand, the present invention also provides a traffic flow data completion system based on a frequency domain diffusion model for performing the above-described method.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes an adaptive frequency domain selection mechanism based on singular spectral analysis (SSA) and spectral entropy compensation. This mechanism dynamically adjusts the cumulative contribution rate threshold by calculating the power spectral entropy of each node, automatically determining the number of globally retained frequency domain components. This overcomes the limitations of the fixed low-rank assumption, enabling more flexible capture of multi-period patterns in traffic data and significantly improving completion performance in scenarios with missing blocks or long time-series data.

[0016] This invention designs a frequency-domain sensing GCN diffusion module that introduces graph structure constraints during the diffusion process. In the forward propagation process, the module uses GCN to propagate noise, ensuring that the added noise conforms to the road network topology. In the backward propagation process, a denoising network is used to simultaneously learn node features and spatial correlations, effectively mitigating the over-smoothing problem. While filling in missing values, it maintains spatial correlations and improves the ability to capture rapid spatial changes.

[0017] This invention constructs a time-frequency dual parallel diffusion path and performs deep fusion through an integrated prediction module. The time-series diffusion module uses a general time-series diffusion model to capture the time-domain evolution law, while the frequency-domain branch extracts spatially enhanced frequency-domain features. Finally, a combination of GCN and GAT is used to collaboratively fuse time-frequency information, achieving complementarity and enhancement of multi-domain information, thereby significantly improving the completion accuracy and robustness. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings: Figure 1 This is a schematic diagram of the overall process of the traffic flow data completion method based on the frequency domain diffusion model in this embodiment of the invention; Figure 2 This is a schematic diagram of a traffic flow data completion method based on a frequency domain diffusion model in an embodiment of the present invention; Figure 3 This is a schematic diagram of the data preprocessing process in an embodiment of the present invention; Figure 4 This is a schematic diagram of the frequency domain sensing GCN diffusion module in an embodiment of the present invention; Figure 5 This is a schematic diagram of the integrated prediction module process in an embodiment of the present invention; Figure 6 This is a comparison chart of the completed data and the original data of PEMS-BAY under different point missing degrees in an embodiment of the present invention; Figure 7 This is a comparison chart of the completed data and the original data of PEMS-BAY under different block missing degrees in an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] This invention discloses a traffic flow data completion method and system based on a frequency domain diffusion model, relating to the field of intelligent transportation systems. The method includes: acquiring original traffic flow data containing missing values; performing singular spectrum analysis on the original traffic flow data and adaptively determining the number of key frequency domain components to be retained using a spectral entropy compensation mechanism to overcome the limitations of the fixed low-rank assumption; utilizing graph structure constraints, performing a forward noise addition and backward noise reduction diffusion process in the frequency domain space to generate spatially correlated frequency domain completed data; simultaneously, employing an autoregressive sliding diffusion model in the time domain to capture the temporal evolution of traffic flow and generate time domain completed data; performing feature-level fusion of the frequency domain completed data and the time domain completed data, and further using a combination of graph convolutional networks and graph attention networks to deeply mine spatial dependencies, outputting the final completion result. This invention effectively solves the problems of poor adaptability to complex spatiotemporal heterogeneity, overly smooth spatial dependency modeling, and insufficient multi-domain information collaboration in existing methods by adaptive frequency domain selection, graph constraint diffusion, and deep fusion of time and frequency paths. It significantly improves the accuracy and robustness of data completion in various scenarios such as point missing and block missing.

[0021] The steps of the traffic flow data completion method based on the frequency domain diffusion model provided by this invention are as follows: S1: Data preprocessing and adaptive frequency domain selection; S2: Frequency domain perceptron convolutional diffusion; S3: Multi-domain fusion and complete output; In S1 above, the specific implementation steps of the data preprocessing and adaptive frequency domain selection process are as follows: S101: Perform singular spectral analysis (SSA) on the input traffic flow data, construct the trajectory matrix of each node and perform singular value decomposition; S102: Calculate the cumulative contribution rate and power spectral entropy of the eigenvalues, and dynamically adjust the cumulative contribution rate threshold through the spectral entropy compensation mechanism; S103: Determine the number of frequency domain components to be retained for each node based on the adjusted threshold, and finally calculate the total number of retained components globally. ; In S2 above, the specific implementation steps of the frequency domain sensing map convolution diffusion process are as follows: S201: During the forward noise addition process, structured noise conforming to the road network topology is generated through a graph convolutional network (GCN) and gradually added to the frequency domain data; S202: In the reverse denoising process, the denoising network is used to learn node features and spatial correlations simultaneously, and the original data is gradually restored. S203: Learn robust representations with graph structure preservation properties through a multi-step diffusion process; In S3 above, the specific implementation steps of the multi-domain fusion and completion output process are as follows: S301: Perform feature-level fusion on the frequency domain features output by the frequency domain diffusion module; S302: Further explore spatial dependencies through a combination of Graph Convolutional Network (GCN) and Graph Attention Network (GAT); S303: After multi-layer feature extraction and fusion, the final completion result is output.

[0022] Furthermore, the spectral entropy compensation mechanism in S102 specifically involves dynamically adjusting the cumulative contribution rate threshold by calculating the power spectral entropy of each node. The calculation formula is: ,in Based on the threshold, To compensate for parameters, This is the normalized spectral entropy value.

[0023] Furthermore, the structured noise generation in S201 specifically involves: propagating independent and identically distributed noise through a graph convolutional network to generate correlated noise that conforms to the road network topology, calculated using the following formula. ,in For the normalized adjacency matrix, For independent and identically distributed noise, For trainable weight matrix, This is the activation function.

[0024] The above technical solution is described in detail below: For the traffic flow data completion scenario of this invention, the problem is first defined as follows: In order to model the completion of missing values ​​in traffic flow data, a binary mask matrix is ​​set at each time step. ,in, Indicates the number of sensors. This represents the dimension of the sensor recording, and if the elements in the mask matrix correspond to the dimensions of the sensor-recorded data, then the elements in the mask matrix... , representing source data The data at the location is missing, and vice versa. Furthermore, we denote the complete source data, without any missing data, as... .

[0025] The goal of traffic flow completion is to estimate missing values ​​in an input data sequence, for a given length of... Graph sequence We can define the optimization objective L(·) as:

[0026] here, yes The reconstructed value; This indicates missing mask data; , , These represent the corresponding reconstruction, source data, and mask matrix, respectively. Represents the element-wise loss function (e.g., MAE, RMSE, etc.); Dot product of vectors This indicates that the modulo operation is performed on the vector.

[0027] To solve the above-mentioned technical problems, the present invention proposes the following technical solution: Firstly, to address the issue that traditional tensor completion methods rely on fixed low-rank assumptions or manually designed thresholds, making them ill-suited to the complex spatiotemporal heterogeneity and multi-scale periodic patterns of traffic data, this invention proposes an adaptive frequency domain selection mechanism based on Singular Spectral Analysis (SSA) and spectral entropy compensation. This mechanism dynamically adjusts the cumulative contribution rate threshold by calculating the power spectral entropy of each node, automatically determining the number of globally retained frequency domain components. This overcomes the limitations of fixed low-rank assumptions, enabling more flexible capture of multi-period patterns in traffic data and significantly improving completion performance in scenarios with missing blocks or long temporal sequences.

[0028] Secondly, addressing the issues of existing diffusion models not explicitly modeling spatial dependencies and traditional GCN methods failing to capture rapid spatial changes due to over-smoothing, this invention designs a frequency-domain-aware GCN diffusion module that introduces graph structure constraints during the diffusion process. This module uses GCN to propagate noise during the forward pass, ensuring that the added noise conforms to the road network topology. During the backward pass, it simultaneously learns node features and spatial correlations through a denoising network, effectively mitigating the over-smoothing problem. It maintains spatial correlation while filling in missing values, thus improving the ability to capture rapid spatial changes.

[0029] Thirdly, to overcome the problem of existing methods independently processing time-domain or frequency-domain information and lacking multi-domain collaboration leading to modeling accuracy loss, this invention constructs a time-frequency dual parallel diffusion path and performs deep fusion through an integrated prediction module. The time-series diffusion module uses a general time-series diffusion model to capture the time-domain evolution law, while the frequency-domain branch extracts spatially enhanced frequency-domain features. Finally, a combination of GCN and GAT is used to collaboratively fuse time-frequency information, achieving complementarity and enhancement of multi-domain information, thereby significantly improving the completion accuracy and robustness.

[0030] The following is a detailed analysis of the specific processing details: Firstly, regarding the first aspect, namely the computational steps of the data preprocessing section, the goal of this step is to perform singular spectral analysis (SSA) on the time series of each node individually, and then determine the number of components to be retained for each node by combining the eigenvalue contribution rate and spectral entropy compensation. Finally, for all The average is then rounded down to obtain the number of globally retained components. , which represents the number of retained components in the entire dataset.

[0031] Let the input data be... ,in Indicates the number of time steps. Indicates the number of nodes. The dimension representing the node features, here, For each node First, construct its trajectory matrix. ,in, The window length representing the SSA needs to be customized; this article sets this value to [value]. trajectory matrix Elements within Defined as:

[0032] in, Indicates the first The time series of each node. This is equivalent to combining the time series... It is transformed into a matrix composed of lag vectors, where m and n represent the row and column indices in the trajectory matrix.

[0033] Next, the trajectory matrix Perform singular value decomposition (SVD), the specific decomposition formula is as follows: ,in, and These are the left and right singular vector matrices, respectively. It is a diagonal matrix, where R represents the dimension of the singular values ​​in the diagonal matrix, and the diagonal elements are the singular values. Therefore, we can conclude that... Eigenvalues Then, calculate the cumulative contribution rate and spectral entropy of the eigenvalues. Cumulative eigenvalue contribution rate The calculation formula is:

[0034] Next, we calculate the spectral entropy, starting with the power spectral density. :

[0035] in, The calculation of the Discrete Fourier Transform is represented. Indicates the number of time steps. Represents the power spectral energy. Indicates the first The time series of each node is then processed. The power spectral density is then normalized to obtain the normalized power spectral density. :

[0036] Calculate the normalized spectral entropy based on the calculated power spectrum. :

[0037] Where f represents the time-series signal, .

[0038] Then calculate the number of components to be retained. and the number of global components First, the cumulative contribution rate threshold is adjusted using spectral entropy compensation. :

[0039] in, The base threshold representing the cumulative contribution rate. This is the spectral entropy compensation parameter. To ensure... The contribution rate threshold obtained above is further adjusted, that is... Then, find the smallest integer. This causes the cumulative contribution rate to exceed the adjusted threshold:

[0040] Finally, for all nodes After averaging and rounding down, we obtain the final globally optimal number of frequency domain components to retain. :

[0041] For a detailed flowchart of the data preprocessing process, please refer to [link / document name]. Figure 3 .

[0042] Next, regarding the second aspect, the processing flow of the frequency-aware GCN diffusion module, after the data preprocessing process described above, the optimal number of frequency components to be retained for a specific dataset is obtained. This number of components will serve as the hard threshold for subsequent training of this module. This step will train the diffusion model on the preprocessed frequency domain data. Overall, it is divided into a forward noise addition process and a backward denoising process. In addition to the usual noise addition operations, this model also adds noise to the relationships between nodes in the representation graph during the noise addition and denoising processes.

[0043] The first step is the forward noise addition process, which is a Markov chain that progressively adds noise to the data. This applies to the input traffic flow data that has been processed using a missing value mask matrix. ( Indicates the number of time steps. This indicates the total number of road network nodes. (representing feature dimension), Indicates the first The node feature matrix at each time step. The mask matrix is ​​processed by setting the original data values ​​at the positions corresponding to missing values ​​in the mask matrix to 0. Assume the total number of diffusion steps is... In the The next step is to generate independent and identically distributed noise with graph-structured correlation. ,in Represents the identity matrix. The dimension representing the noise is a hyperparameter. Since GCN is used for noise propagation, we define the noise propagation rule as:

[0044] in, It is a normalized adjacency matrix ( , yes The degree matrix, where A represents the adjacency matrix, i.e., the graph structure information. IN Represents the identity matrix. (This indicates adding a self-looping adjacency matrix); It is a trainable weight matrix used to map the noise dimension to the target dimension. It is an activation function, such as ReLU. This indicates that noise needs to be added.

[0045] In summary, the general formula for the forward process is as follows:

[0046] in, Indicates the first The node feature matrix at each time step, where the superscript k indicates the k-th time step. These are learnable noise scheduling parameters that control how much of the original signal is retained and how much noise is added at each step. Finally, after... After stepping, It will approximate pure Gaussian noise.

[0047] Next comes the reverse denoising process, which is crucial for reconstructing the original data from the noise. This model needs to learn a denoising network that not only denoises at the node feature level but also utilizes a graph structure to denoise in the "node space." Denoising Network The goal is to predict the diffusion step Noise added during Its input includes: current noise data. Diffusion step index Graph structure information The definition is as follows:

[0048] Then, through the following steps, from By removing noise from the predictions, a clearer image is obtained. :

[0049] in, , Z is a learnable noise scheduling parameter, representing standard Gaussian noise, which is introduced as a random term in the inverse denoising step. Its core function is to inject randomness into the sampling process, ensuring that the model can generate diverse samples that conform to the data distribution, rather than a single, deterministic result. This is the variance related to noise scheduling. For each time step... Repeat the above reverse process, from arrive Finally, the frequency domain completion data generated by the diffusion model is obtained: .

[0050] For a detailed flowchart of the frequency domain sensing GCN diffusion module, please refer to [link / reference]. Figure 4 The data processed by the frequency domain sensing GCN diffusion module needs to undergo singular value analysis beforehand based on the optimal number of frequency domain components to be retained. The singular value analysis results are used as input data for the frequency domain sensing GCN diffusion module. After the frequency domain sensing GCN diffusion module completes its processing, it is further processed through inverse singular spectrum analysis for integration.

[0051] In addition, the timing completion process uses the most common timing diffusion model, so it will not be described in detail here.

[0052] The third aspect concerns the calculation process of the integrated prediction module. After the above steps, the following is obtained: as well as Finally, the model passes through its last module: the integration and prediction module. In this module, spatial correlations are further integrated with the traffic network. To more comprehensively learn local and global correlation patterns, we simultaneously used both GCN and GAT graph processing methods. The first step is to fuse frequency domain and time series data, using the following formula:

[0053] in This represents a dynamically learnable parameter matrix. This is the result of time-frequency fusion. Finally, this invention combines GCN and GAT, defining one layer as... ,in It is the adjacency matrix of the road network. This represents the time-frequency fusion result obtained above. Generally, at least three layers of processing are required to obtain the final completed result. .

[0054] For a detailed flowchart of the integrated prediction module, please refer to [link / document]. Figure 5 .

[0055] The performance data of the above technical solution is described in detail using relevant data: This example uses two real-world traffic datasets to evaluate the performance of the proposed model. PEMS-BAY is traffic flow data collected by the California Department of Transportation (Caltrans) through its Performance Measurement System (PeMS). This dataset contains traffic flow information on highways in the San Francisco Bay Area of ​​California, USA. All datasets were collected at 5-minute intervals. METR-LA is traffic data from the Los Angeles County highway system in California, USA, with data also collected by sensors every 5 minutes. Specific dataset characteristics are shown in Table 1. To ensure the robustness of the model, z-score normalization was performed on the above data before model training.

[0056] Table 1

[0057] FDM-IM was developed using the PyTorch framework. All experiments were conducted on an Ubuntu 20.04 server equipped with an Intel(R) Xeon(R) Gold6226 R CPU @ 2.90GHz and an NVIDIA GeForce RTX 3090 (24G) card. The training, validation, and test sets were divided in a 7:1:2 ratio.

[0058] Below are some default parameter settings for our model: In the frequency-aware GCN diffusion module, the feature dimension used in the intermediate GCN layers is 24, the input dimension is equal to the output dimension, and both denoising and denoising iterations are set to 1000. In the temporal diffusion module, the sliding window length is set to 12 (one hour), and both sliding and desliding iterations are set to 1000. In the ensemble prediction module, the number of GAT&GCN layers is set to 3. Additionally, the Adam optimizer is used to optimize the model, and the batch size is set to 32 by default. The initial learning rate is set to 0.001. CosineAnnealingLR is used as the learning rate scheduler. During training, if the loss function value does not decrease within two consecutive epochs, the learning rate will be reduced to half of its original value.

[0059] Regarding evaluation metrics, we used three: RMSE (Root Mean Square Error), MAE (Mean Absolute Error), and MAPE (Mean Absolute Percentage Error). Among these metrics, RMSE emphasizes larger errors and directly reflects the magnitude of the absolute error in the prediction; MAE intuitively reflects the average error in the prediction, treating all errors equally; and MAPE is more sensitive to small errors. These metrics provide quantitative measures for evaluating prediction accuracy and stability, and their formulas are as follows:

[0060] in, and These are ground truth and predicted values, respectively. It is the length of the predicted data.

[0061] By comparing with the following seven baselines, we have demonstrated the accuracy of FDM-IM: Mean: This method is a simple and direct way to fill in the current missing values ​​using the arithmetic mean of historical data from the same period.

[0062] KNN: This method finds the most similar data points to the missing data points by calculating the Euclidean distance. A complete historical sample was used to fill in the gaps by using a distance-weighted average of its values.

[0063] MIWAE: This model proposes a deep delay variable model based on an importance-weighted autoencoder, which optimizes the model by maximizing the tight lower bound of the log-likelihood of the observed data.

[0064] rGAIN: is an unsupervised version of SSGAN, which can be viewed as Gain with a bidirectional recursive encoder and decoder.

[0065] BRITS: This is an end-to-end model based on a bidirectional recurrent neural network. It can directly learn missing values ​​in time series and simultaneously perform data imputation and classification / regression tasks without making any specific assumptions about the data generation process.

[0066] MPGRU: A GNN-based prediction model similar to DCRNN.

[0067] GRIN is a multivariate time series missing value imputation model based on graph neural networks. It learns the spatiotemporal dependencies between variables through a message passing mechanism.

[0068] Table 2 shows the model's completion performance on the PEMS-BAY dataset with different degrees of missing points; Table 3 shows the model's completion performance on the PEMS-BAY dataset with different degrees of missing blocks; Table 4 shows the model's completion performance on the METR-LA dataset with different degrees of missing points; Table 5 shows the model's completion performance on the METR-LA dataset with different degrees of missing blocks.

[0069] Table 2

[0070] Table 3

[0071] Table 4

[0072] Table 5

[0073] Tables 2 to 5 summarize the completion performance of models such as FDM-IM, GRIN, BRITS, MPGRU, rGAIN, MIWAE, KNN, and Mean on the PEMS-BAY and METR-LA datasets for both point-missing and block-missing modes, with missing rates ranging from 5% to 80%. Based on a comprehensive analysis of the experimental results, this study draws the following conclusions: (1) As the missing rate increases, the completion error of all models shows an upward trend, reflecting the general challenge that high missing rates pose to the model's feature extraction and reconstruction capabilities. Among them, traditional methods such as Mean and KNN are the most sensitive to the missing rate. For example, when 80% of the PEMS-BAY points are missing, Mean's MAE reaches 8.59, and KNN's is 7.29, indicating that simple interpolation methods have obvious limitations when information is severely missing. Deep learning methods generally perform better. For example, BRITS and MPGRU have MAEs of 2.16 and 1.75 respectively under the same conditions, showing the advantage of recurrent architecture in time series completion. It is worth noting that the model's MAE is 0.83 when 50% of the points are missing, showing the effectiveness of spatiotemporal joint modeling. The difference in sensitivity of different models to the missing rate reflects the different adaptability of their internal mechanisms to incomplete data.

[0074] (2) Block missing data has a greater impact on model performance than point missing data, which is evident across all models. For example, in the METR-LA dataset with a 50% missing data rate, GRIN's MAE for point missing data is 2.42, while it rises to 2.65 for block missing data; BRITS increases from 2.97 to 3.06. This indicates that the disruption of temporal continuity has a significant impact on models that rely on sequence coherence. Notably, RNN-based models such as BRITS and MPGRU show a relatively significant performance drop under block missing data, which is related to their strong dependence on temporal continuity. In contrast, GNN-based methods such as FDM-IM exhibit better stability, indicating that the effective use of spatial information can partially compensate for the discontinuity of temporal information. Probabilistic generation methods such as rGAIN show large performance fluctuations under block missing data, reflecting the sensitivity of these methods to the loss of continuous information.

[0075] (3) A comprehensive analysis of the performance of each model shows that GRIN maintains stable performance in most scenarios, and its spatiotemporal joint modeling method demonstrates adaptability under different missing data patterns. BRITS and MPGRU, as representative RNN-based models, perform well in point missing scenarios, but their performance drops significantly under block missing scenarios, which is closely related to their dependence on temporal continuity. Generative methods such as rGAIN and MIWAE perform reasonably well in some scenarios, but their overall performance fluctuates greatly. Traditional methods such as KNN are effective at low missing rates, but cannot cope with the challenge of high missing rates. These differences reflect the inherent characteristics of different methodologies in capturing the spatiotemporal features of traffic data.

[0076] (4) The overall performance of the models on the PEMS-BAY dataset is generally better than that on the METR-LA dataset, and this trend is reflected in all models. For example, when 50% of the points are missing, GRIN has an MAE of 0.83 on PEMS-BAY, while it has an MAE of 2.42 on METR-LA; BRITS has an MAE of 1.65 and 2.97 on the two datasets, respectively. This consistency difference across datasets may be due to the different characteristics of the datasets themselves, such as the complexity of traffic flow patterns and data distribution characteristics. It is worth noting that all models maintain a relatively consistent performance ranking on both datasets, indicating that the advantages of each method have a certain universality in different scenarios. As a comparative model, FDM-IM maintains good performance in both sets of experiments, but the degree of its advantage varies on different datasets, which also reflects the important influence of data characteristics on model performance.

[0077] (5) In Figure 6 and Figure 7 In our study, we selected one day from a specific sensor in PEMS-BAY and plotted the ground truth data for that day against the model's data under different missing rates. The results of a comprehensive evaluation of the two datasets, two missing modes, and three evaluation metrics show that the proposed FDM-IM model achieves best performance in most experimental scenarios. Its column heights are systematically lower than other comparative models, especially at extreme missing rates of 50% and 80%, where its performance advantage is even more pronounced. This fully demonstrates the model's superior accuracy and strong generalization ability in handling spatiotemporal data completion tasks with different missing scenarios.

[0078] Conclusion: This application addresses the shortcomings of existing traffic data completion methods in handling complex spatiotemporal heterogeneity and effectively fusing multi-domain information. It proposes a traffic flow data completion method and system (FDM-IM) based on a frequency domain diffusion model. This model utilizes adaptive frequency domain component selection and graph-structure-constrained frequency domain diffusion. Furthermore, the time-frequency dual-path fusion method enables high-precision reconstruction of missing values ​​in traffic flow data. Experiments on two real-world datasets demonstrate that FDM-IM outperforms existing mainstream methods in various missing value scenarios, significantly reducing completion errors and validating the effectiveness and superiority of the proposed model.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for traffic flow data completion based on a frequency domain diffusion model, characterized in that, The method comprises the following steps: acquiring traffic flow data, performing singular spectrum analysis on the traffic flow data, acquiring cumulative eigenvalue contribution rate and spectral entropy according to the singular spectrum analysis result, calculating a cumulative contribution rate threshold value according to the spectral entropy, and acquiring a global reserved component number according to the cumulative contribution rate threshold value and the cumulative eigenvalue contribution rate; performing mask processing on missing values in the traffic flow data, performing singular spectrum analysis on the traffic flow data after the mask processing according to the global reserved component number, acquiring frequency domain data, performing step-by-step noise addition on the frequency domain data, acquiring noise-added frequency domain data, performing step-by-step noise removal and reverse singular spectrum analysis on the noise-added frequency domain data, and acquiring frequency domain recovery data; and performing time series diffusion on the traffic flow data after the mask processing, and acquiring time domain recovery data; wherein the step-by-step noise addition and the step-by-step noise removal are performed according to the spatial correlation of the traffic flow data; fusing the frequency domain recovery data and the time domain recovery data, and performing feature extraction on the fusion result through a graph deep learning model to obtain a final traffic flow data completion result.

2. The method of claim 1, wherein the cumulative contribution rate threshold value is acquired through the following process:

3. The method of claim 1, wherein the global reserved component number is acquired through the following process: ， wherein, denotes a cumulative contribution rate threshold value, is a base threshold value, is a compensation parameter, is a normalized spectral entropy value, j denotes the index of each sensing node in the traffic flow. The cumulative eigenvalue contribution rate of each node in the traffic flow under different component numbers is subjected to threshold value judgment through the cumulative contribution rate threshold value, and when the cumulative eigenvalue contribution rate is greater than the cumulative contribution rate threshold value, the component number corresponding to each node is recorded. The minimum value of the component number of each node recorded is found to obtain the minimum value component number of each node recorded. The minimum value component data recorded is subjected to mean value calculation and floor calculation in sequence to obtain the global reserved component number.

4. The method of claim 1, wherein the step of performing step-by-step noise addition on the frequency domain data comprises the following steps: constructing distribution noise, wherein the distribution noise is related to the graph topological structure corresponding to the traffic flow data and is independent and identically distributed; and acquiring graph structure information of the graph structure corresponding to the traffic flow data; generating structured noise conforming to the topology of the road network through a graph convolutional neural network according to the distribution noise and the graph structure information; and adding the structured noise to the frequency domain data through a weighted sum in a step-by-step manner.

5. The method of claim 1, wherein the step of performing step-by-step noise removal on the noise-added frequency domain data comprises the following steps: acquiring graph structure information of the graph structure corresponding to the traffic flow data, and predicting the prediction noise at the current time step through a denoising network according to the graph structure information, the denoised data at the current time step, and the current time step index; removing the prediction noise at the corresponding time step from the noise-added frequency domain data at different time steps to obtain denoised frequency domain data.

6. The method of claim 1, wherein the frequency domain recovery data and the time domain recovery data are fused through a deep learning method.

7. The method of claim 1, wherein the step of performing feature extraction on the fusion result comprises the following steps: acquiring an adjacency matrix of the road network structure corresponding to the traffic flow data; ​ ​ ​ ​ ​ ​ The traffic flow data completion result is obtained by processing the adjacency matrix and the fusion result through a graph convolution network and a graph attention network. 8.A system for traffic flow data completion based on a frequency domain diffusion model, characterized in that, A computer program product for performing the method of any of claims 1-7.

Images