A multi-source spatio-temporal data fusion and missing data completion method based on diffusion model
By combining the diffusion model with multilayer perceptron, attention mechanism and convolutional neural network, the problem of missing data completion in multi-source spatiotemporal data is solved, and high-precision and stable data reconstruction is achieved, supporting city-level forecasting and energy optimization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER CO
- Filing Date
- 2026-05-08
- Publication Date
- 2026-07-28
AI Technical Summary
Traditional methods struggle to effectively characterize complex distributions and nonlinear correlations when processing multi-source spatiotemporal data, are ill-suited to irregular missing data and anomalies, and are difficult to quantify and complete uncertainties.
By employing a diffusion model combined with mask guidance and spatiotemporal conditions, and using a multilayer perceptron, attention mechanism, and convolutional neural network, semantic feature fusion and missing type identification of multi-source data are performed. A lightweight and semantically enhanced diffusion model branch is constructed for data completion.
It achieves high-precision and stable data completion, improves data integrity and consistency, and provides high-quality input for city-level forecasting and energy optimization.
Smart Images
Figure CN122471341A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of intelligent transportation and urban energy digitalization technology, specifically to a method for multi-source spatiotemporal data fusion and missing data completion based on a diffusion model. Background Technology
[0002] The data of the four types of urban-level "vehicle-pile-network-meteorology" systems are characterized by diverse sources, inconsistent temporal granularity, inconsistent spatial resolution, and the coexistence of missing and anomalies. Traditional interpolation (linear / spline / time series mean), statistical models (ARIMA / VAR), and adversarial generative models (GAN) often face the following challenges when dealing with high-dimensional, long-span, and multivariate coupled data: (1) insufficient characterization of complex distributions and nonlinear correlations; (2) poor adaptability to irregular missing, blocky missing, and conditional distribution drift; and (3) difficulty in explicitly quantifying and completing uncertainties.
[0003] The diffusion model can approximate complex data manifolds through noise perturbation and stepwise denoising. If combined with mask guidance, spatiotemporal conditions, and physical consistency constraints, it can achieve structural and statistical feature preservation in high-dimensional multivariable scenarios and output uncertainty assessment.
[0004] To address the aforementioned issues, there is an urgent need for a missing data completion method based on diffusion models for multi-source spatiotemporal data fusion, which can solve the problems existing in traditional methods and achieve missing data completion. Summary of the Invention
[0005] The purpose of this invention is to provide a method for multi-source spatiotemporal data fusion and missing data completion based on diffusion models. This method can standardize, align, semantically associate, and complete multi-source data under a unified spatiotemporal grid system, thereby improving data integrity and consistency and providing high-quality input support for subsequent city-level prediction, scheduling, and energy optimization models.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A missing data completion method for multi-source spatiotemporal data fusion and diffusion model-based methods includes: Step 1: Obtain a multi-source heterogeneous raw dataset covering vehicle operation data, charging pile data, power grid area load data, and meteorological monitoring data, and perform preprocessing and spatiotemporal alignment to obtain a multi-source heterogeneous dataset. Step 2: Perform semantic feature fusion on the multi-source heterogeneous original datasets to obtain a multi-source fused dataset; Step 3: Perform data quality checks and missing data type identification on the multi-source fusion dataset; Step 4: Based on the identification results, perform data missing completion on the multi-source fusion dataset using the data missing completion model.
[0007] Further, preprocessing and spatiotemporal alignment are performed, specifically as follows: Perform unified field mapping and unit conversion on data from multi-source heterogeneous original datasets; The processed data is aligned in the spatiotemporal dimension.
[0008] Furthermore, the processed data is aligned in the spatiotemporal dimension, specifically as follows: In terms of time dimension, the data is unified to a fixed time resolution; In terms of spatial dimension, the city regular grid unit is used as the smallest spatial resolution unit, and spatial registration of data from different sources is completed through multi-source geographic mapping relationship.
[0009] Furthermore, the accuracy of the smallest spatial resolution unit is 200 meters × 200 meters.
[0010] Furthermore, in step 2, semantic feature fusion is performed on the multi-source heterogeneous original dataset to obtain a multi-source fused dataset, specifically as follows: The attribute features of different data sources of the multi-source heterogeneous original dataset are input into the multilayer perceptron for processing to generate semantic features; Weighted fusion of semantic features is performed based on feature alignment network and attention mechanism; The fused features are concatenated to form a unified input tensor structure, resulting in a multi-source fusion dataset.
[0011] Furthermore, in step 3, a data quality check is performed on the multi-source fusion dataset, specifically as follows: Based on the detection method of statistical threshold and temporal stability, mutation, drift, duplication and invalid data are identified and labeled.
[0012] Furthermore, in step 3, the missing type identification is performed on the multi-source fusion dataset, specifically as follows: Based on the missing pattern classification criteria, missing data is divided into completely random missing data and conditional missing data, and a missing data mask matrix is generated.
[0013] Furthermore, in step 4, based on the identification results, data missing information is filled into the multi-source fusion dataset using a data missing information completion model. Specifically: A missing data type detection and model selection mechanism is constructed. If the missing data type is completely random, a lightweight diffusion model branch is used to complete the data. If the missing type is conditional missing, then the semantically enhanced conditional diffusion model branch is used for data completion.
[0014] Furthermore, the lightweight diffusion model branch uses a convolutional neural network as its core structure, combining local spatial convolution and neighborhood statistical features to quickly denoise and complete missing values.
[0015] Furthermore, the semantically enhanced conditional diffusion model branch is an encoding network composed of CNN and LSTM.
[0016] In summary, the present invention has the following beneficial technical effects: By integrating multi-source data features from vehicles, charging piles, networks, and meteorology, this method achieves unified modeling and high-precision data completion for complex urban energy systems. Differentiated diffusion model structures are introduced for different types of missing data, significantly improving the quality and stability of data recovery under multi-scenario and multi-granularity conditions. Furthermore, by introducing physical consistency constraints and semantic enhancement mechanisms, the method ensures the interpretability and practical usability of the completion results. This approach exhibits good scalability and versatility, and can be widely applied in fields such as intelligent transportation, energy dispatching, urban operation monitoring, and predictive management, providing a high-quality data foundation for city-level intelligent decision-making and resource optimization. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 A schematic diagram of preprocessing and spatiotemporal alignment of multi-source heterogeneous raw datasets; Figure 3 A schematic diagram of the process for completing missing data. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0019] like Figure 1 As shown, this invention provides a method for multi-source spatiotemporal data fusion and missing data completion based on a diffusion model, including: Step 1: Obtain a multi-source heterogeneous raw dataset covering vehicle operation data, charging pile data, power grid area load data, and meteorological monitoring data, and perform preprocessing and spatiotemporal alignment to obtain a multi-source heterogeneous dataset. Step 2: Perform semantic feature fusion on the multi-source heterogeneous original datasets to obtain a multi-source fused dataset; Step 3: Perform data quality checks and missing data type identification on the multi-source fusion dataset; Step 4: Based on the identification results, perform data missing completion on the multi-source fusion dataset using the data missing completion model.
[0020] like Figure 2 As shown, in step 1, a multi-source heterogeneous raw dataset covering vehicle operation data, charging pile data, power grid area load data, and meteorological monitoring data is obtained, and preprocessed and spatiotemporally aligned to obtain the multi-source heterogeneous dataset, specifically: Based on city-level traffic, power and meteorological data collection standards, we acquire multi-source heterogeneous raw datasets covering vehicle operation data, charging pile data, power grid area load data and meteorological monitoring data. The raw data of the collected multi-source heterogeneous raw datasets are uniformly mapped and converted to fields, and the multi-source data are aligned in the spatiotemporal dimension: In the time dimension, data sources from different sampling periods (such as vehicle operation data, charging pile monitoring data, power grid area load and meteorological observation data) are resampled and interpolated to unify them to a fixed time resolution (such as every 15 minutes), and synchronization gaps and abnormal time periods are recorded in the dataset to ensure the consistency and traceability of the time axis; In the spatial dimension, urban regular grid units are used as the smallest spatial resolution unit (typical accuracy is 200m×200m). Spatial registration of data from different sources is completed through multi-source geographic mapping relationships: vehicle and charging pile data are mapped to grid units through latitude and longitude coordinates; power grid data are spatially projected through the service range of the transformer area and the coverage area of the feeder; meteorological data are weighted and allocated to each grid through interpolation and proximity distance. This forms a three-dimensional aligned data volume of "time-space-data type", providing a unified structural foundation for multi-source feature fusion. At the same time, basic statistical standardization and normalization processing is performed on the data to eliminate the differences in the dimensions and magnitudes of data from different sources, providing a unified input structure for subsequent feature fusion and model training.
[0021] In step 2, semantic feature fusion is performed on the multi-source heterogeneous original dataset to obtain a multi-source fused dataset, specifically as follows: After completing the preprocessing and spatiotemporal alignment of multi-source data, this invention further employs a multi-source data fusion method based on semantic feature fusion. First, the attribute features of different data sources (such as vehicle flow, power consumption, pile occupancy rate, temperature and humidity, wind speed, etc.) are input into a multilayer perceptron for processing to generate semantic features to achieve semantic encoding and feature embedding, establishing a unified feature space representation. Second, the multi-source features are weighted and fused using a feature alignment network and attention mechanism to learn the semantic correlation and importance between different data sources. Finally, the fused multi-source semantic features are concatenated to form a unified input tensor structure, where each grid cell contains a fused multidimensional feature vector in each time slice, comprehensively representing the traffic, energy, and environmental status of the region at that moment.
[0022] In step 3, data quality checks and missing data type identification are performed on the multi-source fusion dataset, specifically as follows: The dataset after multi-source fusion is subjected to integrity and consistency checks to identify missing and anomalous regions. A detection method based on statistical thresholds and temporal stability is used to identify and label mutations, drifts, duplications, and invalid data. According to the missing pattern classification criteria, missing data is divided into completely random missing (MCAR) and conditional missing (MAR) to provide guidance for the generation of conditions in the diffusion model. Based on this, a missing mask matrix is generated, which contains only 0 and 1, where "1" indicates the presence of an observation and "0" indicates a missing value, providing a basis for the mask condition input of the subsequent model.
[0023] like Figure 3 As shown, in step 4, based on the identification results, data missing information is filled in on the multi-source fusion dataset using a data missing information completion model. Specifically: A missing type judgment and model selection mechanism is constructed, and a corresponding diffusion generation model is selected for completion based on different missing types, forming two branches of diffusion models: lightweight and semantic enhancement. When the missing value type is completely random missing (MCAR), a lightweight diffusion model branch is adopted. This branch uses a convolutional neural network as the core structure, combines local spatial convolution and neighborhood statistical features to quickly denoise and fill in the missing value. The convolutional neural network model can preserve local structural features with low computational cost and achieve high-fidelity recovery of spatial details through multi-scale convolutional encoding and skip connections. It is suitable for missing value scenarios with random distribution and small range. When the missing type is conditional missing (MAR), a semantically enhanced conditional diffusion model branch is adopted. This branch is implemented by an encoding network composed of CNN and LSTM. The former extracts spatial pattern features, while the latter captures the time series change patterns. On this basis, exogenous conditions (such as weather, holidays, and functional zone types) are introduced for semantic fusion. The model uses this conditional information to guide the data generation process during the diffusion inversion stage, thereby achieving data completion affected by external factors while considering the contextual spatiotemporal dependence. This is suitable for regular missing scenarios under specific conditions. The two types of diffusion model branches mentioned above share a unified input interface and semantic encoding framework. The input includes standardized multi-source feature tensors, missing mask matrices, spatiotemporal location codes, and exogenous conditional features. The system automatically selects the appropriate diffusion branch to perform the completion task through a missing type judgment function. In scenarios with multiple missing types, the system performs partitioning processing at the time slice and grid cell level, and performs fusion correction and confidence assessment at the output stage to ensure the consistency and continuity of the overall data. For different types of missing data, a unified training and optimization process is designed for the diffusion model branches to ensure the overall performance of the model in terms of spatiotemporal consistency, physical constraints and generation stability. During the training phase, the standardized and aligned multi-source dataset is first masked and sampled to randomly generate training samples with different missing proportions and forms to enhance the robustness of the model to multiple missing patterns. Based on the missing type judgment function, the system automatically assigns the samples to the corresponding diffusion model branches (lightweight diffusion model branch and semantically enhanced conditional diffusion model branch) and trains the generative network of each branch respectively. The lightweight diffusion model branch adopts a standard diffusion noise prediction strategy. By gradually adding noise to the data and learning the reverse denoising process, the model learns the recovery rules of local features. The semantically enhanced conditional diffusion model branch introduces a conditional feature encoding network, using exogenous semantic variables such as meteorology, holidays, and functional zones as generation condition inputs. The model learns the dynamic generation distribution under conditional dependence by minimizing the weighted mean square error (Weighted MSE) and temporal consistency loss. All branch models share a unified optimization objective framework during training. The loss function consists of a noise matching term, an observation-holding term, and a constraint regularization term: the noise matching term ensures that the generated samples are consistent with the true distribution, the observation-holding term ensures that existing observations are not destroyed, and the constraint regularization term maintains physical rationality and statistical stability. The model training adopts an adaptive learning rate optimization algorithm (such as AdamW) combined with an early stopping mechanism to prevent overfitting. After training, the system is evaluated through a validation set and an independent test set. The indicators include missing completion error (RMSE, MAE), distribution fit (KL divergence, EMD distance), and spatial correlation coefficient.
[0024] The present invention also includes step 5: outputting and verifying the results, specifically as follows: After model completion, the system outputs the complete results of multi-source fusion data and its quality verification information. The model predicts and outputs the completed data results for each grid across the entire time period, which can be defined as the comprehensive state vector of that grid within a specific time window. This vector includes continuous values of recovered data such as vehicle operating frequency, charging pile occupancy rate, grid load, and meteorological variables. The completed results can be further statistically analyzed into key indicators, such as charging demand intensity, energy load level, or regional operational activity, for subsequent model prediction and scheduling analysis. Simultaneously, the model generates a quality verification report during the output phase, including numerical accuracy indicators (MAE, RMSE) and distribution consistency. Evaluation results, including performance indicators (KL divergence, EMD distance), spatial correlation indicators, and physical constraint compliance rate, are used to quantify the model's completion performance and credibility, enabling continuous model optimization or verification of completion results. The results support various visualization methods, including time series curves, spatial heat maps, and missing rate comparison charts, and can be imported into GIS platforms or energy dispatch systems for spatial display and dynamic monitoring. This step ensures the completeness, visualization usability, and system integrability of the completed data, enabling the data results generated by this invention to seamlessly connect with subsequent predictive analysis, energy optimization, and smart city operation management.
[0025] It should also be noted that in step 2, the semantic features may include one or more of the following: vehicle traffic distribution, power consumption, pile occupancy rate, temperature, humidity, wind speed, etc. The feature fusion method may adopt a feature weighted network based on attention mechanism or a multilayer perceptron (MLP) structure to achieve semantic feature mapping and unified embedding between different data sources.
[0026] In step 4, during the model training and optimization process, the optimization algorithm can use AdamW or other adaptive learning rate optimizers. Model training can be executed in parallel in a multi-GPU environment, supporting mixed precision acceleration and parameter sharing to improve computational efficiency and model stability.
[0027] This invention provides an embodiment, and the invention will be described in detail with reference to this embodiment: Step 1: Obtain the original multi-source heterogeneous dataset and perform preprocessing and spatiotemporal alignment to obtain the multi-source heterogeneous dataset, specifically: This step is divided into two stages, including the acquisition of raw datasets and data preprocessing. First, in the stage of acquiring raw datasets, this invention acquires multi-source heterogeneous raw datasets based on the data source of urban-level traffic, power and meteorological data collection standards. The raw data includes: (1) vehicle operation data, covering high-precision trajectory data of various types of vehicles such as taxis, ride-hailing vehicles, buses, logistics vehicles and private cars in the city; (2) charging pile data, including the geographical coordinates of the charging pile location, operating status (idle / occupied), charging power and charging amount; (3) power grid area data, including the load, voltage, current, capacity and over-limit status of the area; (4) meteorological monitoring data, including environmental elements such as temperature, humidity, wind speed and rainfall. In the data preprocessing stage, the main features of each type of raw data are first extracted, and the data undergoes field unification, unit conversion, and missing data marking to ensure consistency across multiple data sources. Secondly, all data undergoes time discretization, refining the sampling time granularity to every 15 minutes. Further, the data is spatiotemporally aligned. In the time dimension, data with different sampling frequencies (such as vehicle trajectories, power grid load, and meteorological observations) are time-aligned to ensure consistency. In the spatial dimension, the urban area is divided into non-overlapping regular square grid units, with a spatial grid division precision set to 200 meters × 200 meters. Based on the latitude and longitude coordinates of each trajectory point, its corresponding grid unit is determined, realizing the mapping relationship between trajectory data and the spatial grid. For other data sources (such as charging piles and weather stations), spatial projection and matching to the grid are completed using geographic coordinates or the service area of the station area. Finally, the system statistically analyzes the frequency of occurrence of each vehicle type in each grid within each time slice, and combines this with charging pile status, power grid load, and meteorological characteristics to establish a three-dimensional dataset of "time-space-data type". Each data unit records the multi-source observations and feature variables within the corresponding time slice and grid. This dataset serves as the unified input basis for subsequent semantic feature fusion, missing data detection, and diffusion model completion.
[0028] Step 2: Perform semantic feature fusion on the multi-source heterogeneous original datasets to obtain a multi-source fused dataset, specifically as follows: After completing the preprocessing and spatiotemporal alignment of multi-source data, this invention further adopts a multi-source data fusion method based on semantic feature fusion to achieve unified feature expression of multiple types of data, including vehicle-pile-network-meteorological data.
[0029] First, representative attribute features are extracted from different data sources. Vehicle data is used to extract vehicle traffic distribution, average speed, and travel intensity; charging pile data is used to extract pile occupancy rate, activity level, and output power; power grid data is used to extract load power, load factor, and transformer capacity features; and meteorological data is used to extract environmental variables such as temperature, humidity, wind speed, and precipitation. All of these multi-source features are mapped using time alignment and spatial gridding to ensure consistency across time slices and spatial units.
[0030] Subsequently, the present invention inputs the aforementioned features into a multilayer perceptron (MLP) model for semantic encoding and feature embedding to obtain a unified semantic vector representation. This semantic encoding process can eliminate scale differences and statistical biases in data from different sources, establishing a comparable feature space.
[0031] Building upon this foundation, a feature alignment network and an attention weighting mechanism are employed to achieve deep fusion of multi-source features. The feature alignment network assigns dynamic weights based on the relevance and importance of each feature in the samples, while the attention mechanism further enhances the semantic contribution of key data sources (such as high-frequency vehicle traffic or high-impact meteorological variables), thereby enabling adaptive fusion of multi-source heterogeneous information.
[0032] Finally, the fused semantic features are concatenated into a unified input tensor structure for storage and modeling. Each grid cell contains a fused multidimensional semantic feature vector for each time slice, comprehensively representing the traffic operation status, energy load level, and environmental conditions of the region at that moment. This fusion result serves as the main input data structure for missing feature identification and diffusion model-based completion in subsequent steps.
[0033] Step 3: Perform data quality checks and missing data type identification on the multi-source fusion dataset, specifically: First, basic statistical tests and time-series stability analyses are performed on various feature variables. Based on the temporal continuity, fluctuation amplitude, and distribution patterns of the features, the system uses indicators such as sliding window mean, standard deviation, and rate of change threshold to identify and label abrupt changes, drifts, duplications, and invalid data, thereby locating potential abnormal segments and missing samples.
[0034] Furthermore, the system analyzes and classifies the formation mechanism of missing data through a missing data classification judgment function. Specifically, when missing samples are randomly distributed globally and the missing data is not significantly correlated with time, spatial location, or other variables, the system confirms that the missing data distribution is independent of the observed variables through independence tests (such as chi-square test and mutual information analysis), and determines it as completely random missing data (MCAR). When the occurrence of missing data is statistically correlated with other observed characteristics, such as a significant increase in the missing data rate under specific meteorological conditions or regional attributes, the system calculates the correlation coefficient or mutual information score between the missing data rate and external variables. If the correlation exceeds a preset threshold, it is determined as conditional missing data (MAR).
[0035] After classification, a missing type label table and a missing type mask matrix are generated via code. The label table records the missing type corresponding to each time slice and grid cell, and the mask matrix records the missing information. These results serve as conditional inputs to the diffusion model, guiding subsequent model selection and completion processes.
[0036] Step 4: Based on the identification results, perform data missing completion on the multi-source fusion dataset using a data missing completion model. Specifically: First, the missing data discrimination and missing type determination module determines the missing category (MCAR, MAR) for each time slice and grid cell, and then inputs the multi-source dataset containing missing data markers and type labels into the corresponding diffusion model branch.
[0037] (1) When the missing data type is completely random missing (MCAR), the data is randomly distributed and the missing data size is small. The system calls the lightweight diffusion model branch. This branch uses a convolutional neural network as the core network structure and combines local convolution and skip connection mechanisms to quickly denoise and complete the missing regions. The model reconstructs local spatial details and achieves efficient completion by progressively sampling the mean matrix and variance matrix. (2) When the missing data type is conditional missing (MAR), the system activates the semantically enhanced conditional diffusion model branch. This branch mainly uses a CNN-LSTM structure. CNN is used to extract spatial pattern features, and LSTM is used to capture the dynamic changes of the time series. At the same time, external factors such as weather, holidays, and functional area attributes are introduced as conditional inputs to guide the model to generate completion results consistent with the external environment during the diffusion inversion process. This is suitable for missing data affected by external semantic conditions.
[0038] Each diffusion branch employs a multi-step denoising and sampling strategy during the generation phase, iteratively updating the mean and variance matrices until convergence. After fusing the completed results with the original observation data, the system outputs a highly complete multi-source dataset, providing reliable input for subsequent model training and validation. Furthermore, the model objective function uses weighted mean squared error (MSE) to account for the varying importance of different time periods and data types during the completion process.
[0039] Step 5: Output and Verification of Data Completion Results The model predicts and outputs complete data results for each grid across the entire time period. Depending on the input, this can be categorized into a comprehensive state vector for that grid within a specific time window, including continuous values recovered from multiple sources such as vehicle operating frequency, charging pile occupancy rate, grid load, and meteorological variables. The complete results can be further statistically analyzed into key operational indicators, such as charging demand intensity (the estimated number of vehicles connected to charging piles), energy load level (total power demand within a unit grid), or regional operational activity (a comprehensive indicator of vehicle and charging pile utilization), for subsequent predictive analysis and energy dispatch optimization. Simultaneously, the quality of the complete data is calculated, including numerical accuracy indicators (mean absolute error and root mean square error), distribution consistency indicators (KL divergence and EMD distance), and spatial correlation indicators (spatial autocorrelation coefficient). These indicators can be used to evaluate the degree of fit between the generated data and the actual distribution. Furthermore, the completed data results can be imported into a GIS platform or energy dispatch system for spatial display and dynamic monitoring, and support data interface integration with subsequent model prediction or intelligent dispatch modules.
[0040] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0041] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0042] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0043] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0044] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.
Claims
1. A missing data completion method for multi-source spatiotemporal data fusion and diffusion model-based methods, characterized in that, include: Step 1: Obtain a multi-source heterogeneous raw dataset covering vehicle operation data, charging pile data, power grid area load data, and meteorological monitoring data, and perform preprocessing and spatiotemporal alignment to obtain a multi-source heterogeneous dataset. Step 2: Perform semantic feature fusion on the multi-source heterogeneous original datasets to obtain a multi-source fused dataset; Step 3: Perform data quality checks and missing data type identification on the multi-source fusion dataset; Step 4: Based on the identification results, perform data missing completion on the multi-source fusion dataset using the data missing completion model.
2. The method for multi-source spatiotemporal data fusion and missing data completion based on a diffusion model according to claim 1, characterized in that, Preprocessing and time-space alignment are performed, specifically as follows: Perform unified field mapping and unit conversion on data from multi-source heterogeneous original datasets; The processed data is aligned in the spatiotemporal dimension.
3. The method for multi-source spatiotemporal data fusion and missing data completion based on a diffusion model according to claim 2, characterized in that, The processed data is then aligned in the spatiotemporal dimension, specifically as follows: In terms of time dimension, the data is unified to a fixed time resolution; In terms of spatial dimension, the city regular grid unit is used as the smallest spatial resolution unit, and spatial registration of data from different sources is completed through multi-source geographic mapping relationship.
4. The method for multi-source spatiotemporal data fusion and missing data completion based on a diffusion model according to claim 3, characterized in that, The accuracy of the smallest spatial resolution unit is 200 meters × 200 meters.
5. A method for multi-source spatiotemporal data fusion and missing data completion based on a diffusion model as described in claim 4, characterized in that, In step 2, semantic feature fusion is performed on the multi-source heterogeneous original dataset to obtain a multi-source fused dataset, specifically as follows: The attribute features of different data sources of the multi-source heterogeneous original dataset are input into the multilayer perceptron for processing to generate semantic features; Weighted fusion of semantic features is performed based on feature alignment network and attention mechanism; The fused features are concatenated to form a unified input tensor structure, resulting in a multi-source fusion dataset.
6. The method for multi-source spatiotemporal data fusion and missing data completion based on a diffusion model according to claim 5, characterized in that, In step 3, a data quality check is performed on the multi-source fusion dataset, specifically as follows: Based on the detection method of statistical threshold and temporal stability, mutation, drift, duplication and invalid data are identified and labeled.
7. A method for multi-source spatiotemporal data fusion and missing data completion based on a diffusion model as described in claim 6, characterized in that, In step 3, the missing type is identified in the multi-source fusion dataset, specifically as follows: Based on the missing pattern classification criteria, missing data is divided into completely random missing data and conditional missing data, and a missing data mask matrix is generated.
8. The method according to claim 7, characterized in that, In step 4, based on the identification results, data missing information is filled into the multi-source fusion dataset using a data missing information completion model. Specifically: A missing data type detection and model selection mechanism is constructed. If the missing data type is completely random, a lightweight diffusion model branch is used to complete the data. If the missing type is conditional missing, then the semantically enhanced conditional diffusion model branch is used for data completion.
9. The method according to claim 8, characterized in that, The lightweight diffusion model branch uses a convolutional neural network as its core structure, combining local spatial convolution and neighborhood statistical features to quickly denoise and complete missing values.
10. The method according to claim 9, characterized in that, The semantically enhanced conditional diffusion model branch is an encoding network composed of CNN and LSTM.