A method, device and storage medium for multi-source heterogeneous data fusion in power distribution networks

Abstract

This invention relates to the field of distribution network data management technology, and discloses a method, device, and storage medium for fusing multi-source heterogeneous data from a distribution network. The invention stores the collected multi-source heterogeneous data from the distribution network into corresponding temporary tables according to their data types; establishes corresponding data preprocessing threads based on the data types in the temporary tables, and performs preset data preprocessing operations on the corresponding data. When the data type is structured data, these operations include anomaly analysis, correction of anomalies, and denoising of the corrected data; all preprocessed data are combined into a target data set, and time-series data in the set is populated using time registration and unstructured data; the correlation between data in the set is calculated, and data with correlation values ​​greater than a preset correlation threshold are fused. This invention improves the efficiency and quality of fusing multi-source heterogeneous data from a distribution network.

Description

Technical Field

[0001] This invention relates to the field of power distribution network data management technology, and in particular to a method, device and storage medium for fusion of multi-source heterogeneous data in power distribution networks. Background Technology

[0002] As the automation and informatization of power distribution network systems become increasingly apparent, massive amounts of data from various sources are generated during the operation of the power distribution network. Improper processing of power distribution network data can affect the authenticity and integrity of power system data information.

[0003] Processing distribution network data requires fusing massive amounts of data from different sources and storing the fused data for later use. However, these data come from different systems, each following inconsistent modeling standards, and the sheer volume of data results in poor fusion of multi-source heterogeneous data from the distribution network. This is detrimental to the overall operational efficiency and effectiveness of the power system, posing challenges and risks to its stable operation.

[0004] Various power information measurement devices are installed on each line in the power distribution network. These devices can acquire different types of power measurement data, such as synchronous phasor unit data, meteorological data, and equipment data. The abundance of power data generates a greater variety of noise data, increasing the difficulty of data fusion.

[0005] Currently, data fusion technologies for multi-source heterogeneous data in distribution networks perform well when the fusion dataset is small. However, as the dataset size increases, the fusion efficiency and quality decrease, failing to meet the data fusion needs of distribution networks. Summary of the Invention

[0006] This invention provides a method, device, and storage medium for fusing multi-source heterogeneous data in a power distribution network, solving the technical problem of how to improve the efficiency and quality of fusing multi-source heterogeneous data in a power distribution network.

[0007] The first aspect of this invention provides a method for fusing multi-source heterogeneous data in a power distribution network, comprising:

[0008] Collect multi-source heterogeneous data from the power distribution network;

[0009] The heterogeneous data of the power distribution network is stored in the corresponding temporary table according to the data type.

[0010] A corresponding data preprocessing thread is established based on the data type of the data in the temporary table. The data preprocessing thread performs preset data preprocessing operations on the data in the corresponding temporary table to obtain preprocessed data. When the data type is structured data, the preset data processing operations include: performing anomaly analysis on the data, correcting the abnormal data obtained from the analysis, and denoising the corrected data.

[0011] All preprocessed data are combined into a target data set, and the time-series data in the target data set is filled with data through time registration and unstructured data in the target data set;

[0012] Calculate the correlation degree between data in the target dataset;

[0013] Data with a correlation degree greater than a preset correlation degree threshold will be merged.

[0014] According to one achievable method of the first aspect of the present invention, the anomaly analysis of the data includes:

[0015] Anomaly analysis of the data is performed using a pre-set neural network model; the neural network model is trained based on the particle swarm optimization algorithm.

[0016] According to one achievable method of the first aspect of the present invention, the denoising of the data obtained from the bias correction includes:

[0017] The cross-wavelet transform algorithm is used to process the data obtained from the correction to obtain the corresponding wavelet coherence spectrum.

[0018] The isolated forest is used to randomly partition the data obtained from the correction by selecting the maximum and minimum values ​​of the features until all data points are isolated, thus obtaining the corresponding path length distribution data map.

[0019] The wavelet coherence spectrum is compared with the path length distribution data to identify noise components;

[0020] The identified noise components are then removed.

[0021] According to one achievable method of the first aspect of the present invention, the step of filling the time-series data in the target data set with time registration and unstructured data in the target data set includes:

[0022] Perform time registration on the time-series data in the target dataset to obtain the registered data;

[0023] Based on the unstructured data in the target dataset, a distribution network topology analysis is performed to obtain the distribution network topology analysis results.

[0024] Based on the analysis results of the power distribution network topology, a preset interpolation algorithm is used to fill in the registered data.

[0025] According to one achievable method of the first aspect of the present invention, the time registration of the time-series data in the target dataset includes:

[0026] The time-series data is filtered and predicted based on a Kalman filter.

[0027] The time series data is registered to a reference time based on the obtained filtered and predicted data.

[0028] According to one achievable method of the first aspect of the present invention, the analysis of the distribution network topology based on unstructured data in the target dataset includes:

[0029] Based on the unstructured data in the target dataset, a tree search algorithm is used to analyze the distribution network topology.

[0030] According to one achievable method of the first aspect of the present invention, calculating the correlation degree between data in the target dataset includes:

[0031] Principal component analysis is used to process the data in the target dataset to determine the composite data that are uncorrelated pairwise.

[0032] Calculate the correlation between data in the target dataset other than the comprehensive data.

[0033] A second aspect of the present invention provides a multi-source heterogeneous data fusion device for power distribution networks, comprising:

[0034] The data acquisition module is used to collect multi-source heterogeneous data from the power distribution network.

[0035] The storage module is used to store the multi-source heterogeneous data of the power distribution network into the corresponding temporary table according to the data type;

[0036] The preprocessing module is used to establish a corresponding data preprocessing thread based on the data type of the data in the temporary table, so that the data preprocessing thread performs preset data preprocessing operations on the data in the corresponding temporary table to obtain preprocessed data; when the data type is structured data, the preset data processing operations include: performing anomaly analysis on the data, correcting the abnormal data obtained from the analysis, and denoising the corrected data.

[0037] The filling module is used to assemble all preprocessed data into a target data set, and to fill the time-series data in the target data set with time registration and unstructured data in the target data set;

[0038] The calculation module is used to calculate the correlation between data in the target dataset;

[0039] The fusion module is used to fuse data with a correlation degree greater than a preset correlation degree threshold.

[0040] According to one achievable method of the second aspect of the present invention, when the preprocessing module performs anomaly analysis on the data, it is specifically used for:

[0041] Anomaly analysis of the data is performed using a pre-set neural network model; the neural network model is trained based on the particle swarm optimization algorithm.

[0042] According to one achievable method of the second aspect of the present invention, when the preprocessing module denoises the data obtained from the bias correction, it is specifically used for:

[0043] The cross-wavelet transform algorithm is used to process the data obtained from the correction to obtain the corresponding wavelet coherence spectrum.

[0044] The isolated forest is used to randomly partition the data obtained from the correction by selecting the maximum and minimum values ​​of the features until all data points are isolated, thus obtaining the corresponding path length distribution data map.

[0045] The wavelet coherence spectrum is compared with the path length distribution data to identify noise components;

[0046] The identified noise components are then removed.

[0047] According to one achievable embodiment of the second aspect of the invention, the filling module comprises:

[0048] A time registration unit is used to perform time registration on the time-series data in the target dataset to obtain registered data.

[0049] The topology analysis unit is used to perform distribution network topology analysis based on unstructured data in the target dataset to obtain distribution network topology analysis results.

[0050] An interpolation filling unit is used to fill the registered data with a preset interpolation algorithm based on the analysis results of the power distribution network topology.

[0051] According to one achievable method of the second aspect of the present invention, the time registration unit is specifically used for:

[0052] The time-series data is filtered and predicted based on a Kalman filter.

[0053] The time series data is registered to a reference time based on the obtained filtered and predicted data.

[0054] According to one achievable method of the second aspect of the invention, the topology analysis unit is specifically used for:

[0055] Based on the unstructured data in the target dataset, a tree search algorithm is used to analyze the distribution network topology.

[0056] According to one achievable embodiment of the second aspect of the invention, the computing module includes:

[0057] The uncorrelated data determination unit is used to process the data in the target dataset using principal component analysis to determine the comprehensive data that are uncorrelated in pairs.

[0058] The calculation unit is used to calculate the correlation between data in the target data set other than the comprehensive data.

[0059] A third aspect of the present invention provides a multi-source heterogeneous data fusion device for power distribution networks, comprising:

[0060] A memory for storing instructions; wherein the instructions are used to implement the distribution network multi-source heterogeneous data fusion method as described in any of the above embodiments;

[0061] A processor for executing instructions in the memory.

[0062] The fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for multi-source heterogeneous data fusion in a distribution network as described in any of the above embodiments.

[0063] As can be seen from the above technical solutions, the present invention has the following advantages:

[0064] This invention stores the collected heterogeneous data from multiple sources in the distribution network into corresponding temporary tables based on their data types. A corresponding data preprocessing thread is established based on the data type in the temporary tables, performing preset data preprocessing operations on the data in the corresponding temporary tables. When the data type is structured data, the preset data processing operations include anomaly analysis, correction of the anomaly data obtained from the analysis, and denoising of the corrected data. All the preprocessed data are combined into a target data set. Time registration and unstructured data in the target data set are used to populate the time-series data in the target data set. The correlation between data in the target data set is calculated, and data with correlation values ​​greater than a preset correlation threshold are fused. This invention improves the quality of data used for data fusion by performing preset data preprocessing operations on the data in the corresponding temporary tables through the data preprocessing thread. Time registration removes interference caused by inconsistent data acquisition time periods, and data population ensures consistent density of time-series data from different sources. These processes improve the overall efficiency and quality of fusing heterogeneous data from multiple sources in the distribution network. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A flowchart of a method for fusing multi-source heterogeneous data in a power distribution network, provided as an optional embodiment of the present invention;

[0067] Figure 2 The diagram below shows the structural connection of a multi-source heterogeneous data fusion device for a power distribution network, provided as an optional embodiment of the present invention.

[0068] Figure label:

[0069] 1-Acquisition module; 2-Storage module; 3-Preprocessing module; 4-Filling module; 5-Calculation module; 6-Fusion module. Detailed Implementation

[0070] This invention provides a method, apparatus, and storage medium for fusing multi-source heterogeneous data in a power distribution network, which addresses the technical problem of improving the efficiency and quality of fusing multi-source heterogeneous data in a power distribution network.

[0071] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0072] This invention provides a method for fusing multi-source heterogeneous data in a power distribution network.

[0073] Please see Figure 1 , Figure 1 The flowchart illustrates a method for fusing multi-source heterogeneous data in a power distribution network according to an embodiment of the present invention.

[0074] The present invention provides a method for fusion of multi-source heterogeneous data in a power distribution network, comprising steps S1-S6.

[0075] The first aspect of this invention provides a method for fusing multi-source heterogeneous data in a power distribution network, comprising:

[0076] Step S1: Collect multi-source heterogeneous data of the power distribution network.

[0077] Among them, the heterogeneous data of the power distribution network usually includes structured data and unstructured data. Unstructured data includes GIS (Geographic Information System) data and topology data.

[0078] When collecting multi-source heterogeneous data of the distribution network, the multi-source heterogeneous data of the distribution network can be obtained from the sensors that collect data of the distribution network.

[0079] Step S2: Store the heterogeneous data of the power distribution network into the corresponding temporary table according to the data type.

[0080] The classification and mining of multi-source heterogeneous data from the distribution network can be performed based on existing artificial intelligence technologies. In this embodiment, no specific classification method is limited.

[0081] As a specific implementation method, data types can be divided into structured data and unstructured data, and unstructured data can be further divided into geographic information data and topological data.

[0082] Step S3: Establish a corresponding data preprocessing thread based on the data type of the data in the temporary table, and have the data preprocessing thread perform preset data preprocessing operations on the data in the corresponding temporary table to obtain preprocessed data; when the data type is structured data, the preset data processing operations include: performing anomaly analysis on the data, correcting the abnormal data obtained from the analysis, and denoising the corrected data.

[0083] In this embodiment, by performing preset data preprocessing operations on the data in the corresponding temporary table through a data preprocessing thread, the data quality used for data fusion can be improved. Specifically, correcting abnormal data can yield data with uniform form and no defects, and denoising the corrected data can obtain high-precision measurement data.

[0084] In one feasible approach, unstructured data is divided into topological data and geographic information data. Pre-defined data processing operations for topological data may include: automatically generating SVG graphics (scalable vector graphics) according to the designed layout and wiring rules; pre-defined data processing operations for geographic information data may include: combining the convenience of Baidu Maps components to connect points on the map through walking navigation and line drawing functions to complete the drawing of a geographic connection map.

[0085] After correcting the abnormal data obtained from the analysis, correction records can be generated and saved to facilitate the management of the integrated data by the distribution network data management personnel.

[0086] In one feasible manner, the anomaly analysis of the data includes:

[0087] Anomaly analysis of the data is performed using a pre-set neural network model; the neural network model is trained based on the particle swarm optimization algorithm.

[0088] In this embodiment, an artificial intelligence neural network algorithm is used to process the data for anomaly data mining, which has the advantages of intelligence and convenience. As a specific implementation method, when constructing the pre-set neural network model, a neural network algorithm is first established. Then, during the algorithm feedback process, a particle swarm optimization algorithm is used to optimize the weights. Next, the sample data is standardized and feature selected, the particle swarm and related parameters are initialized, a neural network model is established through features, and the particle swarm optimization algorithm is used to optimize it to the pre-set conditions. Finally, the optimal parameters are output, and the neural network algorithm is established using the optimal parameters to achieve optimized training of the neural network model. Specifically, in the optimization training process, the topology of the neural network is first determined, then the expression of the fitness function is obtained, and finally, anomaly data mining is performed. The optimization training includes calculating the fitness of each particle using mean squared error, updating the current optimal position and global optimal position of each particle, and updating the velocity and position of each particle.

[0089] Another feasible approach is to establish threshold judgment criteria based on actual conditions, and then mine outlier data based on these criteria. In practice, the data is compared with the corresponding threshold range, and data exceeding that range is considered outlier.

[0090] In another feasible approach, since the heterogeneous data from multiple sources in the distribution network is mostly sensor data, the accuracy of the data can be assessed using sensor measurement errors, calculation errors, and empirical formula errors. Sensor measurement errors are primarily related to sensor performance and can be determined through horizontal comparisons of different sensor brands and relative deviations from the total metering output. Calculation errors are related to factors such as information acquisition time, conversion methods, and accuracy, and can be comprehensively determined based on the data calculation, storage, and transmission methods. Empirical formula errors are related to the accuracy of empirical formulas used in mathematical processing and can be determined using the relative value of the empirical formula and the actual deviation. By comparing the weighted sum of these three error values ​​with a preset error threshold, the accuracy of the corresponding data can be assessed, and inaccurate data can be identified as abnormal data. It should be noted that the calculation of sensor measurement errors, calculation errors, and empirical formula errors is existing technology; in this embodiment, the specific calculation process for sensor measurement errors, calculation errors, and empirical formula errors is not limited.

[0091] It should be noted that, in other feasible ways, when analyzing abnormal data, any two or three of the above three anomaly analysis methods can be combined to achieve the mining of abnormal data.

[0092] In one feasible approach, the denoising of the data obtained from the bias correction includes:

[0093] The cross-wavelet transform algorithm is used to process the data obtained from the correction to obtain the corresponding wavelet coherence spectrum.

[0094] The isolated forest is used to randomly partition the data obtained from the correction by selecting the maximum and minimum values ​​of the features until all data points are isolated, thus obtaining the corresponding path length distribution data map.

[0095] The wavelet coherence spectrum is compared with the path length distribution data to identify noise components;

[0096] The identified noise components are then removed.

[0097] Cross-wavelet transform, based on wavelet analysis theory, analyzes two signals in the time-frequency domain, diagnosing their time-frequency correlation and phase relationship characteristics. Noise, due to its randomness and lack of correlation, has minimal impact on signal analysis in cross-wavelet transform. Furthermore, the frequency and phase distribution characteristics of highly vibration-resistant signals are crucial information for signal identification; therefore, feature extraction methods based on cross-wavelet transform can effectively describe features and avoid the influence of noise. The isolated forest algorithm is an unsupervised anomaly detection method suitable for continuous data. Unlike other anomaly detection algorithms that characterize the degree of alienation between samples using quantitative indicators such as distance and density, the isolated forest algorithm detects outliers by isolating sample points. Compared to traditional algorithms such as LOF and K-means, the isolated forest algorithm has better robustness to high-dimensional data. In this embodiment, combining the cross-wavelet transform algorithm and the isolated forest algorithm for data denoising achieves good denoising results.

[0098] As one feasible approach, after denoising the data obtained from the correction, the signal-to-noise ratio can be further calculated and saved to facilitate the management of the fused data by distribution network data management personnel.

[0099] Step S4: All preprocessed data are combined into a target data set, and the time-series data in the target data set is filled with data through time registration and unstructured data in the target data set.

[0100] In one feasible approach, the data filling of the time-series data in the target dataset through time registration and unstructured data in the target dataset includes:

[0101] Perform time registration on the time-series data in the target dataset to obtain the registered data;

[0102] Based on the unstructured data in the target dataset, a distribution network topology analysis is performed to obtain the distribution network topology analysis results.

[0103] Based on the analysis results of the power distribution network topology, a preset interpolation algorithm is used to fill in the registered data.

[0104] In one feasible approach, the time registration of the time-series data in the target dataset includes:

[0105] The time-series data is filtered and predicted based on a Kalman filter.

[0106] The time series data is registered to a reference time based on the obtained filtered and predicted data.

[0107] In this embodiment, time registration employs a time registration tracking method. First, a recursive calculation is performed using the fusion center's moderate period T as the data processing cycle. Then, prediction and estimation are performed using the arrival time of each data point as a timestamp, establishing a confidence interval and filtering related points. Specifically, the process includes: first, initializing the Kalman filter, calculating the time series data state and covariance prediction matrix, gain matrix, updated state equation, and covariance equation; then, determining whether the last data point in the distribution network time series data corresponds to the current time. If it does, the next time point's distribution network data is predicted; otherwise, the updated state equation and covariance equation for the distribution network are recalculated.

[0108] In this embodiment, time registration of time-series data based on Kalman filtering can remove interference caused by inconsistent data acquisition time periods.

[0109] In one feasible approach, the distribution network topology analysis based on unstructured data in the target dataset includes:

[0110] Based on the unstructured data in the target dataset, a tree search algorithm is used to analyze the distribution network topology.

[0111] The topology represents the connection relationship of all devices in the distribution network. Each node in the network is connected to two neighboring nodes along the current direction. The topology analysis process involves adjusting network nodes based on the opening and closing of tie switches to reconstruct the network when changes occur. A tree search algorithm is used as the topology analysis algorithm, and the process is as follows: Starting from the substation, search for adjacent unmarked device nodes along the outgoing line. Then, search for unmarked device nodes connected by closed switches in a top-down and left-to-right order until all nodes are searched, thus completing the topology analysis and obtaining a schematic diagram of the topology.

[0112] As a specific implementation, the tree search algorithm includes a selection, expansion, simulation, backpropagation, and final selection process, comprising the following steps:

[0113] With the power plant as the root node Z, the equipment and line nodes distributed along the power distribution lines are child nodes s1, s2, ..., sn, and the number of times each child node is visited is X. s1 ,X s2 ,...,X sn The importance coefficient of the sub-node to the distribution network is μ. s1 ,μ s2 ,...,μ sn The selection strategy for child nodes is derived based on their visit frequency and importance. The selection strategy formula is: F(sn) = X sn -μ sn ;

[0114] When node sn is a scalable node, the search algorithm randomly assigns a topology distribution set from the distribution network space. The topology distribution set is then simulated and marked. When the node is a fully visited node, the search algorithm adds new actions to the node's topology distribution.

[0115] After the expansion process is completed, the search algorithm moves randomly until the terminal node using a default strategy. Based on the importance coefficient of the terminal node and the selection strategy, it calculates the importance coefficient of all nodes on the path from node si to node sj and related information of their associated nodes.

[0116] After thorough simulation, the unmarked device nodes connected by closed switches are searched in a top-down, left-to-right order until all nodes are searched, completing the topology analysis and obtaining a schematic diagram of the topology. The search algorithm selects the best action from the root node to obtain the topology of the distribution network.

[0117] In the above embodiments of the present invention, the distribution network topology analysis is performed by a tree search algorithm, which can quickly and accurately obtain the distribution network topology, thereby improving the overall data fusion efficiency.

[0118] In one feasible implementation, the preset interpolation algorithm is Kriging interpolation.

[0119] Kriging interpolation, also known as spatial local interpolation, is a method based on variogram theory and structural analysis to perform unbiased optimal estimation of regionalized variables within a finite region.

[0120] It should be noted that other existing interpolation algorithms, such as the Lagrange interpolation algorithm, can also be used for data filling.

[0121] In the above embodiments of the present invention, data filling can ensure that the density of time-series data from different sources is consistent.

[0122] Step S5: Calculate the correlation between data in the target dataset.

[0123] In one feasible approach, calculating the correlation between data in the target dataset includes:

[0124] Principal component analysis is used to process the data in the target dataset to determine the composite data that are uncorrelated pairwise.

[0125] Calculate the correlation between data in the target dataset other than the comprehensive data.

[0126] The process of using principal component analysis to process the data in the target dataset to determine pairwise uncorrelated composite data includes the following steps:

[0127] To calculate the correlation coefficient matrix, first use the correlation coefficient function to obtain the correlation coefficient between the data, then form an n×m symmetric matrix from the correlation coefficients, and use the matrix to determine the magnitude of the correlation coefficient.

[0128] Find the characteristic values ​​of the data, use the Jacobi method to find the characteristic values ​​according to the characteristic equation, and then find the corresponding eigenvectors ei (i = 1, 2, 3, ..., n) according to the characteristic values;

[0129] Determine the number m of principal components and set the information utilization rate to be greater than 80%.

[0130] As one implementation method, the correlation coefficient function is:

[0131]

[0132] In the formula, r(x,y) represents the correlation coefficient between data x and data y, Cov(x,y) is the covariance between data x and data y, Var(x) is the variance of data x, and Var(y) is the variance of data y.

[0133] Step S6: Merge the data corresponding to the correlation degree that is greater than the preset correlation degree threshold.

[0134] The preset correlation threshold can be set according to the actual situation.

[0135] Furthermore, the method may also include:

[0136] The merged data is stored in a MySQL database to await calls from the power distribution network functional modules.

[0137] In the above embodiments of the present invention, cached data is obtained from sensors collecting data from the distribution network. Then, through the steps of classification, anomaly data mining, correction and denoising, corrected registration, and secondary fusion, a database suitable for the distribution network system is obtained. Specifically, an artificial intelligence neural network algorithm is used to process the data, achieving the function of mining anomaly data. Through the joint algorithm of cross wavelet transform and isolated forest, heterogeneous data is denoised to obtain high-quality data. Finally, time registration is used to remove interference caused by inconsistent data collection time periods. The registered data is filled with Kriging interpolation based on the distribution network topology to ensure consistent density of time-series data from different sources. Finally, the correlation between data is calculated through data correlation, and data with high correlation are fused. Overall, this can improve the efficiency and quality of fusing multi-source heterogeneous data from the distribution network.

[0138] The present invention also provides a distribution network multi-source heterogeneous data fusion device, which can be used to execute the distribution network multi-source heterogeneous data fusion method described in any of the above embodiments of the present invention.

[0139] Please see Figure 2 , Figure 2 The diagram shows a structural connection block diagram of a multi-source heterogeneous data fusion device for power distribution networks provided in an embodiment of the present invention.

[0140] This invention provides a multi-source heterogeneous data fusion device for power distribution networks, comprising:

[0141] Acquisition module 1 is used to collect multi-source heterogeneous data from the power distribution network;

[0142] Storage module 2 is used to store the multi-source heterogeneous data of the power distribution network into the corresponding temporary table according to the data type;

[0143] Preprocessing module 3 is used to establish a corresponding data preprocessing thread according to the data type of the data in the temporary table, so that the data preprocessing thread performs preset data preprocessing operations on the data in the corresponding temporary table to obtain preprocessed data; when the data type is structured data, the preset data processing operations include: performing anomaly analysis on the data, correcting the abnormal data obtained from the analysis, and denoising the corrected data.

[0144] Filling module 4 is used to assemble all preprocessed data into a target data set, and fill the time series data in the target data set with time registration and unstructured data in the target data set;

[0145] Calculation module 5 is used to calculate the correlation between data in the target dataset;

[0146] The fusion module 6 is used to fuse data with a correlation degree greater than a preset correlation degree threshold.

[0147] In one feasible implementation, when the preprocessing module 3 performs anomaly analysis on the data, it is specifically used for:

[0148] Anomaly analysis of the data is performed using a pre-set neural network model; the neural network model is trained based on the particle swarm optimization algorithm.

[0149] In one feasible implementation, when the preprocessing module 3 denoises the data obtained from the bias correction, it is specifically used for:

[0150] The cross-wavelet transform algorithm is used to process the data obtained from the correction to obtain the corresponding wavelet coherence spectrum.

[0151] The isolated forest is used to randomly partition the data obtained from the correction by selecting the maximum and minimum values ​​of the features until all data points are isolated, thus obtaining the corresponding path length distribution data map.

[0152] The wavelet coherence spectrum is compared with the path length distribution data to identify noise components;

[0153] The identified noise components are then removed.

[0154] In one feasible implementation, the filling module 4 includes:

[0155] A time registration unit is used to perform time registration on the time-series data in the target dataset to obtain registered data.

[0156] The topology analysis unit is used to perform distribution network topology analysis based on unstructured data in the target dataset to obtain distribution network topology analysis results.

[0157] An interpolation filling unit is used to fill the registered data with a preset interpolation algorithm based on the analysis results of the power distribution network topology.

[0158] In one feasible implementation, the time registration unit is specifically used for:

[0159] The time-series data is filtered and predicted based on a Kalman filter.

[0160] The time series data is registered to a reference time based on the obtained filtered and predicted data.

[0161] In one feasible implementation, the topology analysis unit is specifically used for:

[0162] Based on the unstructured data in the target dataset, a tree search algorithm is used to analyze the distribution network topology.

[0163] In one feasible implementation, the computing module 5 includes:

[0164] The uncorrelated data determination unit is used to process the data in the target dataset using principal component analysis to determine the comprehensive data that are uncorrelated in pairs.

[0165] The calculation unit is used to calculate the correlation between data in the target data set other than the comprehensive data.

[0166] The present invention also provides a multi-source heterogeneous data fusion device for power distribution networks, comprising:

[0167] A memory is provided for storing instructions; wherein the instructions are used to implement the multi-source heterogeneous data fusion method for power distribution networks as described in any of the above embodiments;

[0168] A processor for executing instructions in the memory.

[0169] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for fusion of multi-source heterogeneous data in a distribution network as described in any of the above embodiments.

[0170] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and the specific beneficial effects of the above-described device and module can be referred to the corresponding beneficial effects in the foregoing method embodiments, and will not be repeated here.

[0171] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or modules may be electrical, mechanical, or other forms.

[0172] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0173] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0174] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A power distribution network multi-source heterogeneous data fusion method, characterized in that, The method comprises the following steps: Collecting multi-source heterogeneous data of a power distribution network; The multi-source heterogeneous data of the power distribution network is obtained from sensors collecting data of the power distribution network; According to the data type, the multi-source heterogeneous data of the power distribution network is stored in a corresponding temporary table; According to the data type of the data in the temporary table, a corresponding data preprocessing thread is established, and the data preprocessing thread performs a preset data preprocessing operation on the data in the corresponding temporary table to obtain preprocessed data; When the data type is structured data, the preset data processing operation includes: performing anomaly analysis on the data, correcting the abnormal data obtained by the analysis, and denoising the data obtained by the correction; All preprocessed data is combined into a target data set, and time registration and unstructured data in the target data set are used to fill in the time series data in the target data set; Calculate the correlation degree between the data in the target data set; The data corresponding to the correlation degree greater than the preset correlation degree threshold is fused. The data filling of the time series data in the target data set through time registration and unstructured data in the target data set comprises: Time registration is performed on the time series data in the target data set to obtain registered data; Based on the unstructured data in the target data set, the topology structure of the power distribution network is analyzed to obtain the topology structure analysis result of the power distribution network; Based on the topology structure analysis result of the power distribution network, the registered data is filled in by using a preset interpolation algorithm; The time registration of the time series data in the target data set comprises: Filtering and data prediction are performed on the time series data based on a Kalman filter; The time series data is registered to a reference time according to the obtained filtered data and predicted data; Based on the unstructured data in the target data set, the topology structure of the power distribution network is analyzed by using a tree search algorithm. The anomaly analysis of the data comprises:

2. The power distribution network multi-source heterogeneous data fusion method according to claim 1, characterized in that, A preset neural network model is used to analyze the anomaly of the data; the neural network model is trained based on a particle swarm algorithm. The denoising of the data obtained by correction comprises:

3. The power distribution network multi-source heterogeneous data fusion method according to claim 1, characterized in that, A cross wavelet transform algorithm is used to process the data obtained by correction to obtain a corresponding wavelet coherence spectrum; An isolation forest algorithm is used to randomly partition the data obtained by correction by selecting the maximum and minimum values of the features until all data points are isolated to obtain a corresponding path length distribution data graph; The wavelet coherence spectrum and the path length distribution data graph are compared to identify noise components; The identified noise components are removed. The calculation of the correlation degree between the data in the target data set comprises:

4. The power distribution network multi-source heterogeneous data fusion method according to claim 1, characterized in that, Principal component analysis is used to process the data in the target data set to determine two-by-two uncorrelated comprehensive data; The correlation degree between the data in the target data set except the comprehensive data is calculated. The method comprises the following steps:

5. A power distribution network multi-source heterogeneous data fusion device, characterized in that, A collecting module is used to collect multi-source heterogeneous data of a power distribution network; The multi-source heterogeneous data of the power distribution network is obtained from sensors collecting data of the power distribution network; ​ The storage module is configured to store the power distribution network multi-source heterogeneous data into corresponding temporary tables according to data types; The preprocessing module is configured to establish corresponding data preprocessing threads according to data types of data in the temporary tables, and to make the data preprocessing threads perform preset data preprocessing operations on the data in the corresponding temporary tables to obtain preprocessed data; When the data types are structured data, the preset data preprocessing operations include: performing abnormality analysis on the data, performing rectification on the abnormal data obtained by the analysis, and performing denoising on the data obtained by the rectification; The filling module is configured to group all the preprocessed data into a target data set, and to perform data filling on time series data in the target data set by time registration and unstructured data in the target data set; The calculation module is configured to calculate correlation degrees between data in the target data set; The fusion module is configured to fuse data corresponding to correlation degrees greater than a preset correlation degree threshold; The filling module includes: The time registration unit is configured to perform time registration on time series data in the target data set to obtain registered data; The topology analysis unit is configured to perform power distribution network topology structure analysis based on unstructured data in the target data set to obtain power distribution network topology structure analysis results; The interpolation filling unit is configured to perform data filling on the registered data based on the power distribution network topology structure analysis results by using a preset interpolation algorithm. The time registration unit is specifically configured to: Perform filtering processing and data prediction on the time series data based on a Kalman filter; Register the time series data to a reference time according to obtained filtered data and predicted data; The topology analysis unit is specifically configured to: Perform power distribution network topology structure analysis based on the unstructured data in the target data set by using a tree search algorithm.

6. A power distribution network multi-source heterogeneous data fusion device, characterized in that, The memory is configured to store instructions; wherein the instructions are used to implement the power distribution network multi-source heterogeneous data fusion method according to any one of claims 1-4; The processor is configured to execute the instructions in the memory. The computer program stored on the computer readable storage medium is executed by the processor to implement the power distribution network multi-source heterogeneous data fusion method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, ​

Images