Power distribution network data resource fusion method, device and equipment and storage medium
By calculating the correlation coefficient using a multiple linear regression model, obtaining the fitted value of abnormal data, and replacing the abnormal data, the problem of inaccurate fusion results in the data resource fusion of the power distribution network is solved, and higher precision data fusion is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GLOBAL ENERGY INTERCONNECTION RES INST CO LTD
- Filing Date
- 2023-03-29
- Publication Date
- 2026-05-29
Smart Images
Figure CN116340883B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of resource fusion technology, and in particular to a method, apparatus, equipment and storage medium for data resource fusion in power distribution networks. Background Technology
[0002] Currently, distribution network business systems and platforms are integrating increasingly more and richer data resources. The accelerated formation of new distribution networks has intensified this evolutionary trend, and the degree of data resource sharing and integration determines the level of business development and management improvement of the distribution network. With the proposal of the strategy to build a world-class power grid, more urgent needs and higher requirements have been placed on data integration and sharing.
[0003] Data fusion in power distribution networks presents certain unique challenges. During the IT infrastructure development process, various power companies, influenced by the phased nature of their business system construction and implementation, technical aspects, and other economic and human factors, have resulted in significant "information silos" in their data resources. Firstly, there is data heterogeneity: different business systems use different database systems, leading to inconsistent information coding, technical specifications, and data formats, making data conversion and fusion difficult. Secondly, there is a lack of a unified standard system. Currently, the power industry has not adopted a unified standard system for IT development, resulting in low integration and poor interconnectivity between information systems, hindering collaborative business operations. Therefore, data anomalies are prone to occur during the data fusion process in power distribution networks, leading to inaccurate fusion results. Summary of the Invention
[0004] In view of this, embodiments of the present invention provide a method, apparatus, device and storage medium for data resource fusion in power distribution networks, in order to solve the technical problem of inaccurate fusion results when data distribution networks are fused for resource fusion.
[0005] The technical solution proposed in this invention is as follows:
[0006] The first aspect of this invention provides a method for data resource fusion in a power distribution network, comprising: fusing data from at least two datasets to be fused to obtain an initial data fusion result set; acquiring abnormal data and corresponding abnormal attributes based on the initial data fusion result set; constructing a sample dataset based on all attributes contained in the initial data fusion result set and normal data corresponding to each attribute; calculating a correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset based on the sample dataset and the abnormal attribute; calculating a first fitted value of the abnormal data based on the initial data fusion result set and the correlation coefficient; and replacing the abnormal data in the initial data fusion result set with the first fitted value of the abnormal data to obtain an updated data fusion result set.
[0007] Optionally, calculating the correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset based on the sample dataset and the abnormal attribute includes: inputting the sample dataset and the abnormal attribute into a multiple linear regression model, obtaining the linear correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset through the multiple linear regression model, and using the linear correlation coefficient as the correlation coefficient.
[0008] Optionally, obtaining the linear correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset through a multiple linear regression model includes: calculating a second fitted value of the data corresponding to the abnormal attribute in the sample dataset based on a preset linear coefficient and the sample dataset; extracting the true value of the data corresponding to the abnormal attribute from the sample dataset; and using the preset linear coefficient corresponding to the minimum sum of variances between the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset as the linear correlation coefficient.
[0009] Optionally, the step of using the preset linear coefficient corresponding to the minimum sum of variances of the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset as the linear correlation coefficient includes: constructing an expression for the sum of variances of the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset; taking the derivative of the expression, using the preset linear coefficient corresponding to the minimum sum of variances as the preset linear coefficient corresponding to the minimum sum of variances, and using the preset linear coefficient corresponding to the minimum sum of variances as the linear correlation coefficient.
[0010] Optionally, abnormal data can be obtained based on the initial data fusion result set, including: sequentially determining whether each data in the initial data fusion result set belongs to a set abnormal threshold range; if it belongs to the set abnormal threshold range, the corresponding data is considered to be abnormal data.
[0011] Optionally, before calculating the correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset based on the sample dataset and the abnormal attribute, the method further includes: determining whether the number of data groups in the sample dataset is less than the number of attributes in the initial data fusion result set; if it is less than the number of attributes in the initial data fusion result set, then reconstructing the sample dataset until the number of data groups in the sample dataset is greater than or equal to the number of attributes in the initial data fusion result set.
[0012] Optionally, calculating the first fitted value of the abnormal data based on the initial data fusion result set and the correlation coefficient includes: multiplying the initial data fusion result set and the correlation coefficient to obtain the first fitted value of the abnormal data.
[0013] A second aspect of this invention provides a power distribution network data resource fusion device, comprising: an initial fusion module, configured to fuse data from at least two datasets to be fused to obtain an initial data fusion result set; an abnormal data acquisition module, configured to acquire abnormal data and corresponding abnormal attributes based on the initial data fusion result set; a sample construction module, configured to construct a sample dataset based on a complete historical data fusion result set; a correlation coefficient calculation module, configured to calculate a correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset based on the sample dataset and the abnormal attributes; a fitting value calculation module, configured to calculate a first fitting value of the abnormal data based on the initial data fusion result set and the correlation coefficient; and an update module, configured to replace the abnormal data in the initial data fusion result set with the first fitting value of the abnormal data to obtain an updated data fusion result set.
[0014] Optionally, the correlation coefficient calculation module includes: an input module for inputting the sample dataset and the abnormal attribute into a multiple linear regression model; and a linear regression module for obtaining the linear correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset through the multiple linear regression model, and using the linear correlation coefficient as the correlation coefficient.
[0015] Optionally, the linear regression module includes: a first acquisition module, configured to calculate a second fitted value of the data corresponding to the abnormal attribute in the sample dataset based on a preset linear coefficient and the sample dataset; an extraction module, configured to extract the true value of the data corresponding to the abnormal attribute from the sample dataset; and a second acquisition module, configured to use the preset linear coefficient corresponding to the minimum sum of variances between the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset as the linear correlation coefficient.
[0016] Optionally, the second acquisition module includes: a construction module, configured to construct an expression for the sum of the variances of the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset; and a differentiation module, configured to differentiate the expression, take the preset linear coefficient corresponding to the time when the derivative is zero as the preset linear coefficient corresponding to the time when the sum of variances is minimized, and take the preset linear coefficient corresponding to the time when the sum of variances is minimized as the linear correlation coefficient.
[0017] Optionally, the abnormal data acquisition module includes: a first judgment module, used to sequentially judge whether each data in the initial data fusion result set belongs to the set abnormal threshold range; if it belongs to the set abnormal threshold range, the corresponding data is considered to be abnormal data.
[0018] Optionally, the power distribution network data resource fusion device further includes: a second judgment module, used to judge whether the number of data groups in the sample dataset is less than the number of attributes in the initial data fusion result set; if it is less than the number of attributes in the initial data fusion result set, the sample dataset is reconstructed until the number of data groups in the sample dataset is greater than or equal to the number of attributes in the initial data fusion result set.
[0019] Optionally, the fitted value calculation module includes a multiplication module, used to multiply the initial data fusion result set and the correlation coefficient to obtain the first fitted value of the abnormal data.
[0020] A third aspect of the present invention provides an electronic device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the power distribution network data resource fusion method as described in any of the first aspects of the present invention.
[0021] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the power distribution network data resource fusion method as described in any of the first aspects of the present invention.
[0022] As can be seen from the above technical solutions, the embodiments of the present invention have the following advantages:
[0023] This invention provides a method, apparatus, device, and storage medium for data resource fusion in a power distribution network. The method involves fusing data from at least two datasets to obtain an initial data fusion result set; acquiring abnormal data and corresponding abnormal attributes based on the initial data fusion result set; constructing a sample dataset based on all attributes included in the initial data fusion result set and the normal data corresponding to each attribute; calculating the correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset based on the sample dataset and the abnormal attribute; calculating a first fitted value for the abnormal data based on the initial data fusion result set and the correlation coefficient; and replacing the abnormal data in the initial data fusion result set with the first fitted value of the abnormal data to obtain an updated data fusion result set. This invention predicts the abnormal data to obtain the first fitted value. Compared to the abnormal data, the first fitted value predicted using the correlation coefficient between data is closer to the correct data. Therefore, replacing the abnormal data with the predicted first fitted value makes the updated data fusion result set more accurate. Attached Figure Description
[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a flowchart of the power distribution network data resource fusion method in an embodiment of the present invention;
[0026] Figure 2 This is a schematic diagram illustrating the data resource fusion implementation process in an embodiment of the present invention;
[0027] Figure 3 This is a schematic diagram of the structure of the power distribution network data resource fusion device in an embodiment of the present invention;
[0028] Figure 4 This is a schematic diagram of the structure of the electronic device in an embodiment of the present invention;
[0029] Figure 5 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Detailed Implementation
[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0031] This invention provides a method for data resource fusion in a power distribution network, such as... Figure 1 As shown, the method includes:
[0032] Step S100: Merge the data from at least two datasets to be merged to obtain an initial data fusion result set. Specifically, data resources from different business systems are merged as different datasets to be merged. The number of datasets to be merged can be 2, 3, 4, etc. For example, the number of datasets to be merged is 2. Taking the fusion of measurement data and equipment asset data of new energy equipment in a new power distribution network as an example, the measurement data includes the power consumption data of the equipment, mainly the voltage, current, and other data of user power consumption collected from the equipment. The equipment asset data includes the basic file information of the equipment. Each measurement data and equipment asset dataset is independent of each other and has a strong linear correlation. Assuming that the measurement dataset has j attributes and the equipment asset dataset has k attributes, as shown in Table 1, the measurement dataset is defined as [Mea1,Mea2,...,Mea...]. j The equipment asset dataset is [Ea1, Ea2, ..., Ea...]. k The two datasets are then linked by device number.
[0033] Table 1 Measurement Data and Equipment Asset Data
[0034]
[0035]
[0036] After fusing the measurement dataset and the equipment asset dataset, an initial data fusion result set is obtained:
[0037] [Mea1,Mea2,...,Mea i-1 ,...,Mea i+1 Mea j ,Ea1,Ea2,...,Ea k ]
[0038] For ease of representation, as shown in Table 2, the initial data fusion result set is redefined as P. 初始 =[P1,P2,...,P m ].
[0039] Table 2 Redefining Data Columns
[0040] <![CDATA[P1]]> <![CDATA[P2]]> ... Q ... <![CDATA[P j-1 ]]> <![CDATA[P j ]]> <![CDATA[P j+1 ]]> ... <![CDATA[P m ]]> 3.15 35 ... ？ ... 20 102 1 ... 0.979 1.01 45 ... ？ ... 10 104 2 ... 0.993 2.01 32 ... ？ ... 13 112 3 ... 0.985 0.99 21 ... ？ ... 15 103 1 ... 0.994 0.71 32 ... ？ ... 27 113 2 ... 0.998 1.32 27 ... ？ ... 21 99 3 ... 0.983 1.09 30 ... ？ ... 18 112 2 ... 0.993 1.12 21 ... ？ ... 12 101 3 ... 0.991 1.21 32 ... ？ ... 11 99 2 ... 0.990 1.89 40 ... ？ ... 23 100 2 ... 0.981 ... ... ... ... ... ... ... ... ... ...
[0041] Step S200: Based on the initial data fusion result set, obtain the abnormal data and the corresponding abnormal attributes. Specifically, abnormal data includes missing data or values that deviate significantly from normal values. Therefore, each data point in the initial data fusion result can be verified to determine whether the data is abnormal, thereby finding the abnormal data and its corresponding abnormal attributes. For example, Mea in Table 1...i The corresponding column of data shows 99, while the normal value fluctuates between 20 and 50. Therefore, Mea... i The corresponding column is defined as the anomaly set, as shown in Table 2. After redefinition, let P be... i The abnormal data in the dataset is the abnormal set Q, where i represents the sequence number of the abnormal set Q in the data fusion result set. The abnormal attributes are the column names in the measurement dataset and the equipment asset dataset. For example, the abnormal attributes are current, voltage, etc.
[0042] Step S300: Construct a sample dataset based on all attributes contained in the initial data fusion result set and the normal data corresponding to each attribute. Specifically, define the sample dataset as P = [P1, P2, ..., P...]. m The sample dataset contains all attributes from the initial data fusion result set, and the data corresponding to each attribute is normal data. That is, the sample dataset is a data fusion result set without abnormal data after fusion. Therefore, the data fusion result set without abnormal data can be obtained from the historical fusion data and used to construct the sample dataset.
[0043] Step S400: Calculate the correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset based on the sample dataset and the abnormal attributes. Specifically, since measurement data have a strong linear correlation, the data in the outlier set Q has a correlation coefficient with the data in the sample dataset under normal data conditions. First, obtain the abnormal attribute P corresponding to the outlier set Q. i Then obtain the attribute P in the sample dataset. i The data, i.e., the data corresponding to the abnormal attributes in the sample dataset, is denoted as the sample set Q. 样本 The anomaly set Q and the sample set Q 样本 All are attribute P i The corresponding data sets differ in that the data in the outlier set Q is from the initial data fusion result set, which was identified as outlier data in step S200, while the data in the sample set Q... 样本 The data in the sample dataset contains attribute P. i The corresponding data is normal data. This is achieved through analysis of the sample set Q. 样本 The correlation coefficient is obtained by analyzing the relationship between the data and the data in the sample dataset P.
[0044] Step S500: Calculate the first fitted value of the outlier data based on the initial data fusion result set and the correlation coefficient. Specifically, the correlation coefficient between the initial data fusion result set and the normal data corresponding to the outlier set Q is the same as the correlation coefficient calculated in step S400. Therefore, the predicted value of the normal data corresponding to the outlier set Q, i.e., the first fitted value, can be calculated based on the initial data fusion result set and the aforementioned correlation coefficient.
[0045] Step S600: Replace the outlier data in the initial data fusion result set with the first fitted value of the outlier data to obtain an updated data fusion result set. Specifically, after obtaining the updated data fusion result set, it is stored in the data fusion library. The first fitted value is the predicted value obtained by predicting the normal data corresponding to the outlier set Q based on the correlation coefficient and the initial data fusion result set. By re-predicting the first fitted value of the outlier data through the correlation between data, it can better reflect the true value of the data compared to the outlier data that appears during the fusion process.
[0046] This invention provides a data resource fusion method for power distribution networks. The method involves fusing data from at least two datasets to be fused to obtain an initial data fusion result set. Based on the initial data fusion result set, abnormal data and their corresponding abnormal attributes are obtained. A sample dataset is constructed based on all attributes included in the initial data fusion result set and the normal data corresponding to each attribute. The correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset is calculated according to the sample dataset and the abnormal attributes. A first fitted value for the abnormal data is calculated based on the initial data fusion result set and the correlation coefficient. The abnormal data in the initial data fusion result set is replaced with the first fitted value of the abnormal data to obtain an updated data fusion result set. This invention predicts the abnormal data to obtain the first fitted value. Compared to the abnormal data, the first fitted value predicted using the correlation coefficient between data is closer to the correct data. Therefore, replacing the abnormal data with the predicted first fitted value makes the updated data fusion result set more accurate.
[0047] In one embodiment, step S400, calculating the correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset based on the sample dataset and the abnormal attributes, includes:
[0048] Step S410: Input the sample dataset and outlier attributes into the multiple linear regression model;
[0049] Step S420: Obtain the linear correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset through a multiple linear regression model, and use the linear correlation coefficient as the association coefficient.
[0050] Given the strong linear correlation among some data in distribution network data resources, such as measurement data, abnormal data are analyzed. Specifically, for example... Figure 2 As shown, by using a multiple linear regression model to analyze the data corresponding to abnormal attributes in the sample dataset and the sample dataset, data correlation characteristics are discovered from heterogeneous data resources, and the correlation coefficient, i.e., the linear correlation coefficient, is found. This satisfies the data fusion needs, provides technical support for the fusion and sharing of distribution network data resources, and realizes the enhancement of distribution network data value and the deepening of business applications.
[0051] In one embodiment, step S420, obtaining the linear correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset through a multiple linear regression model, includes:
[0052] Step S421: Calculate the second fitted value of the data corresponding to the abnormal attributes in the sample dataset based on the preset linear coefficients and the sample dataset;
[0053] Step S422: Extract the true values of the data corresponding to the abnormal attributes from the sample dataset;
[0054] Step S423: The preset linear coefficient corresponding to the minimum sum of variances between the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset is taken as the linear correlation coefficient.
[0055] Specifically, there is a linear relationship between the data corresponding to the outlier attributes in the sample dataset and the sample dataset itself. Therefore, the sample set Q, which consists of the data corresponding to the outlier attributes in the sample dataset, is... 样本 The linear relationship between Q and the sample dataset P is expressed as: Q = WP, Q = W1P1 + W2P2 + W3P3 + ... + W m P m +b, where b is W0, W0 = 1, and W is a preset linear coefficient. Assuming the sample dataset P has n sets of data, the corresponding relationship is:
[0056]
[0057] The relationship between Q, W, and P can be described using matrices:
[0058]
[0059] Substitute the data from the sample dataset P into formula (2) to calculate the second fitted value. Then calculate the second fitted value. The sample set Q corresponding to the abnormal data 样本 The sum of variances between the true values when When the value is minimized, the corresponding W vector is the optimal preset linear coefficient, which is used as the final linear correlation coefficient. Specifically, when calculating the second fitted value, W is first initialized, that is, W is given an initial value; W and the sample dataset P are substituted into formula (2) to calculate the second fitted value. Calculate the second fitted value The sum of variances between the true value and the actual value is saved; W is updated, that is, W is reassigned, and the above steps are repeated based on the updated W to obtain a new sum of variances. The number of repetitions is set according to the requirements. The magnitude of all the sums of variances is compared and the minimum value is found. The W corresponding to the minimum sum of variances is obtained and used as the final linear correlation coefficient.
[0060] In one embodiment, step S423, obtaining the preset linear coefficient as the linear correlation coefficient when the sum of the variances of the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset is minimized, includes:
[0061] Step S4231: Construct an expression for the sum of the variances of the second fitted value and the true value of the data corresponding to the abnormal attributes in the sample dataset;
[0062] Step S4232: Differentiate the expression, take the preset linear coefficient corresponding to the minimum sum of variances as the preset linear coefficient corresponding to the minimum sum of variances, and take the preset linear coefficient corresponding to the minimum sum of variances as the linear correlation coefficient.
[0063] Specifically, construct the expression for the sum of variances: Then β=(Q-WP) 2 To find the vector W corresponding to the minimum value of β, first take the derivative of W. When the derivative is 0, β is at its minimum, and the corresponding W is the linear correlation coefficient we are looking for. For example, let the derivative... Solving for 2P T PW = 2P T Q, then W = (P) T P) -1 P T Q.
[0064] By using the linear correlation coefficient in the linear relationship where the derivative is zero as the linear correlation coefficient in the linear relationship where the sum of variances is minimized, the cost of calculating the linear correlation coefficient is reduced.
[0065] In one embodiment, step S200, obtaining abnormal data based on the initial data fusion result set, includes:
[0066] The system sequentially checks whether each data point in the initial data fusion result set falls within the set abnormal threshold range. If a data point falls within the set abnormal threshold range, the corresponding data is considered abnormal.
[0067] Specifically, based on the anomalies in some data within the initial data fusion result set—including missing data or significant deviations from normal values—each data point in the fused initial data fusion result can be verified to determine its anomaly, thereby identifying anomalous data and their corresponding anomalous attributes. Missing data is replaced with a specified value or character in the initial data fusion result set, such as the character "null" or the value 999. The anomaly threshold range includes this specified value or character. For example, an anomaly threshold range greater than 100 or equal to "null" indicates data anomaly.
[0068] In one embodiment, step S500, calculating the first fitted value of the outlier data based on the initial data fusion result set and correlation coefficient, includes:
[0069] The initial data fusion result set is multiplied by the correlation coefficient to obtain the first fitted value of the outlier data.
[0070] Specifically, the initial data fusion result set and correlation coefficient are substituted into the linear relationship Q=WP, that is, the initial data fusion result set and correlation coefficient are multiplied to obtain the corresponding first fitted value. When multiplying the initial data fusion result set and correlation coefficient, the values of abnormal data in the initial data fusion result set are replaced with a set average value to avoid the influence of excessive deviation of abnormal data on the first fitted value.
[0071] After the first fitted value is calculated, its reasonableness needs to be verified manually. Only after the verification is passed can the next step be carried out to avoid abnormal results of the first fitted value.
[0072] In one embodiment, before calculating the correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset based on the sample dataset and the abnormal attributes, the power distribution network data resource fusion method further includes:
[0073] Determine whether the number of data groups in the sample dataset is less than the number of attributes in the initial data fusion result set;
[0074] If the number of attributes in the sample dataset is less than the number of attributes in the initial data fusion result set, the sample dataset is reconstructed until the number of data groups in the sample dataset is greater than or equal to the number of attributes in the initial data fusion result set.
[0075] Specifically, the number of data sets is the number of data entries, and a data set is a row of data in Table 1 or Table 2. For example, if the number of data sets n in the sample dataset is less than the number of data attributes m, the sample dataset is reconstructed. If the number of data entries n in the sample dataset is greater than or equal to the number of data attributes m, the correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset is calculated based on the sample dataset and the abnormal attributes. When the number of data sets n in the sample dataset is less than the number of data attributes m, formula (2) cannot be valid because the amount of data is too small. Therefore, the number of data entries n in the sample dataset needs to be greater than or equal to the number of data attribute classes m in order to obtain the correlation coefficient by formula (2).
[0076] This invention also provides a distribution network data resource fusion device, such as... Figure 3 As shown, the power distribution network data resource fusion device includes:
[0077] The initial fusion module 301 is used to perform resource fusion on the data of at least two datasets to be fused to obtain an initial data fusion result set; for details, please refer to the corresponding part of the above method embodiment, which will not be repeated here.
[0078] The abnormal data acquisition module 302 is used to acquire abnormal data and the abnormal attributes corresponding to the abnormal data based on the initial data fusion result set; for details, please refer to the corresponding part of the above method embodiment, which will not be repeated here.
[0079] The sample construction module 303 is used to construct a sample dataset based on the complete historical data fusion result set; for details, please refer to the corresponding part of the above method embodiment, which will not be repeated here.
[0080] The correlation coefficient calculation module 304 is used to calculate the correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset based on the sample dataset and the abnormal attributes; for details, please refer to the corresponding part of the above method embodiment, which will not be repeated here.
[0081] The fitting value calculation module 305 is used to calculate the first fitting value of the abnormal data based on the initial data fusion result set and the correlation coefficient; for details, please refer to the corresponding part of the above method embodiment, which will not be repeated here.
[0082] The update module 306 is used to replace the abnormal data in the initial data fusion result set with the first fitted value of the abnormal data, thereby obtaining an updated data fusion result set. For details, please refer to the corresponding section of the above method embodiment; further elaboration will not be repeated here.
[0083] A data resource fusion device for a power distribution network fused data resources from at least two datasets to be fused, obtaining an initial data fusion result set; acquiring abnormal data and corresponding abnormal attributes based on the initial data fusion result set; constructing a sample dataset based on all attributes contained in the initial data fusion result set and the normal data corresponding to each attribute; calculating the correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset based on the sample dataset and the abnormal attributes; calculating a first fitted value for the abnormal data based on the initial data fusion result set and the correlation coefficient; and replacing the abnormal data in the initial data fusion result set with the first fitted value for the abnormal data to obtain an updated data fusion result set. In this embodiment, the first fitted value is predicted based on the abnormal data. Compared to the abnormal data, the first fitted value predicted using the correlation coefficient between the data is closer to the correct data. Therefore, replacing the abnormal data with the predicted first fitted value makes the updated data fusion result set more accurate.
[0084] In one embodiment, the correlation coefficient calculation module 304 includes:
[0085] The input module is used to input the sample dataset and outlier attributes into the multiple linear regression model;
[0086] The linear regression module is used to obtain the linear correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset through a multiple linear regression model, and to use the linear correlation coefficient as the association coefficient.
[0087] In one embodiment, the linear regression module includes:
[0088] The first acquisition module is used to calculate the second fitting value of the data corresponding to the abnormal attribute in the sample dataset based on the preset linear coefficients and the sample dataset;
[0089] The extraction module is used to extract the true values of the data corresponding to the abnormal attributes from the sample dataset;
[0090] The second acquisition module is used to take the preset linear coefficient corresponding to the minimum sum of variances between the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset as the linear correlation coefficient.
[0091] In one embodiment, the second acquisition module includes:
[0092] The building module is used to construct an expression for the sum of the variances of the second fitted value and the true value of the data corresponding to the outlier attribute in the sample dataset;
[0093] The differentiation module is used to differentiate the expression, and take the preset linear coefficient corresponding to the time when the derivative is zero as the preset linear coefficient corresponding to the time when the sum of variances is minimized, and take the preset linear coefficient corresponding to the time when the sum of variances is minimized as the linear correlation coefficient.
[0094] In one embodiment, the abnormal data acquisition module 302 includes:
[0095] The first judgment module is used to sequentially judge whether each data in the initial data fusion result set belongs to the set abnormal threshold range. If it belongs to the set abnormal threshold range, the corresponding data is considered to be abnormal data.
[0096] In one embodiment, the power distribution network data resource fusion device further includes:
[0097] The second judgment module is used to determine whether the number of data groups in the sample dataset is less than the number of attributes in the initial data fusion result set. If it is less than the number of attributes in the initial data fusion result set, the sample dataset is reconstructed until the number of data groups in the sample dataset is greater than or equal to the number of attributes in the initial data fusion result set.
[0098] In one embodiment, the fitted value calculation module 305 includes:
[0099] The multiplication module is used to multiply the initial data fusion result set and the correlation coefficient to obtain the first fitted value of the outlier data.
[0100] This invention also provides an electronic device, such as... Figure 4As shown, the system includes a memory 420 and a processor 410, which are interconnected. The memory 420 stores computer instructions, and the processor 410 executes these computer instructions to perform the power distribution network data resource fusion method described in the above embodiments of the present invention. The processor 410 and the memory 420 can be connected via a bus or other means. The processor 410 can be a central processing unit (CPU). The processor 410 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The memory 420, as a non-transitory computer storage medium, can be used to store non-transitory software programs, non-transitory executable programs, and modules, such as the corresponding program instructions / modules in the embodiments of the present invention. The processor 410 executes various functional applications and data processing by running non-transitory software programs, instructions, and modules stored in the memory 420, thereby implementing the power distribution network data resource fusion method in the above method embodiments. The memory 420 may include a program storage area and a data storage area. The program storage area may store the operating device and the application program required for at least one function; the data storage area may store data created by the processor 410, etc. Furthermore, the memory 420 may include a high-speed random access memory 420, and may also include non-transitory memory 420, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 420 may optionally include remotely located memories 420 relative to the processor 410, which can be connected to the processor 410 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. One or more modules are stored in the memory 420, and when executed by the processor 410, perform the power distribution network data resource fusion method as described in the above method embodiments. The specific details of the aforementioned electronic device can be understood by referring to the relevant descriptions and effects in the above method embodiments, and will not be repeated here.
[0101] This invention also provides a computer-readable storage medium, such as... Figure 5As shown, a computer program 510 is stored on the storage medium. When executed by a processor, this program implements the steps of the power distribution network data resource fusion method in the above embodiments. The storage medium also stores audio and video stream data, feature frame data, interactive request signaling, encrypted data, and preset data sizes. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program 13 can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.
[0102] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for integrating data resources in a power distribution network, characterized in that, include: The data from at least two datasets to be merged are fused to obtain an initial data fusion result set; Based on the initial data fusion result set, obtain the abnormal data and the abnormal attributes corresponding to the abnormal data; A sample dataset is constructed based on all attributes contained in the initial data fusion result set and the normal data corresponding to each attribute. The sample dataset and outlier attributes are input into a multiple linear regression model. The model obtains the linear correlation coefficient between the data corresponding to the outlier attributes in the sample dataset and the sample dataset itself. This linear correlation coefficient is used as the association coefficient. The process of obtaining the linear correlation coefficient between the data corresponding to the outlier attributes in the sample dataset and the sample dataset through the multiple linear regression model includes: calculating a second fitted value of the data corresponding to the outlier attributes in the sample dataset based on a preset linear coefficient and the sample dataset; extracting the true values of the data corresponding to the outlier attributes from the sample dataset; and constructing an expression for the squared difference between the second fitted value and the true value of the data corresponding to the outlier attributes in the sample dataset. The expression is... , For the true value, The second fitted value is used; the expression is differentiated, and the preset linear coefficient corresponding to the time when the derivative is zero is taken as the preset linear coefficient corresponding to the time when the square of the difference is minimized, and the preset linear coefficient corresponding to the time when the square of the difference is minimized is taken as the linear correlation coefficient; Calculate the first fitted value of the abnormal data based on the initial data fusion result set and the correlation coefficient; The abnormal data in the initial data fusion result set are replaced with the first fitted value of the abnormal data to obtain the updated data fusion result set.
2. The distribution network data resource fusion method according to claim 1, characterized in that, Abnormal data is obtained based on the initial data fusion result set, including: The system sequentially checks whether each data point in the initial data fusion result set falls within the set abnormal threshold range. If a data point falls within the set abnormal threshold range, the corresponding data is considered abnormal.
3. The distribution network data resource fusion method according to claim 1, characterized in that, Before calculating the correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset based on the sample dataset and the abnormal attribute, the method further includes: Determine whether the number of data groups in the sample dataset is less than the number of attributes in the initial data fusion result set; If the number of attributes in the sample dataset is less than the number of attributes in the initial data fusion result set, the sample dataset is reconstructed until the number of data groups in the sample dataset is greater than or equal to the number of attributes in the initial data fusion result set.
4. The distribution network data resource fusion method according to claim 1, characterized in that, The first fitted value of the outlier data is calculated based on the initial data fusion result set and the correlation coefficient, including: The initial data fusion result set is multiplied by the correlation coefficient to obtain the first fitted value of the abnormal data.
5. A data resource fusion device for power distribution networks, characterized in that, include: The initial fusion module is used to perform resource fusion on data from at least two datasets to be fused, and obtain an initial data fusion result set. The abnormal data acquisition module is used to acquire abnormal data and the abnormal attributes corresponding to the abnormal data based on the initial data fusion result set; The sample construction module is used to construct a sample dataset based on the complete historical data fusion result set. The correlation coefficient calculation module is used to calculate the correlation coefficient between the data corresponding to the abnormal attribute in the sample dataset and the sample dataset based on the sample dataset and the abnormal attribute. The fitting value calculation module is used to calculate the first fitting value of the abnormal data based on the initial data fusion result set and the correlation coefficient. The update module is used to replace the abnormal data in the initial data fusion result set with the first fitted value of the abnormal data to obtain the updated data fusion result set; The correlation coefficient calculation module includes: The input module is used to input the sample dataset and outlier attributes into the multiple linear regression model; The linear regression module is used to obtain the linear correlation coefficient between the data corresponding to the abnormal attributes in the sample dataset and the sample dataset through a multiple linear regression model, and to use the linear correlation coefficient as the association coefficient. The linear regression module includes: The first acquisition module is used to calculate the second fitting value of the data corresponding to the abnormal attribute in the sample dataset based on the preset linear coefficients and the sample dataset; The extraction module is used to extract the true values of the data corresponding to the abnormal attributes from the sample dataset; The second acquisition module is used to take the preset linear coefficient corresponding to the minimum squared difference between the second fitted value and the true value of the data corresponding to the abnormal attribute in the sample dataset as the linear correlation coefficient. The second acquisition module includes: The construction module is used to construct an expression for the squared difference between the second fitted value and the true value of the data corresponding to the outlier attribute in the sample dataset, wherein the expression is: , For the true value, This is the second fitted value; The differentiation module is used to differentiate the expression, take the preset linear coefficient corresponding to the time when the derivative is zero as the preset linear coefficient corresponding to the time when the square of the difference is minimized, and take the preset linear coefficient corresponding to the time when the square of the difference is minimized as the linear correlation coefficient.
6. The power distribution network data resource fusion device according to claim 5, characterized in that, The abnormal data acquisition module includes: The first judgment module is used to sequentially judge whether each data in the initial data fusion result set belongs to the set abnormal threshold range. If it belongs to the set abnormal threshold range, the corresponding data is considered to be abnormal data.
7. The power distribution network data resource fusion device according to claim 5, characterized in that, The power distribution network data resource fusion device also includes: The second judgment module is used to determine whether the number of data groups in the sample dataset is less than the number of attributes in the initial data fusion result set. If it is less than the number of attributes in the initial data fusion result set, the sample dataset is reconstructed until the number of data groups in the sample dataset is greater than or equal to the number of attributes in the initial data fusion result set.
8. The power distribution network data resource fusion device according to claim 5, characterized in that, The fitted value calculation module includes: The multiplication module is used to multiply the initial data fusion result set and the correlation coefficient to obtain the first fitted value of the outlier data.
9. An electronic device, characterized in that, include: The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the power distribution network data resource fusion method as described in any one of claims 1 to 4.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the power distribution network data resource fusion method as described in any one of claims 1 to 4.