Multi-dimensional power failure monitoring data quality evaluation method and system based on multi-source heterogeneous power grid big data

Abstract

The application provides a multi-dimensional power failure monitoring data quality evaluation method and system based on multi-source heterogeneous power grid big data. By fully utilizing multiple data sources, cross-validation between data is performed, and a deep learning method is used to realize all-around monitoring and evaluation of power failure monitoring data quality in consistency, integrity, accuracy and reliability, thereby improving the overall data quality. Through this new method, we are committed to solving the key problems of data quality monitoring in the power failure monitoring system, and providing more reliable data support for the stable operation of the power system.

Description

Technical Field

[0001] This invention relates to power distribution network data quality assessment technology, specifically a multi-dimensional power outage monitoring data quality assessment method and system based on multi-source heterogeneous power grid big data. Background Technology

[0002] Power outage monitoring plays a crucial role in the stable operation of power systems. As an indispensable infrastructure of modern society, electricity's stable supply is vital for industry, commerce, and residential life. Timely and accurate monitoring of power outage events is essential for preventing potential power failures and improving grid reliability. Power outages not only affect users' normal electricity consumption but can also lead to serious consequences such as production interruptions and data loss. Therefore, establishing an effective power outage monitoring system has become an urgent need to ensure the normal operation of the power system.

[0003] Establishing an effective power outage monitoring system relies on massive amounts of data. The quality of this data directly determines the stability and reliability of the system. Low-quality data can lead to misleading decisions, inaccurate outage predictions, and negatively impact the overall reliability of the power system, ultimately affecting users' electricity plans and experience. Therefore, data quality monitoring is an indispensable part of ensuring the accuracy and stability of the monitoring system. Only high-quality data can provide a reliable foundation for the power outage monitoring system, ensuring accurate identification and response to outage events.

[0004] Previous methods often relied on power data collected from the power grid to monitor power outages. However, due to sensor aging, signal transmission errors, and sparse equipment deployment, the power data collected from the power grid sometimes contains errors and missing data. Therefore, relying solely on power data collected from the power grid is insufficient to meet the requirements for fine-grained and real-time accurate fault diagnosis and analysis capabilities in the distribution network. With the widespread use of the internet and user-side equipment, user-side data is a highly valuable data source. Cross-verifying user-side data with power grid data for data quality monitoring can effectively alleviate the uncertainty and data gaps associated with single data sources, thereby enhancing the reliability of the power outage monitoring system. Therefore, how to effectively utilize multi-source data for cross-data quality monitoring is a crucial issue currently facing us.

[0005] Currently, data quality assessment relies heavily on human intervention, which has several significant drawbacks: First, due to subjective factors, different assessors may introduce subjective errors, leading to inconsistent results. Second, human assessment typically consumes substantial time and resources, especially when dealing with large-scale, complex, multi-source data, where its efficiency is significantly lower than automated methods. Furthermore, human assessment may fail to capture hidden problems or potential patterns in the data, limiting a comprehensive understanding of data quality. In contrast, utilizing deep learning algorithms for data quality assessment offers clear advantages. First, deep learning algorithms can effectively reduce the need for human resources, completing quality monitoring tasks with lower resource consumption. Second, deep learning algorithms can automatically learn complex patterns and features in the data, enabling them to more comprehensively and accurately capture data quality issues, improving the accuracy and consistency of the assessment.

[0006] Furthermore, introducing multiple data quality assessment metrics is crucial for a comprehensive evaluation of data quality. A single assessment metric may not adequately cover all aspects of data quality; introducing multi-dimensional metrics provides a more comprehensive and multi-faceted evaluation result. Different data quality issues may require different measurement methods, and considering multiple metrics comprehensively allows for a more complete understanding of the data quality status. This comprehensive assessment approach helps decision-makers gain a more holistic understanding of data quality issues and develop targeted improvement strategies. Summary of the Invention

[0007] The technical problem to be solved by this invention is how to select high-quality data for power outage analysis.

[0008] The present invention solves the above-mentioned technical problems through the following technical means: A multi-dimensional power outage monitoring data quality assessment method based on multi-source heterogeneous power grid big data includes the following steps: Step 1. Unify the encoding of multi-source data; extract data features from multi-source data with the same encoding, and align the features of multi-source data to obtain preprocessed features; Step 2. The preprocessed features are used as input to the power outage assessment network, which outputs power outage-related features. The power outage assessment network first extracts static and temporal features from the preprocessed features, performs further feature extraction and integration on the temporal features, and then concatenates them with the static features to obtain fused features. Based on the fused features, the power outage-related features are output. The extraction and integration method of the temporal features is as follows: first, the temporal features are... Divide the data into windows of size T, each containing data features from T consecutive time points, and represent them as follows: In the temporal feature extraction and integration, T GRU modules are stacked. Each GRU module corresponds to a dynamic feature at a corresponding time as input and generates a vectorized representation of the hidden state as input to the next GRU module. Step 3. Multi-dimensional cross-verification of power outage data quality assessment; based on the consistency, completeness, accuracy, and uncertainty of various multi-source data related to the power outage.

[0009] Furthermore, the specific calculation process for consistency in step 3 is as follows: Assuming there are N data sources, let the power outage-related features be as follows: , , , ; Consistency calculation: For power outage times from N data sources, calculate the time difference between the two data sources and normalize it using a set duration to obtain the power outage time consistency index.

[0010] in, This represents the absolute value operation. This represents the operation of finding the minimum value. It is a number between 0 and 1; the larger the value, the more consistent the predicted power outage time. For power outage areas, the crossover ratio of these areas is calculated as the consistency index of the power outage areas.

[0011] in, This represents the area of ​​the intersection of two regions. This represents the area of ​​the union of two regions; It is a number between 0 and 1. The larger the value, the greater the overlap of the predicted fault areas, that is, the more consistent the prediction of the fault areas. For each fault cause, the cosine similarity between the probability distribution vectors of the two fault causes is calculated as the consistency index of the fault causes.

[0012] in, Represents the dot product operation; It is a number between 0 and 1. The larger the value, the more consistent the predicted causes of failures. The consistency metric between the last two sources is defined as follows:

[0013] Consistency is a number between 0 and 1. The greater the consistency, the more consistent the overall power outage information contained in the two sources. For N data sources, the consistency model is the average of the consistency between each pair of sources:

[0014] in, The numerator represents the consistency between data source i and data source j. The numerator is the sum of the consistency obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. That is, the consistency of N data sources is the average consistency between each pair of sources.

[0015] Furthermore, the specific calculation process for integrity in step 3 is as follows: First, calculate the consistency index between the two sources regarding the outage time, outage area, and cause of the fault.

[0016]

[0017]

[0018] The definitions of each item are the same as in 1); the integrity index is defined as follows:

[0019] Where #{} represents the counting operation. The AND symbol indicates that only indices that satisfy both sides of the AND symbol are counted; the numerator represents the inconsistency between the prediction results from the two sources, and the reliability of one source is much higher than that of the other. This means that the former source's data is sufficient to predict the corresponding power outage information, while the latter source cannot make a reliable prediction of the power outage information, which means that there is missing data in the source; completeness is a number between 0 and 1. The larger the completeness, the more complete the power outage information contained in the corresponding source. For N data sources, the integrity model is the average of the integrity between each pair of sources:

[0020] in, The numerator represents the integrity between data source i and data source j. The numerator is the sum of the integrity obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. In other words, the integrity of N data sources is the average integrity between each pair of sources.

[0021] Furthermore, the specific calculation process for accuracy in step 3 is as follows: 1) Given the actual power outage time, outage area, and cause of the fault (indicated by the subscript gt), we calculate the accuracy index for each of these three items.

[0022]

[0023]

[0024] The definitions of each symbol are the same as in 1); the accuracy index is a value between 0 and 1, and the subscript represents the fault time, fault area and fault cause. The closer the accuracy value is to 1, the more accurate the current source prediction is. The accuracy metric corresponding to source 1 is defined as follows:

[0025] Accuracy is a number between 0 and 1. The higher the accuracy, the more accurate the currently uploaded data. Correspondingly, by replacing the data from source 1 with the data from source 2, we can obtain the accuracy index of source 2. For N data sources, the accuracy of each data source can be determined by replacing the data from source 1 with the data from that source.

[0026] Furthermore, the specific reliability calculation process in step 3 is as follows: Reliability index is defined as

[0027] The subscript j represents the data source, which can be 1 or 2. U represents the uncertainty predicted by the power outage assessment network, which is a number between 0 and 1. The reliability obtained from this formula is a number between 0 and 1. The higher the reliability, the more reliable the prediction result of the currently uploaded data. For N data sources, the reliability of the data from each source can be obtained by simply replacing the corresponding uncertainty in the formula with the uncertainty obtained from the network prediction of the data from that source.

[0028] Corresponding to the above method, the present invention also provides a multi-dimensional power outage monitoring data quality assessment system based on multi-source heterogeneous power grid big data, including: The data preprocessing module is used to uniformly encode multi-source data; it is used to extract data features from multi-source data with the same encoding, and to align the features of the multi-source data to obtain preprocessed features. The analysis module is used to preprocess features as input to the power outage analysis network and output power outage-related features. The power outage analysis network first extracts static and temporal features from the preprocessed features, further extracts and integrates the temporal features, and then concatenates them with the static features to obtain fused features. Based on the fused features, the power outage-related features are output. The extraction and integration method of the temporal features is as follows: first, the temporal features... Dividing the data into windows of width T, each containing data features from T consecutive time points, can be represented as follows: In the temporal feature extraction and integration, T GRU modules are stacked. Each GRU module corresponds to a dynamic feature at a corresponding time as input and generates a vectorized representation of the hidden state as input to the next GRU module. The data quality assessment module is used for multi-dimensional cross-verification of power outage data quality assessment; it is based on four dimensions of indicators: consistency, completeness, accuracy, and uncertainty of various calculation multi-source data related to the power outage.

[0029] Furthermore, the specific calculation process for consistency in the data quality assessment module is as follows: Assuming there are N data sources, let the power outage-related features be as follows: , , , ; Consistency calculation: For power outage times from N data sources, calculate the time difference between the two data sources and normalize it using a set duration to obtain the power outage time consistency index.

[0030] in, This represents the absolute value operation. This represents the operation of finding the minimum value. It is a number between 0 and 1; the larger the value, the more consistent the predicted power outage time. For power outage areas, the crossover ratio of these areas is calculated as the consistency index of the power outage areas.

[0031] in, This represents the area of ​​the intersection of two regions. This represents the area of ​​the union of two regions; It is a number between 0 and 1. The larger the value, the greater the overlap of the predicted fault areas, that is, the more consistent the prediction of the fault areas. For each fault cause, the cosine similarity between the probability distribution vectors of the two fault causes is calculated as the consistency index of the fault causes.

[0032] in, Represents the dot product operation; It is a number between 0 and 1. The larger the value, the more consistent the predicted causes of failures. The consistency metric between the last two sources is defined as follows:

[0033] Consistency is a number between 0 and 1. The greater the consistency, the more consistent the overall power outage information contained in the two sources. For N data sources, the consistency model is the average of the consistency between each pair of sources:

[0034] in, The numerator represents the consistency between data source i and data source j. The numerator is the sum of the consistency obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. That is, the consistency of N data sources is the average consistency between each pair of sources.

[0035] Furthermore, the specific calculation process for integrity in the data quality assessment module is as follows: First, calculate the consistency index between the two sources regarding the outage time, outage area, and cause of the fault.

[0036]

[0037]

[0038] The definitions of each item are the same as in 1); the integrity index is defined as follows:

[0039] Where #{} represents the counting operation. The AND symbol indicates that only indices that satisfy both sides of the AND symbol are counted; the numerator represents the inconsistency between the prediction results from the two sources, and the reliability of one source is much higher than that of the other. This means that the former source's data is sufficient to predict the corresponding power outage information, while the latter source cannot make a reliable prediction of the power outage information, which means that there is missing data in the source; completeness is a number between 0 and 1. The larger the completeness, the more complete the power outage information contained in the corresponding source. For N data sources, the integrity model is the average of the integrity between each pair of sources:

[0040] in, The numerator represents the integrity between data source i and data source j. The numerator is the sum of the integrity obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. In other words, the integrity of N data sources is the average integrity between each pair of sources.

[0041] Furthermore, the specific calculation process for accuracy in the data quality assessment module is as follows: Given the actual power outage time, affected area, and cause of the fault (denoted by gt), we calculate the accuracy indices for these three parameters.

[0042]

[0043]

[0044] The definitions of each symbol are the same as in 1); the accuracy index is a value between 0 and 1, and the subscript represents the fault time, fault area and fault cause. The closer the accuracy value is to 1, the more accurate the current source prediction is. The accuracy metric corresponding to source 1 is defined as follows:

[0045] Accuracy is a number between 0 and 1. The higher the accuracy, the more accurate the currently uploaded data. Correspondingly, by replacing the data from source 1 with the data from source 2, we can obtain the accuracy index of source 2. For N data sources, the accuracy of each data source can be determined by replacing the data from source 1 with the data from that source.

[0046] Furthermore, the specific reliability calculation process in the data quality assessment module is as follows: Reliability index is defined as

[0047] The subscript j represents the data source, which can be 1 or 2. U represents the uncertainty predicted by the power outage assessment network, which is a number between 0 and 1. The reliability obtained from this formula is a number between 0 and 1. The higher the reliability, the more reliable the prediction result of the currently uploaded data. For N data sources, the reliability of the data from each source can be obtained by simply replacing the corresponding uncertainty in the formula with the uncertainty obtained from the network prediction of the data from that source.

[0048] The advantages of this invention are: 1. This invention proposes integrating multi-source data. Compared to traditional single-source power outage monitoring, introducing multi-source data increases the amount of power outage information and complements grid data, thereby alleviating the problems of uncertainty and data gaps associated with single data sources. Cross-verification between multiple data sources further improves the reliability and stability of the power outage monitoring data quality control system, making it more adaptable to complex and ever-changing power system environments.

[0049] 2. This invention proposes a data quality monitoring method based on deep learning. It innovatively employs deep learning technology for data quality monitoring, using gated recurrent units to fully capture and model temporal relationships, and employing a multilayer perceptron for power outage assessment. Through deep learning, it comprehensively and accurately understands the potential patterns in large-scale, complex, multi-source data, improving the accuracy and consistency of data quality assessment. This effectively frees up human resources, achieves automated data quality monitoring, improves monitoring efficiency, and provides more reliable support for power system operation.

[0050] 3. This invention proposes a multi-dimensional data quality monitoring and evaluation index, innovatively considering the relationships between multiple data sources and cross-validating multiple data sources. It introduces four dimensions of indicators: consistency, completeness, accuracy, and reliability, providing a more comprehensive and multi-faceted data quality assessment result. These indicators not only help to gain a deeper understanding of data quality issues but also provide decision-makers with more comprehensive information, facilitating targeted decision-making. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the process of the method in Embodiment 1 of the present invention; Figure 2 This is a framework diagram of the data preprocessing algorithm in Embodiment 1 of the present invention; Figure 3 This is a diagram of the deep learning-based power outage data analysis algorithm framework in Embodiment 1 of the present invention. Figure 4 This is a schematic diagram of the GRU structure in Embodiment 1 of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1 This embodiment describes a multi-dimensional power outage monitoring data quality assessment method based on multi-source heterogeneous power grid big data. The specific algorithm consists of three parts: preprocessing of data before the algorithm, power outage data analysis based on deep learning, definition of power outage data quality through multi-dimensional cross-verification, and assessment. Figure 1 As shown, the three-part principle and actual algorithm process are explained in detail.

[0054] Step 1: Data Preprocessing Before Algorithm Data obtained from different data sources often has different formats due to varying data management standards. For example, data stored in relational databases is generally structured, while early warning notices and network activity records are typically semi-structured, and fault reports are generally unstructured. To achieve the model's representation and storage of power outage monitoring data for power distribution network operation scenarios, it is necessary to encode large amounts of text and other data in a unified format.

[0055] To address the issues of data redundancy, missing data, and inconsistent data formats present in power outage monitoring data from different sources, this embodiment first filters and categorizes the data, resulting in three types of data: structured, semi-structured, and unstructured. For each of these three types of data, corresponding pre-trained networks are used to extract relevant data features. Specifically, for example... Figure 2 As shown, for structured text data, a low-level semantic extraction model is used to obtain a one-hot encoded representation; for semi-structured text data, a word2vec pre-trained model is used to extract its feature representation; and for unstructured text data, a BERT pre-trained model is used to extract the corresponding text data features.

[0056] It should be noted that since the feature spaces corresponding to different pre-trained models are not aligned, after extracting features from the three types of data using specific pre-trained feature extractors, this embodiment still needs to further reconstruct the features. To this end, after obtaining the extracted features, this embodiment uses a multilayer perceptron as a feature reprojection module to project the features corresponding to different pre-trained models into a common unified feature space.

[0057] By using a pre-trained text feature extractor on three different types of data, this embodiment transforms a large amount of data with different data formats obtained from different data sources into feature representations of the same form. Furthermore, a multilayer perceptron is used to align the three feature spaces, which on the one hand facilitates the unification of information input for subsequent data quality monitoring models, and on the other hand reduces the differences between heterogeneous data.

[0058] Step 2: Power outage data analysis based on deep learning To evaluate the quality of current power outage monitoring data using a deep learning model, this embodiment first needs to train a power outage data analysis model with sufficient accuracy. Therefore, this embodiment first trains the model using accurate power outage monitoring data from the past. The model takes the features obtained from the power outage monitoring data after preprocessing as input and outputs the actual power outage time, outage area, and cause of the fault.

[0059] Specifically, the model is designed as follows: 1) Since power outage monitoring data includes both time-series monitoring signals that are closely related to time and static signals that remain unchanged and valid over a period of time, in order to make full use of the characteristics of these two attributes, this embodiment first extracts static features from the initial features obtained by data preprocessing before the algorithm. and temporal characteristics .

[0060] 2) To fully model the long-short-term dependencies in temporal features, a gated recurrent unit (GRU) is used for further feature extraction and integration. Compared to LSTM recurrent neural networks, the GRU requires less data while maintaining high performance, thus being relatively faster. Specifically, for example... Figure 3 As shown, the time series features are first divided into a window of width T, which contains data features from T consecutive time points, and can be represented as follows: In the temporal feature extraction and integration process, T GRU modules are stacked. Each module takes a dynamic feature at a corresponding time step as input and generates a vectorized representation of the hidden state. The structure of GRU is as follows: For the t-th GRU module, it uses the temporal characteristics at time t. and the hidden state of the GRU output at time t-1 As input, the hidden state It contains information about previous nodes. The current GRU pair. and Processing this will yield the current hidden state. The GRU includes operations on two structured gates: the reset gate and the update gate. For the reset gate, the gating signal controlling the reset is first calculated.

[0061] in The weight matrix in the network, The activation function is sigmoid. After receiving the reset gating signal, the hidden state from the previous time step is... Input at the current time We perform weighted analysis to obtain the candidate hidden states.

[0062] Where tanh is the activation function. Reset gate Controlled the hidden state of the previous time step How does the hidden state flow into the candidate hidden state at the current time step? The hidden state at the previous time step contains all historical information of the time series up to that time step. Therefore, the reset gate can be used to discard historical information that is irrelevant to the prediction.

[0063] For the update gate, the gating signal controlling the update is calculated.

[0064] Use the updated gating signal to update the hidden state from the previous time step. and the current hidden candidate state We perform weighted summation to obtain the hidden state at the current moment. ,

[0065] The update gate determines how much information from the previous step can be used to continue iterating to the current step. This can help address the gradient decay problem in RNNs and better capture long-term dependencies in time series.

[0066] 3) Obtain static features and the timing characteristics after GRU integration Then, the features of these two attributes are concatenated to obtain the fused features.

[0067] in, For the splicing operation, the fused features are fed into a multilayer perceptron, which outputs the power outage time, power outage area, fault prediction, and corresponding uncertainty. The power outage time is the initial power outage moment, the power outage area is the geographical region, and the fault prediction is a probability distribution over all fault causes. Furthermore, there are corresponding uncertainties for the power outage time, area, and fault cause, representing a value indicating the reliability of the current result.

[0068] For the designed power outage data analysis network, this embodiment first needs to train the network using historical power outage monitoring data and corresponding known power outage times and areas, enabling the network to make power outage judgments based on the monitoring data. After obtaining the pre-trained power outage analysis network, this embodiment can perform power outage analysis on the current power outage monitoring data and evaluate the data quality based on the analysis results.

[0069] Step 3: Multi-dimensional cross-verification of power outage data quality assessment For multi-source power outage monitoring data that needs to be monitored, data preprocessing is performed to obtain an initial unified feature representation. Then, the features from the two sources are fed into a deep learning-based power outage data analysis network to obtain multi-source power outage analysis results, namely, power outage time, power outage area, fault cause, and corresponding uncertainty. Next, in this embodiment, we calculate the corresponding consistency, completeness, accuracy, and uncertainty indicators based on the multi-source power outage analysis results.

[0070] Taking two sources as an example, let's record the power outage time, outage area, fault cause, and corresponding uncertainty obtained from the power outage analysis network using data from the two sources. , , , The indicator is defined and calculated as follows: 1) Consistency: Consistency refers to whether the information contained in data from multiple sources is consistent. Specifically, for power outage times from two sources, the time difference between the two times is calculated and normalized in one-hour increments to obtain the power outage time consistency index.

[0071] in, This represents the absolute value operation. This represents the operation of finding the minimum value. It is a number between 0 and 1. The larger the value, the more consistent the power outage time prediction.

[0072] For power outage areas, the crossover ratio of these areas is calculated as the consistency index of the power outage areas.

[0073] in, This represents the area of ​​the intersection of two regions. This represents the area of ​​the union of two regions. It is a number between 0 and 1. The larger the value, the greater the overlap of the predicted fault areas, that is, the more consistent the prediction of the fault areas.

[0074] For each fault cause, the cosine similarity between the probability distribution vectors of the two fault causes is calculated as the consistency index of the fault causes.

[0075] in, This represents the dot product operation. It is a number between 0 and 1. The larger the value, the more consistent the predicted cause of the failure.

[0076] The consistency metric between the last two sources is defined as follows:

[0077] Consistency is a number between 0 and 1. The greater the consistency, the more consistent the overall power outage information contained in the two sources.

[0078] For N data sources, the consistency model is the average of the consistency between each pair of sources:

[0079] in, The numerator represents the consistency between data source i and data source j. The numerator is the sum of the consistency obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. In other words, the consistency of N data sources is the average consistency between each pair of sources.

[0080] 2) Completeness: Completeness refers to whether the currently uploaded data contains complete information sufficient for power outage monitoring. Specifically, it involves first calculating the consistency index between the two sources regarding the outage time, outage area, and cause of the fault.

[0081]

[0082]

[0083] The definitions for each item are the same as in 1). The integrity index is defined as follows:

[0084] Where #{} represents the counting operation. The AND symbol indicates that only indices satisfying both sides of the AND symbol are counted. The numerator represents the inconsistency between predictions from two sources, where one source is significantly more reliable than the other. This means the former source's data is sufficient to predict the corresponding power outage information, while the latter source cannot reliably predict it, indicating missing data in that source. Completeness is a number between 0 and 1; the higher the completeness, the more complete the power outage information contained in the corresponding source.

[0085] For N data sources, the integrity model is the average of the integrity between each pair of sources:

[0086] in, The numerator represents the integrity between data source i and data source j. The numerator is the sum of the integrity obtained by combining N data sources in pairs, and the denominator is the total number of combinations of N data sources in pairs. That is, the integrity of N data sources is the average integrity between each pair of sources.

[0087] 3) Accuracy: Accuracy refers to whether the currently uploaded data is consistent with the reported power outage data. Specifically, given the actual power outage time, outage area, and cause of the fault (indicated by the suffix gt), this embodiment calculates the accuracy index for these three items respectively.

[0088]

[0089]

[0090] The definitions of each symbol are the same as in 1). The accuracy index is a value between 0 and 1, and the subscript represents the time of failure, the area of ​​failure, and the cause of failure. The closer the accuracy value is to 1, the more accurate the current source prediction is.

[0091] The accuracy metric corresponding to source 1 is defined as follows:

[0092] Accuracy is a number between 0 and 1; the higher the accuracy, the more accurate the uploaded data. Correspondingly, replacing the data from source 1 with the data from source 2 will give you the accuracy index for source 2.

[0093] For N data sources, the accuracy of each data source can be determined by replacing the data from source 1 with the data from that source.

[0094] 4) Reliability: Reliability refers to the reliability of the power outage assessment structure predicted using currently uploaded data. The reliability index is defined as follows:

[0095] Here, the subscript j represents the data source, which can be 1 or 2, and U represents the uncertainty predicted by the power outage assessment network, a number between 0 and 1. The reliability obtained from this formula is a number between 0 and 1; the higher the reliability, the more credible the prediction result based on the uploaded data.

[0096] For N data sources, the reliability of the data from each source can be obtained by simply replacing the corresponding uncertainty in the formula with the uncertainty obtained from the network prediction of the data from that source.

[0097] In summary, this embodiment cross-verifies the results of power outage data analysis from the source power grid and the user-side network equipment, achieving the goal of monitoring the quality of data from different sources from multiple dimensions.

[0098] This embodiment proposes a research framework for multi-source data cross-validation. By fully utilizing multiple data sources, namely customer-side data and grid data, cross-validation between data is performed. Deep learning methods are used to comprehensively monitor and evaluate the quality of power outage monitoring data across three dimensions: consistency, completeness, accuracy, and reliability, thereby improving overall data quality. Through this novel approach, we aim to solve key issues in data quality monitoring within power outage monitoring systems, providing more reliable data support for the stable operation of power systems.

[0099] Example 2 Based on the method of Embodiment 1, this embodiment also proposes a corresponding system, including: The data preprocessing module is used to uniformly encode multi-source data; it is also used to extract data features from multi-source data with the same encoding, and to align the features of the multi-source data to obtain preprocessed features. The analysis module is used to preprocess features as input to the power outage analysis network and output power outage-related features. The power outage analysis network first extracts static and temporal features from the preprocessed features, further extracts and integrates the temporal features, and then concatenates them with the static features to obtain fused features. Based on the fused features, the power outage-related features are output. The extraction and integration method of the temporal features is as follows: first, the temporal features... Dividing the data into windows of width T, each containing data features from T consecutive time points, can be represented as follows: In the temporal feature extraction and integration, T GRU modules are stacked. Each GRU module corresponds to a dynamic feature at a corresponding time as input and generates a vectorized representation of the hidden state as input to the next GRU module. The data quality assessment module is used for multi-dimensional cross-verification of power outage data quality assessment; it is based on four dimensions of indicators: consistency, completeness, accuracy, and uncertainty of various calculation multi-source data related to the power outage.

[0100] The execution process of the above three modules refers to each step of Example 1.

[0101] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-dimensional power outage monitoring data quality assessment method based on multi-source heterogeneous power grid big data, characterized in that, Includes the following steps: Step 1. Unify the encoding of multi-source data; extract data features from multi-source data with the same encoding, and align the features of multi-source data to obtain preprocessed features; Step 2. The preprocessed features are used as input to the power outage assessment network, and the output is the power outage-related features. The power outage assessment network first extracts the static features and time-series features of the preprocessed features, performs further feature extraction and integration on the time-series features, and then concatenates them with the static features to obtain the fused features. Based on the fused features, the power outage-related features are output. The extraction and integration method of the temporal features is as follows: First, the temporal features are... Divide the data into windows of size T, each containing data features from T consecutive time points, and represent them as follows: In the temporal feature extraction and integration, T GRU modules are stacked. Each GRU module corresponds to a dynamic feature at a corresponding time as input and generates a vectorized representation of the hidden state as input to the next GRU module. Step 3. Multi-dimensional cross-verification of power outage data quality assessment; based on the consistency, completeness, accuracy, and uncertainty of various multi-source data related to the power outage; The specific calculation process for accuracy in step 3 is as follows: Given the actual power outage time, power outage area, and cause of the fault, represented by the index gt, calculate the accuracy index for each of these three items: in, This represents the absolute value operation. This represents the operation of finding the minimum value; , , These represent the outage time, outage area, and cause of the fault for source 1, respectively; the accuracy index is a value between 0 and 1, and the closer the accuracy value is to 1, the more accurate the current source prediction is; The accuracy metric corresponding to source 1 is defined as follows: Accuracy is a number between 0 and 1. The higher the accuracy, the more accurate the currently uploaded data. Correspondingly, by replacing the data from source 1 with the data from source 2, we can obtain the accuracy index of source 2. For N data sources, the accuracy of each data source can be determined by replacing the data from source 1 with the data from that source.

2. The method for assessing the quality of multi-dimensional power outage monitoring data based on multi-source heterogeneous power grid big data according to claim 1, characterized in that, The specific consistency calculation process in step 3 is as follows: Assuming there are N data sources, let the power outage-related features be respectively... , , , ; Consistency calculation: For power outage times from N data sources, calculate the time difference between the two data sources and normalize it using a set duration to obtain the power outage time consistency index. in, It is a number between 0 and 1; the larger the value, the more consistent the predicted power outage time. For power outage areas, the crossover ratio of these areas is calculated as the consistency index of the power outage areas. in, This represents the area of ​​the intersection of two regions. This represents the area of ​​the union of two regions; It is a number between 0 and 1. The larger the value, the greater the overlap of the predicted fault areas, that is, the more consistent the prediction of the fault areas. For each fault cause, the cosine similarity between the probability distribution vectors of the two fault causes is calculated as the consistency index of the fault causes. in, Represents the dot product operation; It is a number between 0 and 1. The larger the value, the more consistent the predicted causes of failures. The consistency metric between the last two sources is defined as follows: Consistency is a number between 0 and 1. The greater the consistency, the more consistent the overall power outage information contained in the two sources. For N data sources, the consistency model is the average of the consistency between each pair of sources: in, The numerator represents the consistency between data source i and data source j. The numerator is the sum of the consistency obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. That is, the consistency of N data sources is the average consistency between each pair of sources.

3. The method for assessing the quality of multi-dimensional power outage monitoring data based on multi-source heterogeneous power grid big data according to claim 1, characterized in that, The specific calculation process for integrity in step 3 is as follows: First, calculate the consistency index between the two sources regarding the outage time, outage area, and cause of the fault. Integrity index is defined as Where #{} represents the counting operation. The AND symbol indicates that only indices that satisfy both sides of the AND symbol are counted; the numerator represents the inconsistency between the prediction results from the two sources, and the reliability of one source is much higher than that of the other. This means that the former source's data is sufficient to predict the corresponding power outage information, while the latter source cannot make a reliable prediction of the power outage information, which means that there is missing data in the source. Completeness is a number between 0 and 1. The larger the completeness, the more complete the power outage information contained in the corresponding source. For N data sources, the integrity model is the average of the integrity between each pair of sources: in, The numerator represents the integrity between data source i and data source j. The numerator is the sum of the integrity obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. In other words, the integrity of N data sources is the average integrity between each pair of sources.

4. The method for assessing the quality of multi-dimensional power outage monitoring data based on multi-source heterogeneous power grid big data according to any one of claims 1 to 3, characterized in that, The specific reliability calculation process in step 3 is as follows: Reliability index is defined as Where the subscript j represents the data source, which can be 1 or 2; U represents the uncertainty predicted by the power outage assessment network, which is a number between 0 and 1; the reliability obtained from this formula is a number between 0 and 1. The higher the reliability, the more reliable the prediction result of the currently uploaded data. For N data sources, the reliability of the data from each source can be obtained by simply replacing the corresponding uncertainty in the formula with the uncertainty obtained from the network prediction of the data from that source.

5. A multi-dimensional power outage monitoring data quality assessment system based on multi-source heterogeneous power grid big data, characterized in that, include: The data preprocessing module is used to perform unified encoding on multi-source data; Data features are extracted from multi-source data with the same encoding, and the features of the multi-source data are aligned to obtain preprocessed features; The analysis module is used to preprocess features as input to the power outage analysis network and output power outage-related features. The power outage analysis network first extracts static features and temporal features from the preprocessed features, performs further feature extraction and integration on the temporal features, and then concatenates them with the static features to obtain fused features. Based on the fused features, the power outage-related features are output. The extraction and integration method of the temporal features is as follows: First, the temporal features are... Dividing the data into windows of width T, each containing data features from T consecutive time points, can be represented as follows: In the temporal feature extraction and integration, T GRU modules are stacked. Each GRU module corresponds to a dynamic feature at a corresponding time as input and generates a vectorized representation of the hidden state as input to the next GRU module. The data quality assessment module is used for multi-dimensional cross-verification of power outage data quality assessment; based on the consistency, completeness, accuracy, and uncertainty of various multi-source data related to the power outage; The specific calculation process for accuracy in the data quality assessment module is as follows: Given the actual power outage time, outage area, and cause of the fault, indicated by the index gt, calculate the accuracy index for each of these three items: in, This represents the absolute value operation. This represents the operation of finding the minimum value; , , These represent the outage time, outage area, and cause of the fault for source 1, respectively; the accuracy index is a value between 0 and 1, and the closer the accuracy value is to 1, the more accurate the current source prediction is; The accuracy metric corresponding to source 1 is defined as follows: Accuracy is a number between 0 and 1. The higher the accuracy, the more accurate the currently uploaded data. Correspondingly, by replacing the data from source 1 with the data from source 2, we can obtain the accuracy index of source 2. For N data sources, the accuracy of each data source can be determined by replacing the data from source 1 with the data from that source.

6. The multi-dimensional power outage monitoring data quality assessment system based on multi-source heterogeneous power grid big data according to claim 5, characterized in that, The specific calculation process for consistency in the data quality assessment module is as follows: Assuming there are N data sources, let the power outage-related features be respectively... , , , ; Consistency calculation: For power outage times from N data sources, calculate the time difference between the two data sources and normalize it using a set duration to obtain the power outage time consistency index. in, It is a number between 0 and 1; the larger the value, the more consistent the predicted power outage time. For power outage areas, the crossover ratio of these areas is calculated as the consistency index of the power outage areas. in, This represents the area of ​​the intersection of two regions. This represents the area of ​​the union of two regions; It is a number between 0 and 1. The larger the value, the greater the overlap of the predicted fault areas, that is, the more consistent the prediction of the fault areas. For each fault cause, the cosine similarity between the probability distribution vectors of the two fault causes is calculated as the consistency index of the fault causes. in, Represents the dot product operation; It is a number between 0 and 1. The larger the value, the more consistent the predicted causes of failures. The consistency metric between the last two sources is defined as follows: Consistency is a number between 0 and 1. The greater the consistency, the more consistent the overall power outage information contained in the two sources. For N data sources, the consistency model is the average of the consistency between each pair of sources: in, The numerator represents the consistency between data source i and data source j. The numerator is the sum of the consistency obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. That is, the consistency of N data sources is the average consistency between each pair of sources.

7. The multi-dimensional power outage monitoring data quality assessment system based on multi-source heterogeneous power grid big data according to claim 5, characterized in that, The specific calculation process for integrity in the data quality assessment module is as follows: First, calculate the consistency index between the two sources regarding the outage time, outage area, and cause of the fault. The definitions of each item are the same as in 1); the integrity index is defined as follows: Where #{} represents the counting operation. The AND symbol indicates that only indices that satisfy both sides of the AND symbol are counted; the numerator represents the inconsistency between the prediction results from the two sources, and the reliability of one source is much higher than that of the other. This means that the former source's data is sufficient to predict the corresponding power outage information, while the latter source cannot make a reliable prediction of the power outage information, which means that there is missing data in the source. Completeness is a number between 0 and 1. The larger the completeness, the more complete the power outage information contained in the corresponding source. For N data sources, the integrity model is the average of the integrity between each pair of sources: in, The numerator represents the integrity between data source i and data source j. The numerator is the sum of the integrity obtained by pairwise combinations of N data sources, and the denominator is the total number of pairwise combinations of N data sources. In other words, the integrity of N data sources is the average integrity between each pair of sources.

8. The multi-dimensional power outage monitoring data quality assessment system based on multi-source heterogeneous power grid big data according to any one of claims 5 to 7, characterized in that, The specific reliability calculation process in the data quality assessment module is as follows: Reliability index is defined as Where the subscript j represents the data source, which can be 1 or 2; U represents the uncertainty predicted by the power outage assessment network, which is a number between 0 and 1; the reliability obtained from this formula is a number between 0 and 1. The higher the reliability, the more reliable the prediction result of the currently uploaded data. For N data sources, the reliability of the data from each source can be obtained by simply replacing the corresponding uncertainty in the formula with the uncertainty obtained from the network prediction of the data from that source.

Images