Power grid operation data abnormal value identification method and system for power grid bearing capacity calculation
By combining large-window and small-window volatility calculations with multi-level judgment rules, abnormal values in power grid operation data are automatically identified, solving the problems of low efficiency and poor accuracy of manual screening in traditional methods, and realizing efficient and accurate power grid carrying capacity calculation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- STATE GRID LIAONING ECONOMIC TECHN INST
- Filing Date
- 2025-12-17
- Publication Date
- 2026-05-19
AI Technical Summary
In existing power grid carrying capacity calculations, manual screening of outliers is inefficient and inaccurate. Traditional methods rely on human experience and lack unified quantitative judgment standards, leading to calculation results that deviate from the actual physical situation.
A volatility calculation method based on large and small windows is adopted, combined with primary and backup rules, and outliers in power grid operation data are automatically identified by comparing median deviations. Robust statistical methods and multi-level judgment mechanisms are used to dynamically define anomaly criteria.
It achieves automated and adaptive outlier identification, improves identification accuracy and robustness, reduces false positives and false negatives, and is suitable for real-time or batch analysis of massive power grid operation data.
Smart Images
Figure CN122065201A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of power grid operation data processing, and specifically to a method and system for identifying outliers in power grid operation data for power grid capacity calculation. Background Technology
[0002] With the rapid increase in demand for grid connection of new energy sources, calculating the grid carrying capacity is of great significance for promoting the healthy development of distributed photovoltaic power. However, unreasonable data in the calculation process is mainly identified through manual screening. This screening requires understanding the operating status of the equipment. For example, if an equipment's operating value is low for a period of time, it needs to be determined whether the equipment is operating normally to identify anomalies. Simultaneously, manual verification is required for each device to show any anomalies, which is a very large workload. Furthermore, outliers such as jumps need to be removed. Since jumps do not occur regularly, a comprehensive screening of all data is necessary. If outliers are used in the carrying capacity calculation, the accuracy of the calculation will be affected.
[0003] Existing methods for calculating load-bearing capacity rely heavily on measurement data from the most typical moments of the equipment within a specific time period, and use this data to calculate the load-bearing capacity under extreme conditions. However, in actual operation, equipment often experiences significant deviations in some measurement data due to operational changes (such as adjustments to operating modes or changes in switch states), leading to abnormally high or low calculation results that deviate from the actual physical situation. There is a lack of unified quantitative criteria for judging such historical outliers, and traditional methods often rely on subjective judgment based on human experience, making them significantly affected by differences in business scenarios.
[0004] Therefore, there is an urgent need to design a method that can efficiently and accurately identify abnormal values in power grid operation data. Summary of the Invention
[0005] To address the low efficiency and accuracy of outlier identification in existing power grid operation data technologies, this invention proposes a method for identifying outliers in power grid operation data for power grid capacity calculation, comprising: The large-window volatility and small-window volatility of power grid operation data are calculated based on the normal fluctuation benchmark within the large window and the normal fluctuation benchmark within the small window. The first abnormal data is obtained by filtering the power grid operation data based on the large window volatility and small window volatility. The first abnormal data is filtered using the main rule to obtain the second abnormal data; The second abnormal data is compared with the median deviation of the normal data within a preset range. If it is not within the preset range of the median deviation of the normal fluctuation, it is the target abnormal data. The normal fluctuation benchmarks within the large window and the small window are obtained based on normal power grid operation data, with the large window containing more data points than the small window.
[0006] Optionally, the main rule is: whether the volatility of the data to be judged within a large window is greater than k times the normal volatility benchmark within the large window. If it is greater than k times the normal volatility benchmark within the large window, then the data to be judged is abnormal data, where k is an adjustable sensitivity coefficient.
[0007] Optionally, the step of using a master rule to filter the first abnormal data to obtain the second abnormal data further includes: If the first abnormal data does not meet the main rule, then the backup rule is used to filter the first abnormal data, wherein the backup rule is: Filter the data in the first abnormal data that are greater than m times the normal fluctuation amplitude to obtain the second abnormal data, where m is greater than or equal to 2.
[0008] Optionally, the volatility of the large window is characterized by standard deviation or median absolute deviation.
[0009] Optionally, the volatility of the small window is characterized by standard deviation or median absolute deviation.
[0010] Optionally, the specific steps for filtering the power grid operation data based on the large-window volatility and small-window volatility to obtain the first abnormal data are as follows: If the large-window volatility and small-window volatility simultaneously satisfy the following rule, then the data is identified as the first abnormal data: a, Small window volatility ÷ Large window volatility > Set threshold b, n% of small window volatility > large window volatility, where 0 < n ≤ 15.
[0011] Optionally, after filtering the power grid operation data based on the large window volatility and small window volatility to obtain the first abnormal data, the method further includes: determining the filtered non-first abnormal data according to the following rules: Calculate |Non-first abnormal data - Large window center value| ÷ Normal fluctuation benchmark value within the large window; if the value is greater than a preset threshold, then the non-first abnormal data is determined to be the first abnormal data.
[0012] Optionally, before filtering the power grid operation data based on the large window volatility and small window volatility to obtain the first abnormal data, the method further includes: removing zero-value data points and absolute value maximum data points.
[0013] Optionally, the small window may contain no fewer than three data points.
[0014] A second aspect of the present invention provides a system for identifying outlier values in power grid operation data for calculating power grid carrying capacity, comprising: The calculation module is used to calculate the large-window volatility and small-window volatility of power grid operation data based on the normal fluctuation benchmark within a large window and the normal fluctuation benchmark within a small window. The first filtering module is used to filter the power grid operation data based on the large window volatility and the small window volatility to obtain the first abnormal data; The second filtering module is used to filter the first abnormal data using the main rules to obtain the second abnormal data; The target acquisition module is used to compare the second abnormal data with the median deviation of the normal data within a preset range. If the data is not within the preset range of the median deviation of the normal fluctuation, it is the target abnormal data. The normal fluctuation benchmarks within the large window and the small window are obtained based on normal power grid operation data, with the large window containing more data points than the small window.
[0015] Optionally, the main rule in the second filtering module is: whether the volatility of the data to be judged within a large window is greater than k times the normal volatility benchmark within the large window. If it is greater than k times the normal volatility benchmark within the large window, then the data to be judged is abnormal data, where k is an adjustable sensitivity coefficient.
[0016] Optionally, the second filtering module, while using the main rule to filter the first abnormal data to obtain the second abnormal data, also includes: If the first abnormal data does not meet the main rule, then the backup rule is used to filter the first abnormal data, wherein the backup rule is: Filter the data in the first abnormal data that are greater than m times the normal fluctuation amplitude to obtain the second abnormal data, where m is greater than or equal to 2.
[0017] Optionally, the volatility of the large window in the first screening module is characterized by the standard deviation or the absolute deviation of the median.
[0018] Optionally, the volatility of the small window in the first screening module is characterized by the standard deviation or the absolute deviation of the median.
[0019] Optionally, the first filtering module filters the power grid operation data based on the large window volatility and small window volatility to obtain the first abnormal data. The specific steps are as follows: If the large-window volatility and small-window volatility simultaneously satisfy the following rule, then the data is identified as the first abnormal data: a, Small window volatility ÷ Large window volatility > Set threshold b, n% of small window volatility > large window volatility, where 0 < n ≤ 15.
[0020] Optionally, after the first filtering module filters the power grid operation data based on the large window volatility and small window volatility to obtain the first abnormal data, it further includes: judging the filtered non-first abnormal data according to the following rules: Calculate |Non-first abnormal data - Large window center value| ÷ Normal fluctuation benchmark value within the large window; if the value is greater than a preset threshold, then the non-first abnormal data is determined to be the first abnormal data.
[0021] Optionally, before the first filtering module filters the power grid operation data based on the large window volatility and small window volatility to obtain the first abnormal data, it further includes: removing zero-value data points and absolute value maximum data points.
[0022] Optionally, the number of data points in the small window of the first filtering module shall not be less than three.
[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a method and system for identifying outliers in power grid operation data for power grid capacity calculation. The method includes: calculating the large-window volatility and small-window volatility of power grid operation data based on normal fluctuation benchmarks within a large window and a small window; filtering the power grid operation data based on the large-window volatility and small-window volatility to obtain first outlier data; filtering the first outlier data using a master rule to obtain second outlier data; comparing the second outlier data with the median deviation of normal data within a preset range; if the second outlier data is not within the preset range of the median deviation of normal fluctuations, it is considered target outlier data. This invention has the following beneficial effects: 1) Overcoming the limitations of traditional fixed threshold methods. This invention achieves adaptive definition of anomaly criteria through dynamic comparison in a dual-window manner (overall fluctuation vs. local fluctuation), improving adaptability to complex operating scenarios.
[0024] 2) Improve the accuracy and robustness of anomaly identification. Robust statistical methods (such as median, MAD, and IQR) are used to calculate the fluctuation baseline, effectively eliminating the interference of outliers on the baseline calculation; a micro-fluctuation filtering mechanism is introduced (e.g., a median deviation of less than 10% is considered normal) to avoid false alarms caused by data smoothing or minor fluctuations; combined with multi-level judgment rules (primary rule, backup rule, and Z-score supplementary monitoring), a multi-layered anomaly screening system is formed to reduce missed and false alarms.
[0025] 3) Enhance adaptability to complex data patterns. By using mechanisms such as extreme value filtering and data sufficiency verification, misidentification caused by zero values, maximum values, or insufficient data can be avoided.
[0026] 4) Achieve automated identification, reducing human error and subjective bias. This invention automatically completes anomaly detection through algorithms, significantly improving processing speed. It is suitable for real-time or batch analysis of massive power grid operation data, providing unified and quantitative judgment standards, and improving the consistency and objectivity of results.
[0027] This invention provides an adaptive, robust, and automated method for identifying outliers in power grid operation data. Through dual-window dynamic comparison, robust statistics, and a multi-level judgment mechanism, it effectively solves the problems of traditional methods that rely on manual labor, have fixed thresholds, and have a high false positive rate. It significantly improves the accuracy, efficiency, and applicability of anomaly identification, and provides reliable data support for power grid carrying capacity calculation and equipment status monitoring. Attached Figure Description
[0028] Figure 1 This is a flowchart of the method for identifying outliers in power grid operation data for power grid carrying capacity calculation proposed in this invention. Figure 2 This is a schematic diagram illustrating the application process of the power grid operation data anomaly identification method for power grid carrying capacity calculation proposed in this invention. Figure 3 This is a schematic diagram of the power grid operation data anomaly identification system for power grid carrying capacity calculation proposed in this invention. Detailed Implementation
[0029] This invention proposes a method and system for identifying outliers in power grid operation data for power grid capacity calculation. It achieves preliminary identification of sensitive data by comparing the "overall fluctuation range" and "local fluctuation situation" of the measured values before and after the test data. This method overcomes the limitations of the traditional fixed threshold method by comprehensively analyzing the overall fluctuation trend of the data, dynamically formulating behavioral standards for abnormal fluctuations, and combining the analysis results of local fluctuations to achieve adaptive definition of the abnormal standards. Simultaneously, a micro-fluctuation filtering mechanism is introduced to effectively avoid misidentification problems caused by data overly smooth or complex fluctuation patterns, thereby improving the accuracy and robustness of the detection results.
[0030] Example 1: A method for identifying outliers in power grid operation data used in power grid capacity calculation, such as... Figure 1 As shown, it includes the following steps S1 to S4.
[0031] S1, calculate the large-window volatility and small-window volatility of power grid operation data based on the normal fluctuation benchmark within the large window and the normal fluctuation benchmark within the small window; wherein, the normal fluctuation benchmark within the large window and the normal fluctuation benchmark within the small window are obtained based on the normal data of power grid operation, and the large window has more data points than the small window.
[0032] A normal fluctuation benchmark within a large window is obtained based on normal power grid operation data. For example, data from 50 points before and after a target time point is obtained, and the overall fluctuation trend is calculated as the normal fluctuation benchmark within the large window. Similarly, a normal fluctuation benchmark within a small window is obtained based on normal power grid operation data. For example, data from 10 points before and after a target time point is obtained, and the overall fluctuation trend is calculated as the normal fluctuation benchmark within the small window.
[0033] The large-window volatility of the power grid operation data to be measured is calculated based on the normal fluctuation benchmark within the large window. Preferably, the large-window volatility is characterized by the standard deviation or the absolute deviation of the median.
[0034] The small-window volatility of the power grid operation data to be measured is calculated based on the normal fluctuation benchmark within the small window. Preferably, the small-window volatility is characterized by the standard deviation or the absolute deviation of the median.
[0035] In a further preferred embodiment, the small window contains no fewer than three data points.
[0036] S2, based on the large window volatility and small window volatility, the power grid operation data is filtered to obtain the first abnormal data.
[0037] In a further preferred embodiment, the specific steps for filtering the power grid operation data based on the large-window volatility and small-window volatility to obtain the first abnormal data are as follows: If the large-window volatility and small-window volatility simultaneously satisfy the following rule, then the data is identified as the first abnormal data: a. Small window volatility ÷ Large window volatility > Set threshold (e.g., 3 times) b, the volatility of the small window is greater than that of the volatility of the large window by n%, where 0 < n ≤ 15, and n is further preferred to be 10 (to avoid small fluctuation errors).
[0038] In a further optimized scheme, the following rules are used to supplement the Z-score monitoring and judgment for the selected non-first abnormal data (i.e., normal data): Calculate |Non-first anomaly data - Large window center value| ÷ Normal fluctuation baseline value within the large window; if the value is greater than a preset threshold, the non-first anomaly data is determined as first anomaly data. That is, secondary filtering is performed on data that still meets the dual-window detection criteria but remains sensitive.
[0039] In a further preferred embodiment, before filtering the power grid operation data based on the large window volatility and small window volatility to obtain the first abnormal data, the method further includes: removing zero-value data points and absolute value maximum data points. Absolute value maximum data points are often caused by normal operation of equipment connecting to or disconnecting from the power grid, and they need to be removed first to avoid misjudgment.
[0040] S3, use the main rule to filter the first abnormal data to obtain the second abnormal data.
[0041] In a further preferred embodiment, the main rule is: If the volatility of the data to be judged within a large window is greater than k times the normal volatility benchmark within the large window, then the data to be judged is abnormal data, where k is an adjustable sensitivity coefficient.
[0042] In a further preferred embodiment, the step of using a master rule to filter the first abnormal data to obtain the second abnormal data further includes: If the first abnormal data does not meet the main rule, then the backup rule is used to filter the first abnormal data, wherein the backup rule is: Filter the first abnormal data to obtain the second abnormal data, where m is greater than or equal to 2, for example, m is 2.5. When the first abnormal data is greater than 2.5 times the normal fluctuation amplitude, it is determined to be the second abnormal data.
[0043] S4, compare the median deviation of the second abnormal data with the median deviation of the normal data within a preset range. If it is not within the preset range of the median deviation of the normal fluctuation, it is the target abnormal data. For example, the second abnormal data is compared with the median deviation of the normal data by 10%. This step is mainly used for filtering out small fluctuations. If it is within 10%, it is considered normal data to avoid excessive alarms. If it is not within 10%, it is considered the final target abnormal data.
[0044] like Figure 2 As shown, in the application, the zero value and the maximum value in the running data are first extracted, and then the first abnormal data are obtained by filtering with large window and small window. The first abnormal data are judged by the main rule or the backup rule to obtain the second abnormal data. The second abnormal data is further filtered by small fluctuation to obtain the target abnormal data.
[0045] Example 2: Based on the same inventive concept, this invention also provides a system for identifying outlier values in power grid operation data for calculating power grid carrying capacity, such as... Figure 3 As shown, it includes: The calculation module is used to calculate the large-window volatility and small-window volatility of power grid operation data based on the normal fluctuation benchmark within a large window and the normal fluctuation benchmark within a small window. The first filtering module is used to filter the power grid operation data based on the large window volatility and the small window volatility to obtain the first abnormal data; The second filtering module is used to filter the first abnormal data using the main rules to obtain the second abnormal data; The target acquisition module is used to compare the second abnormal data with the median deviation of the normal data within a preset range. If the data is not within the preset range of the median deviation of the normal fluctuation, it is the target abnormal data. The normal fluctuation benchmarks within the large window and the small window are obtained based on normal power grid operation data, with the large window containing more data points than the small window.
[0046] In a further preferred embodiment, the main rule in the second screening module is: whether the volatility of the data to be judged within a large window is greater than k times the normal volatility benchmark within the large window; if it is greater than k times the normal volatility benchmark within the large window, then the data to be judged is abnormal data, where k is an adjustable sensitivity coefficient.
[0047] In a further preferred embodiment, the second filtering module, while using the main rule to filter the first abnormal data to obtain the second abnormal data, also includes: If the first abnormal data does not meet the main rule, then the backup rule is used to filter the first abnormal data, wherein the backup rule is: Filter the data in the first abnormal data that are greater than m times the normal fluctuation amplitude to obtain the second abnormal data, where m is greater than or equal to 2.
[0048] In a further preferred embodiment, the volatility of the large window in the first screening module is characterized by the standard deviation or the absolute deviation of the median.
[0049] In a further preferred embodiment, the volatility of the small window in the first screening module is characterized by the standard deviation or the absolute deviation of the median.
[0050] In a further preferred embodiment, the first screening module filters the power grid operation data based on the large-window volatility and small-window volatility to obtain the first abnormal data. The specific steps are as follows: If the large-window volatility and small-window volatility simultaneously satisfy the following rule, then the data is identified as the first abnormal data: a, Small window volatility ÷ Large window volatility > Set threshold b, n% of small window volatility > large window volatility, where 0 < n ≤ 15.
[0051] In a further preferred embodiment, after the first screening module obtains the first abnormal data by screening the power grid operation data based on the large window volatility and small window volatility, it further includes: judging the screened non-first abnormal data according to the following rules: Calculate |Non-first abnormal data - Large window center value| ÷ Normal fluctuation benchmark value within the large window; if the value is greater than a preset threshold, then the non-first abnormal data is determined to be the first abnormal data.
[0052] In a further preferred embodiment, before the first screening module filters the power grid operation data based on the large window volatility and small window volatility to obtain the first abnormal data, it further includes: removing zero-value data points and absolute value maximum data points.
[0053] In a further preferred embodiment, the number of data points in the small window of the first filtering module is no less than 3.
[0054] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.
Claims
1. A method for identifying outliers in power grid operation data used in power grid capacity calculation, characterized in that, include: The large-window volatility and small-window volatility of power grid operation data are calculated based on the normal fluctuation benchmark within the large window and the normal fluctuation benchmark within the small window. The first abnormal data is obtained by filtering the power grid operation data based on the large window volatility and small window volatility. The first abnormal data is filtered using the main rule to obtain the second abnormal data; The second abnormal data is compared with the median deviation of the normal data within a preset range. If it is not within the preset range of the median deviation of the normal fluctuation, it is the target abnormal data. The normal fluctuation benchmarks within the large window and the small window are obtained based on normal power grid operation data, with the large window containing more data points than the small window.
2. The method for identifying abnormal values in power grid operation data according to claim 1, characterized in that, The main rule is: whether the volatility of the data to be judged within a large window is greater than k times the normal volatility benchmark within the large window. If it is greater than k times the normal volatility benchmark within the large window, then the data to be judged is abnormal data, where k is an adjustable sensitivity coefficient.
3. The method for identifying abnormal values in power grid operation data according to claim 2, characterized in that, The process of using a master rule to filter the first abnormal data to obtain the second abnormal data also includes: If the first abnormal data does not meet the main rule, then the backup rule is used to filter the first abnormal data, wherein the backup rule is: Filter the data in the first abnormal data that are greater than m times the normal fluctuation amplitude to obtain the second abnormal data, where m is greater than or equal to 2.
4. The method for identifying abnormal values in power grid operation data according to claim 1, characterized in that, The volatility of the large window is characterized by the standard deviation or the absolute deviation of the median.
5. The method for identifying abnormal values in power grid operation data according to claim 1 or 4, characterized in that, The volatility of the small window is characterized by the standard deviation or the absolute deviation of the median.
6. The method for identifying abnormal values in power grid operation data according to claim 1, characterized in that, The specific steps for filtering the power grid operation data based on the large-window volatility and small-window volatility to obtain the first abnormal data are as follows: If the large-window volatility and small-window volatility simultaneously satisfy the following rule, then the data is identified as the first abnormal data: a, Small window volatility ÷ Large window volatility > Set threshold b, Small window volatility > Large window volatility n%, where 0 < n ≤ 15.
7. The method for identifying abnormal values in power grid operation data according to claim 1, characterized in that, After filtering the power grid operation data based on the large-window volatility and small-window volatility to obtain the first abnormal data, the method further includes: judging the filtered non-first abnormal data according to the following rules: Calculate |Non-first abnormal data - Large window center value| ÷ Normal fluctuation benchmark value within the large window; if the value is greater than a preset threshold, then the non-first abnormal data is determined to be the first abnormal data.
8. The method for identifying abnormal values in power grid operation data according to claim 1, characterized in that, Before filtering the power grid operation data based on the large window volatility and small window volatility to obtain the first abnormal data, the process also includes: removing zero-value data points and absolute value maximum data points.
9. The method for identifying abnormal values in power grid operation data according to claim 1, characterized in that, The small window contains no fewer than three data points.
10. A system for identifying outlier values in power grid operation data for calculating power grid carrying capacity, characterized in that, include: The calculation module is used to calculate the large-window volatility and small-window volatility of power grid operation data based on the normal fluctuation benchmark within a large window and the normal fluctuation benchmark within a small window. The first filtering module is used to filter the power grid operation data based on the large window volatility and the small window volatility to obtain the first abnormal data; The second filtering module is used to filter the first abnormal data using the main rules to obtain the second abnormal data; The target acquisition module is used to compare the second abnormal data with the median deviation of the normal data within a preset range. If the data is not within the preset range of the median deviation of the normal fluctuation, it is the target abnormal data. The normal fluctuation benchmarks within the large window and the small window are obtained based on normal power grid operation data, with the large window containing more data points than the small window.