Intelligent power data anomaly detection method and system

Abstract

An intelligent power data anomaly detection method and system collects power operation data from each measuring point of each power device in the automated master station system to be tested, cleans the data to obtain valid offline data samples; then reduces the dimension of the valid offline data samples, calculates a time series sample sequence, and inputs it into an improved recurrent neural network for training to obtain a power data anomaly detection model; then uses the trained power data anomaly detection model to detect abnormal data and adopts an improved clustering algorithm to cluster abnormal power data; finally, uses an adaptive setting method to set an abnormal alarm interval, and issues an alarm when the abnormal data exceeds the abnormal alarm interval. The present invention accurately detects various anomalies and classifications generated during equipment operation. The adaptive alarm interval used can obtain a reasonable dynamic alarm interval, and can adaptively adjust the interval to adapt to different power equipment, thereby better reflecting the current status of the equipment.

Description

Technical Field

[0001] The present invention relates to an intelligent power data anomaly detection method and system, belonging to the technical field of power data processing. Background Art

[0002] With the integration of large-scale distributed generation and the widespread, random access to diverse loads, the scope of monitoring equipment in regional smart grid control systems is expanding. The number and variety of directly sourced and controlled power plants and stations are rapidly increasing, posing significant challenges to the safe operation and control of power grids. As the core support system for grid operation control and dispatching production management, the grid dispatching and control system is the hub for real-time data aggregation, processing, and dispatching control of various grid devices. Ensuring the quality of basic system data, the safety of remote equipment operation and control, and the reliability of the system's own operating status is crucial and urgently needs to be addressed for the safe and stable operation of the power grid.

[0003] Existing power grid control and management systems still have weaknesses in terms of safe and reliable operation. The lack of a quality control system for basic data within regional power grid control and management systems is hindering further improvements in safe grid operation. With the rapid development of regional power grids, the amount and variety of business data within regional smart grid control systems has gradually increased. However, data quality varies widely, and effective basic data quality assessment methods are lacking, impacting monitoring and dispatching decisions.

[0004] Specifically, 1) there is a lack of effective anomaly identification rules, and thresholds are mainly set based on experience, resulting in frequent false alarms and missed reports; 2) the level of investigation into the causes of data anomalies is low, and the correlation between faults has not been established, which limits the ability to eliminate data quality risks from the root causes such as missing measurements and incorrect model parameters; 3) data quality control is lagging, data quality information is scattered, and a complete data quality control system and a comprehensive and effective data quality assurance mechanism have not been formed, which restricts the in-depth mining of data value.

[0005] Therefore, it is necessary to increase the comprehensive monitoring and analysis of basic data quality based on the current status of power grid control and control system construction and application, and to identify, monitor and analyze all real-time data, historical data, calculated data and model abnormal data of the system, consolidate the data foundation, improve data quality, ensure the accuracy, timeliness, effectiveness and credibility of the data, and provide strong guarantees for the safe operation of the power grid. Summary of the Invention

[0006] In order to address the deficiencies in the prior art, the present invention aims to provide an intelligent power data anomaly detection method and system.

[0007] The present invention adopts the following technical solutions:

[0008] An intelligent power data anomaly detection method comprises the following steps:

[0009] Step 1: Collect the power operation data of each measuring point of each power equipment in the automated master station system to be tested. The power operation data includes data directly obtained through measurement, also known as direct data, which is divided into offline data and real-time data.

[0010] Step 2: Clean the abnormal data in the offline data caused by collection to obtain valid offline data samples;

[0011] Step 3: Reduce the dimension of the valid offline data samples and calculate the time series sample sequence;

[0012] Step 4: Input the time series sample sequence obtained in step 3 into the improved recurrent neural network for training to obtain a power data anomaly detection model;

[0013] Step 5: Use the trained power data anomaly detection model to detect abnormal data;

[0014] Step 6: clustering abnormal power data using an improved clustering algorithm;

[0015] Step 7: Use the adaptive setting method to set the dynamic alarm interval, and issue an alarm when the abnormal data exceeds the upper limit of the dynamic alarm interval.

[0016] In step 1, the offline data is the historical power operation data, including historical abnormal power data and historical normal power data, and the two types of data are labeled 1 and 0 respectively.

[0017] In step 2, abnormal data caused by collection include duplicate data, missing data, and erroneous data;

[0018] Repeated data refers to data measured repeatedly at the same measurement point;

[0019] Missing data refers to data that was not measured at a certain measurement point;

[0020] The method for determining erroneous data is to randomly select a data point in the offline data, calculate the Euler distance between all other data and it, and set the Euler distance threshold. If the Euler distance of a data point is less than the set threshold, the data is considered to be erroneous data; the setting value of the Euler distance threshold should ensure that at most 5% of the offline data are judged to be erroneous data.

[0021] In step 2, more than 75% of the offline data is selected as the offline data sample, and duplicate, missing, and erroneous data are cleaned to obtain a valid offline data sample:

[0022] The processing process includes:

[0023] For duplicate data, a deletion strategy is adopted to process it;

[0024] For missing data, first, establish the upper and lower limits of the data field; second, fill in the missing data by randomly selecting from the upper and lower limits;

[0025] The upper limit value is the maximum value of the selected offline data sample, and the lower limit value is the minimum value of the selected offline data sample;

[0026] For erroneous data, delete them first and then fill them in using the same method as missing data.

[0027] Step 3 includes the following:

[0028] Step 301: Use PCA principal component analysis to perform dimensionality reduction processing on the valid offline data samples, remove the correlation of each dimensional feature above three dimensions, and obtain the offline data samples after dimensionality reduction;

[0029] Step 302: Serialize the offline data samples after dimensionality reduction to obtain a time series sample sequence;

[0030] The time series sample sequence is divided into two parts by means of a sliding window. The window width is set to 1 to 5 times the sliding step size. The time series sample sequence is then normalized using the following formula to transform data of different dimensions to the same scale and convert data values ​​to the interval [0, 1]:

[0031]

[0032] Among them, E' is the offline data sample after time series, E is the offline data sample before time series, and E min is the minimum value of offline data samples before timing, E MAX It is the maximum value of offline data samples before timing.

[0033] In step 4, the improved recurrent neural network is composed of nn neural units A, where nn is at least 3 and is generally a multiple of 2 greater than 3;

[0034] Each neural unit A receives the stored information S of the previous moment t-1 , status information t-1 , the sample input information x at this moment t Then, generate the storage information S at the current moment t , status information t And output to the next neural unit A; for the last neural unit A, the state information O of the last neural unit A t This is the final prediction result;

[0035] Each of the neural units A includes a receiving link, a filtering link, a storage link and a state link.

[0036] The receiving link accepts the input information x at the current time t t The status information O at the previous moment t-1 , after performing a connection operation 7 on the two, the connection results are input into three first forward neural networks (2) based on the sigmoid activation function and a forward neural network based on the tanh activation function;

[0037] The connection operation 7 refers to the state information O t-1 The number followed by the input information x t numbers.

[0038] The filtering step combines the prediction result of the first forward neural network 2 based on the sigmoid activation function with the stored information S at the previous moment t-1 Perform bitwise multiplication to obtain the output of the filtering stage.

[0039] The storage link performs a bitwise multiplication operation 1 on the prediction result of the second forward neural network 3 based on the sigmoid activation function and the prediction result of the forward neural network 5 based on the tanh activation function, and then performs a bitwise addition operation on the bitwise multiplication result and the output of the filtering link to obtain the storage information S at the current moment. t .

[0040] The state link combines the prediction result of the third forward neural network 4 based on the sigmoid activation function with the stored information S at the current moment t After performing the bitwise multiplication operation 1, the current state information O is obtained t .

[0041] Feedforward neural networks contain at least two hidden layers.

[0042] Step 6 includes the following:

[0043] Step 601: select multiple abnormal power data as the abnormal power data sample set D, then the m-th dimension data of the i-th abnormal sample point is x im , centralize each dimension of data:

[0044]

[0045] It represents the result after abnormal sample point i is centralized in the m-th dimension data. The centralized sample set is composed of x jm Refers to the data of the mth dimension of the jth abnormal sample point, and n represents the total number of abnormal sample points around the selected abnormal sample point i.

[0046] Step 602: Calculate the covariance matrix G of the centralized sample set according to the following formula:

[0047]

[0048] in, Represents a centralized sample set The transpose of x jm Represents the data of the mth dimension of the jth abnormal sample point around the abnormal sample point i, j = 1, 2, ... n, m = 1, 2, ... d, d represents the total number of dimensions;

[0049] Step 603, perform eigenvalue decomposition on G, select the three largest eigenvalues ​​A1, A2, and A3, and use w1, w2, and w3 to represent their eigenvectors; at the same time, make these three eigenvectors orthogonal to form a new three-dimensional feature space.

[0050] Step 604, let W = (w1, w2, w3), by You can Projected into the three-dimensional feature space, a new abnormal sample set D'=(z1, z2, ..., z n’ ), where z n’ represents the n′th sample point in the new three-dimensional space, Represents the data of all dimensions of abnormal sample point i;

[0051] In step 605, the Euclidean distance formula is used to calculate the Euclidean distance d(i′, j′) between the abnormal sample point j′ and the abnormal sample point i′. The average distance MDT(D′) of the abnormal samples is calculated. Assuming the density of the sample point i′ is ρ(i′), the density expression formula can be obtained as follows:

[0052]

[0053]

[0054] Where i′ represents the i′th abnormal sample point in the abnormal sample set, j′ represents the j′th abnormal sample point in the abnormal sample set, and n′ represents the total number of samples in the abnormal sample set;

[0055] Select K largest sample point densities ρ(i′) as the initial cluster centers;

[0056] According to the cluster center, at least K cluster clusters are output to complete the clustering, that is, to complete the classification of abnormal sample points.

[0057] In step 7, the adaptive setting method is:

[0058] Divide the time period into periods based on the equipment's daily operating time. Each period can be no more than 0.5 times the equipment's operating time. Collect power operation data for mm consecutive periods and calculate the mean, standard deviation, and confidence interval of the power operation data for mm periods:

[0059]

[0060] Wherein, α represents 1-confidence level. In this embodiment, the confidence level is 95%, so α=1-0.95=0.05. It is called the z value, which is found by looking up the corresponding confidence level in the normal distribution table. represents the average value of the power operation data in mm consecutive cycles, and nn is the total number of power operation data in mm consecutive cycles;

[0061] The calculated confidence interval is used as the dynamic alarm interval for the current cycle for alarm monitoring. If the number of abnormal power data is within the confidence interval, the lower limit of the dynamic alarm interval is adjusted upward by 1% of the original lower limit value, or the upper limit of the dynamic alarm interval is adjusted downward by 1% of the original upper limit value. If the number of abnormal power data exceeds the confidence interval, an alarm is triggered.

[0062] After the alarm is triggered, the dynamic alarm interval is updated, the power operation data of the mm period is recollected, and the new confidence interval is calculated as the new dynamic alarm interval.

[0063] The present invention also discloses an intelligent power data anomaly detection system based on the intelligent power data anomaly detection method, comprising a data acquisition module, a data cleaning module, a data annotation module, a calculation module, a storage module, an abnormal power data determination module, a clustering module, an alarm dynamic interval setting module, and a reminder module;

[0064] Among them, the data acquisition module is used to collect direct data and transmit the direct data to the data cleaning module for data cleaning;

[0065] The data cleaning module inputs the cleaned offline data into the data annotation module to mark the historical abnormal power data and the historical normal power data; the data annotation module inputs the marked data into the calculation module;

[0066] The calculation module reduces the dimension of the input data and calculates the time series sample sequence;

[0067] The storage module is connected to the data acquisition module and the calculation module respectively, and is used to store the directly collected data and the calculated time series sample sequence; the storage module inputs the data into the abnormal power data judgment module;

[0068] The abnormal power data determination module uses the time sample sequence of offline data to train a power data anomaly detection model. The power data anomaly detection model is then used to determine the time sample sequence of real-time data, filter out abnormal power data, and input it into the storage module for storage.

[0069] The storage module inputs the determined abnormal power data into the clustering module, the alarm dynamic interval setting module and the reminder module;

[0070] The clustering module clusters the input data to obtain the classification of abnormal power data;

[0071] The alarm dynamic interval setting module adjusts the alarm dynamic interval according to the amount of input data, and inputs the updated alarm dynamic interval into the reminder module;

[0072] The reminder module alerts users based on the input alarm dynamic range and the amount of abnormal power data. The reminder module also includes an alarm module and a status marking module, which are connected to the status marking module. The alarm module uses audible and visual alarms and color warning lights to alert users.

[0073] The intelligent power data anomaly detection system also includes an equipment labeling module, a main control system, and a backup control system;

[0074] The backup control system is connected in parallel with the main control system. The equipment marking module is connected to the storage module, reminder module, main control system, and backup control system respectively. It is used to mark the key equipment in the system to be tested, obtain the power operation data of the key equipment, and process it;

[0075] The marked key equipment is highlighted in the detection system; the position of the key equipment is located using the positioning system, and the detection data and results are transmitted to the cloud server. When abnormal data occurs in the key equipment or the detection is interrupted, the backup control system is enabled, and the cloud server transmits the data to the backup control system to continue the detection. At the same time, the information of the backup control system is regularly transmitted to the cloud server for storage. When the main control system is restored to use, the local data of the backup control system is automatically cleared.

[0076] The beneficial effect of the present invention is that, compared with the prior art,

[0077] 1. The improved recurrent neural network and improved classification algorithm proposed in the present invention enable the detection method to timely and accurately detect various anomalies generated during equipment operation and classify them, thereby improving the detection efficiency, thereby enhancing the quality management level of the basic data of the control system and promoting the safe, stable and high-quality operation of the power grid.

[0078] 2. The present invention uses an adaptive alarm interval to obtain a reasonable dynamic alarm interval. The dynamic alarm interval can adaptively adjust the interval to adapt to different power equipment, thereby better reflecting the current status of the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 This is a flow chart of an intelligent power data anomaly detection method according to the present invention;

[0080] Figure 2 This is a structural diagram of an intelligent power data anomaly detection system according to the present invention;

[0081] Figure 3 This is a schematic diagram of the structure of the improved recurrent neural network used in the present invention;

[0082] Figure 4 Schematic diagram of the internal structure of each neural unit in the improved recurrent neural network used in the present invention.

[0083] Description of reference numerals:

[0084] A – Improving neural units in recurrent neural networks;

[0085] 1 – bitwise multiplication operation;

[0086] 2 – the first feedforward neural network based on the sigmoid activation function;

[0087] 3-The second forward neural network based on the sigmoid activation function;

[0088] 4-The third forward neural network based on the sigmoid activation function;

[0089] 5 – Feedforward neural network based on tanh activation function;

[0090] 6 – bitwise addition operation;

[0091] 7–Connection operation. DETAILED DESCRIPTION

[0092] The present application will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present application.

[0093] like Figure 1 As shown, the present invention discloses an intelligent power data anomaly detection method, comprising the following steps:

[0094] Step 1: The automated master station system controls multiple power devices and collects power operation data from each measurement point of each power device in the automated master station system. Power operation data includes data directly obtained through measurement, also known as directly collected data, and is divided into offline data and real-time data. Offline data is historical power operation data, including historical abnormal power data and historical normal power data, and these two types of data are labeled 1 and 0 respectively.

[0095] Each measurement point is separated by a time period of tt, preferably 1 hour;

[0096] Step 2: Clean the abnormal data in the offline data caused by collection to obtain valid offline data samples;

[0097] Abnormal data caused by collection include duplicate data, missing data, and erroneous data;

[0098] Repeated data refers to data measured repeatedly at the same measurement point;

[0099] Missing data refers to data that was not measured at a certain measurement point;

[0100] The method for determining erroneous data is to randomly select a data point in the offline data, calculate the Euler distance between it and all other data, and set the Euler distance threshold. If the Euler distance of a data point is less than the set threshold, the data is considered erroneous. The Euler distance threshold should be set so that at most 5% of the offline data is judged as erroneous data. More than 75% of the offline data is selected as the offline data sample, and duplicate, missing, and erroneous data are cleaned to obtain a valid offline data sample:

[0101] The processing process includes:

[0102] For duplicate data, a deletion strategy is adopted to process it;

[0103] For missing data, first, establish the upper and lower limits of the data field; second, fill in the missing data by randomly selecting from the upper and lower limits;

[0104] The upper limit value is the maximum value of the selected offline data sample, and the lower limit value is the minimum value of the selected offline data sample;

[0105] For erroneous data, delete them first and then fill them in using the same method as missing data.

[0106] Step 3: Reduce the dimension of the valid offline data samples and calculate the time series sample sequence;

[0107] Step 301: Use PCA principal component analysis to perform dimensionality reduction processing on the valid offline data samples, remove the correlation of each dimensional feature above three dimensions, and obtain the offline data samples after dimensionality reduction;

[0108] This step is to use fewer data dimensions while retaining more characteristics of the original data points;

[0109] Step 302: Serialize the offline data samples after dimensionality reduction to obtain a time series sample sequence;

[0110] The time series sample sequence is divided into two parts by a sliding window. The window width is set to 1 to 5 times the sliding step size. The time series sample sequence is then normalized using the following formula to transform data of different dimensions to the same scale. In this embodiment, the data values ​​are converted to the interval [0, 1]:

[0111]

[0112] Among them, E' is the offline data sample after time series, E is the offline data sample before time series, and E min is the minimum value of offline data samples before timing, E MAX It is the maximum value of offline data samples before timing.

[0113] Step 4: Input the time series sample sequence obtained in step 3 into the improved recurrent neural network for training to obtain a power data anomaly detection model;

[0114] The time series sample sequence is divided into a training set and a test set. The ratio of the training set to the test set must be greater than or equal to 7:3. In this embodiment, 7:3 is selected.

[0115] Due to time series reasons, the division of the data set cannot be randomly disrupted. Instead, according to the timeline of traffic collection, the first 70% of the time sample sequence is divided into the training set, and the last 30% of the time series sample sequence is divided into the test set.

[0116] It's important to note that in the actual operation of a power system, the system model is often not static, but rather changes with changes in the system's operating state. Therefore, a lagged historical feature is incorporated into the training set, indicating how long the model needs input to predict the next moment's data. Therefore, to achieve the above objectives, the power data anomaly detection model proposed in this paper is an improved recurrent neural network.

[0117] like Figure 3 As shown, the improved recurrent neural network is composed of n neural units A, where n is at least 3, and is generally a multiple of 2 greater than 3. In this embodiment, 16 is selected. Each neural unit A receives the stored information S of the neuron A at the previous moment. t-1, status information t-1 , the sample input information x at this moment t Then, generate the storage information S at the current moment t , status information t And output to the next neural unit A.

[0118] The initial state information is the label initially corresponding to the time series sample sequence, that is, the label corresponding to the original historical abnormal power data and the historical normal power data in the offline data, which are represented by the number 1 or 0 respectively.

[0119] Each neural unit A includes a receiving link, a filtering link, a storage link, and a state link; specifically, the receiving link processes the input information at the current moment and the state information at the previous moment and inputs them into four neural networks respectively; the filtering link performs a bitwise multiplication operation on the prediction result of the first neural network and the stored information at the previous moment and then outputs it; the storage link performs a bitwise multiplication operation on the prediction result of the second neural network and the fourth neural network, and then performs a bitwise addition operation with the output of the filtering link to obtain the stored information at the current moment; the state link performs a bitwise multiplication operation on the prediction result of the third neural network and the stored information at the current moment, and then obtains the current state information, and inputs the current state information into the neural unit A at the next moment; for the last neural unit A, its state information is the final prediction result. Preferably, the first to third neural networks are feed-forward neural networks based on the sigmoid activation function, and the fourth neural network is a feed-forward neural network based on the tanh activation function;

[0120] like Figure 4 As shown, the receiving link accepts the input information x at the current time t t The status information O at the previous moment t-1 , after connecting the two 7, the connected results are input into three forward neural networks based on sigmoid activation function and one forward neural network based on tanh activation function respectively;

[0121] The connection operation 7 here refers to the connection operation in the state information 0 t-1 The number followed by the input information x t digits;

[0122] The filtering step combines the prediction result of the first forward neural network 2 based on the sigmoid activation function with the stored information S at the previous moment t-1 Perform bitwise multiplication to obtain the output of the filtering step;

[0123] The storage link performs a bitwise multiplication operation 1 on the prediction result of the second forward neural network 3 based on the sigmoid activation function and the prediction result of the forward neural network 5 based on the tanh activation function, and then performs a bitwise addition operation on the bitwise multiplication result and the output of the filtering link to obtain the storage information S at the current moment. t ;

[0124] The state link combines the prediction result of the third forward neural network 4 based on the sigmoid activation function with the stored information S at the current moment t After performing the bitwise multiplication operation 1, the current state information O is obtained t ; For the last neural unit A, its state information O t This is the final prediction result;

[0125] The feedforward neural networks used in this model all contain at least 2 hidden layers;

[0126] Step 5: Use the trained power data anomaly detection model to detect abnormal data;

[0127] Use the methods of steps 2 and 3 to clean, reduce the dimension, and extract the time series of the real-time data, and then input it into the power data anomaly detection model trained in step 4 to obtain abnormal power data and normal power data;

[0128] Step 6: clustering abnormal power data using an improved clustering algorithm;

[0129] The selection of initial cluster centers in traditional clustering algorithms is random, so it is necessary to improve this clustering algorithm to reduce the randomness of the algorithm and thus improve the accuracy of the algorithm results:

[0130] The improved clustering method includes the following steps:

[0131] Step 601: select abnormal power data that accounts for more than 70% of the total as the abnormal power data sample set D. The abnormal power data in the abnormal power data sample set are called abnormal sample points. The m-th dimension data of the randomly selected abnormal sample point i is x im , centralize each dimension of data:

[0132]

[0133] It represents the result after abnormal sample point i is centralized in the m-th dimension data. The centralized sample set is composed of x jmIt refers to the data of the mth dimension of the jth abnormal sample point, and n represents the total number of abnormal sample points around the abnormal sample point i. When the Euclidean distance between the data of each dimension of the abnormal sample point in the abnormal power data sample set D and the dimension corresponding to the abnormal sample point i is less than the centralization distance threshold, the data is determined to be the data around the abnormal sample point i. The centralization distance threshold can be set according to actual conditions. The set centralization distance threshold ensures that at most 2% of the data are determined to be the data around the abnormal sample point i.

[0134] Step 602: Calculate the covariance matrix G of the centralized sample set according to the following formula:

[0135]

[0136] in, Represents a centralized sample set The transpose of x jm Represents the data of the mth dimension of the jth abnormal sample point around the abnormal sample point i, j = 1, 2, ... n, m = 1, 2, ... d, d represents the total number of dimensions;

[0137] Step 603: Use the SVD decomposition method to perform eigenvalue decomposition on G, sort all the obtained eigenvalues, select the three largest eigenvalues ​​A1, A2, and A3, and use w1, w2, and w3 to represent their eigenvectors respectively; at the same time, make these three eigenvectors orthogonal to form a new three-dimensional feature space.

[0138] Step 604, let W = (w1, w2, w3), by You can Projected into the three-dimensional feature space, a new abnormal sample set D'=(z1, z2, ..., z n’ ), where z n’ represents the n′th sample point in the new three-dimensional space, Represents the data of all dimensions of abnormal sample point i;

[0139] In step 605, the Euclidean distance formula is used to calculate the Euclidean distance d(i′, j′) between the abnormal sample point j′ and the abnormal sample point i′, and the average distance MDT(D′) from all other abnormal sample points to the abnormal sample point i′ is calculated. Assuming the density of the sample point i′ is ρ(i′), the density expression formula can be obtained as follows:

[0140]

[0141]

[0142] Where i′ represents the i′th abnormal sample point in the abnormal sample set, j′ represents the j′th abnormal sample point in the abnormal sample set, and n′ represents the total number of samples in the abnormal sample set;

[0143] Select K largest sample point densities ρ(i′) as the initial cluster centers;

[0144] According to the cluster center, at least K cluster clusters are output to complete the clustering, that is, to complete the classification of abnormal sample points.

[0145] The method of forming clusters is as follows: first, form a first-level cluster, select half of the average distance from all abnormal sample points to the initial cluster center of the required cluster as t1, and assign the abnormal sample points within t1 from the initial cluster center to the first-level cluster of the initial cluster center; then perform a second clustering to form a second-level cluster, the method is: randomly select an abnormal sample point PP from the abnormal sample points, if there is no second-level cluster, then use the sample point as the center of a second-level cluster; if there is a second-level cluster, then determine whether the distance between PP and each second-level cluster center is within t2, if it is within t2, then delete the abnormal sample point PP; if the distance from PP to all second-level cluster centers is greater than t2, then select PP as a new second-level cluster center; if the distance between PP and the second-level cluster center is equal to t2, then assign PP to any second-level cluster with a distance equal to 2 The second-level clustering is repeated until all abnormal sample points have been judged. The value of t2 can be set according to the actual situation. The setting principle is to ensure that at least K clusters can be formed.

[0146] Step 7: Use an adaptive setting method to set a dynamic alarm interval. When the abnormal data exceeds the upper limit of the dynamic alarm interval, an alarm is issued to realize abnormal online alarm of the measurement data of the automated master station.

[0147] Divide the time period according to the equipment's operating time in a day. The divided time period can be no more than 0.5 times the equipment's operating time. Collect the power operation data for mm consecutive periods and calculate the average value of the power operation data for mm periods. And the standard deviation σ, and the confidence interval:

[0148]

[0149] Wherein, α represents 1-confidence level. In this embodiment, the confidence level is 95%, so α=1-0.95=0.05. It is called the z value, which is found by looking up the corresponding confidence level in the normal distribution table. represents the average value of the power operation data in mm consecutive cycles, and nn is the total number of power operation data in mm consecutive cycles;

[0150] The calculated confidence interval is used as the dynamic alarm interval for the current cycle for alarm monitoring. If the number of abnormal power data is within the confidence interval, the dynamic alarm interval is fine-tuned by adjusting the lower limit of the dynamic alarm interval upward by 1% of the original lower limit value or adjusting the upper limit of the dynamic alarm interval downward by 1% of the original upper limit value. If the number of abnormal power data exceeds the upper limit of the dynamic alarm interval, an alarm is triggered.

[0151] After the alarm is triggered, the dynamic alarm interval is updated, the power operation data of the mm period is recollected, and the new confidence interval is calculated. Then, the dynamic alarm interval is fine-tuned in the same way according to the position of the number of abnormal power data within the confidence interval. This method enables the dynamic alarm interval to be updated.

[0152] During the update data collection period, an anomaly detection is performed using a manually set fixed threshold value, and an alarm is triggered when the value exceeds the alarm threshold value. Preferably, the manually set fixed threshold value is 90% of the upper limit value of the current dynamic alarm interval.

[0153] This method can obtain a more reasonable floating threshold value, and can adaptively adjust the threshold value to adapt to different power equipment, which can better reflect the current status of the equipment.

[0154] In addition, key power equipment in the power grid needs to be marked in advance, and the power operation data of key equipment will be obtained. The power operation data of key equipment will be defined as a set according to certain correlations for monitoring, such as system frequency, total network addition, interconnection line flow, etc. As long as any measurement value in the monitored set is not refreshed, the user will be reminded through alarms or status markings.

[0155] like Figure 2 As shown, the present invention discloses an intelligent power data anomaly detection system based on an intelligent power data anomaly detection method, comprising: a data acquisition module, a data cleaning module, a data annotation module, a calculation module, a storage module, an abnormal power data determination module, a clustering module, an alarm dynamic interval setting module, and a reminder module;

[0156] The data acquisition module is used to collect direct data and transmit the direct data to the storage module;

[0157] The data cleaning module reads the directly collected data from the storage module and cleans it, and inputs the valid offline data samples obtained after cleaning into the data annotation module;

[0158] The data labeling module labels the valid offline data samples as historical abnormal power data and historical normal power data, and inputs the labeled data into the calculation module;

[0159] The calculation module reduces the dimension of the input data, calculates the time series sample sequence, and inputs it into the storage module;

[0160] The storage module is used to store the directly collected data, the calculated time series sample sequence and the abnormal power data screened by the abnormal power data determination module; the storage module inputs the data into the abnormal power data determination module;

[0161] The abnormal power data determination module uses the time sample sequence of offline data to train a power data anomaly detection model. The power data anomaly detection model is then used to determine the time sample sequence of real-time data, filter out abnormal power data, and input it into the storage module for storage.

[0162] The storage module inputs the determined abnormal power data into the clustering module and the alarm dynamic interval setting module;

[0163] The clustering module clusters the input abnormal power data to obtain the classification of abnormal power data;

[0164] The alarm dynamic interval setting module adjusts the alarm dynamic interval according to the amount of abnormal power data input, and inputs the updated alarm dynamic interval into the reminder module;

[0165] The reminder module alerts the user based on the input alarm dynamic range and the amount of abnormal power data. The reminder module also includes an alarm module and a status annotation module, which are connected to the alarm module. The status annotation module annotates the current state, marking it as abnormal or normal based on whether the amount of power data exceeds the upper limit of the alarm dynamic range, and inputs the status to the alarm module. When the alarm module detects an abnormal state, it uses an audible and visual alarm and a color warning light to issue an alarm.

[0166] This detection system also includes an equipment marking module, a main control system, and a backup control system. The backup control system is connected in parallel with the main control system. The equipment marking module is connected to the storage module, the reminder module, the main control system, and the backup control system respectively, and is used to mark the key equipment in the system to be tested, obtain the power operation data of the key equipment and process it. The marked key equipment is highlighted in the detection system, for example, by using obvious colors to distinguish them in the display. Among them, the position of the key equipment is located using a positioning system, and the detection data and results are transmitted to the cloud server. When abnormal data occurs in the key equipment or the detection is interrupted, the backup control system is enabled, and the cloud server transmits the data to the backup control system to continue the detection. At the same time, the information of the backup control system is regularly transmitted to the cloud server for storage. When the main control system is restored to use, the local data of the backup control system is automatically cleared to achieve lightweight operation of the backup control system.

[0167] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation plans of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, and is not a limitation on the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A smart power data anomaly detection method, characterized in that: The intelligent power data anomaly detection method comprises the following steps: Step 1: Collect the power operation data of each measuring point of each power equipment in the automated master station system to be tested. The power operation data includes data directly obtained through measurement, also known as direct data, which is divided into offline data and real-time data. Step 2: Clean the abnormal data in the offline data caused by collection to obtain valid offline data samples; Step 3: Reduce the dimension of the valid offline data samples and calculate the time series sample sequence; Step 4: Input the time series sample sequence obtained in step 3 into the improved recurrent neural network for training to obtain a power data anomaly detection model; The improved recurrent neural network is composed of n neural units (A), each of which includes a receiving link, a filtering link, a storage link, and a state link; Step 5: Use the trained power data anomaly detection model to detect abnormal data; Step 6: clustering abnormal power data using an improved clustering algorithm; Abnormal power data is selected to form a sample set D. Each dimension of data is centered, and the covariance matrix G of the centralized sample set is calculated. The eigenvalues ​​of G are decomposed, and a three-dimensional feature space is constructed using the three largest eigenvalues ​​and orthogonal eigenvectors. The centralized sample set is projected into the three-dimensional feature space to form a new abnormal sample set. The K largest sample point densities are selected as the initial cluster centers using the density formula. At least K cluster clusters are output based on the cluster centers to complete the classification of abnormal sample points. Step 7: Use an adaptive setting method to set a dynamic alarm interval, and issue an alarm when the abnormal data exceeds the upper limit of the dynamic alarm interval; The adaptive setting method is to divide the time period according to the daily operation time of the equipment, calculate the mean, standard deviation and confidence interval based on the periodic power operation data, and use the confidence interval as the dynamic alarm interval of the current period.

2. The intelligent power data anomaly detection method according to claim 1, characterized in that: In step 1, the offline data is the historical power operation data, including historical abnormal power data and historical normal power data, and the two types of data are labeled 1 and 0 respectively.

3. The intelligent power data anomaly detection method according to claim 1, characterized in that: In step 2, the abnormal data caused by collection includes duplicate data, missing data and erroneous data; The repeated data are data measured repeatedly at the same measurement point; The missing data refers to data that is not measured at a certain measurement point; The method for determining the erroneous data is to randomly select a data point in the offline data, calculate the Euler distance between all other data and it, set the Euler distance threshold, and if the Euler distance of a data point is less than the set threshold, the data is considered to be erroneous data.

4. The intelligent power data anomaly detection method according to claim 3, characterized in that: In step 2, duplicate, missing, and erroneous data in the offline data sample are cleaned to obtain a valid offline data sample: The processing process includes: For duplicate data, a deletion strategy is adopted to process it; For missing data, first, establish the upper and lower limits of the data field; second, fill in the missing data by randomly selecting from the upper and lower limits; The upper limit value is the maximum value of the selected offline data sample, and the lower limit value is the minimum value of the selected offline data sample; For erroneous data, delete them first and then fill them in using the same method as missing data.

5. The intelligent power data anomaly detection method according to claim 1, characterized in that: The step 3 includes the following contents: Step 301: Use PCA principal component analysis to perform dimensionality reduction processing on the valid offline data samples, remove the correlation of the features of each dimension above three dimensions, and obtain the offline data samples after dimensionality reduction; Step 302: Serialize the offline data samples after dimensionality reduction to obtain a time series sample sequence; The time series sample sequence is divided into two parts by means of a sliding window. The window width is set to 1 to 5 times the sliding step size. The time series sample sequence is then normalized using the following formula to transform data of different dimensions to the same scale and convert data values ​​to the interval [0, 1]: Among them, E' is the offline data sample after time series, E is the offline data sample before time series, and E min is the minimum value of offline data samples before timing, E MAX It is the maximum value of offline data samples before timing.

6. The intelligent power data anomaly detection method according to claim 1, characterized in that: In step 4, nn is at least 3.

7. The intelligent power data anomaly detection method according to claim 6, characterized in that: The receiving link receives the sample input information x at the current time t t The status information O at the previous moment t-1 After connecting the two, the connected results are input into three first forward neural networks based on sigmoid activation function and one forward neural network based on tanh activation function respectively; The connection operation refers to the state information O t-1 The number followed by the input information x t numbers.

8. The intelligent power data anomaly detection method according to claim 6, characterized in that: The filtering link combines the prediction result of the first forward neural network 2 based on the sigmoid activation function with the stored information S at the previous moment t-1 Perform bitwise multiplication to obtain the output of the filtering stage.

9. The intelligent power data anomaly detection method according to claim 6, characterized in that: The storage link performs a bitwise multiplication operation 1 on the prediction result of the second forward neural network 3 based on the sigmoid activation function and the prediction result of the forward neural network 5 based on the tanh activation function, and then performs a bitwise addition operation on the bitwise multiplication result and the output of the filtering link to obtain the storage information S at the current moment t .

10. The intelligent power data anomaly detection method according to claim 6, characterized in that: The state link combines the prediction result of the third forward neural network 4 based on the sigmoid activation function with the stored information S at the current moment t After performing the bitwise multiplication operation 1, the current state information O is obtained t .

11. The intelligent power data anomaly detection method according to claim 10, characterized in that: The feedforward neural network includes at least two hidden layers.

12. The intelligent power data anomaly detection method according to claim 11, characterized in that: The step 6 includes the following contents: Step 601: select multiple abnormal power data as the abnormal power data sample set D, then the m-th dimension data of the i-th abnormal sample point is x im , centralize each dimension of data: It represents the result after abnormal sample point i is centralized in the m-th dimension data. The centralized sample set is composed of x jm Refers to the data of the mth dimension of the jth abnormal sample point, and n represents the total number of abnormal sample points around the selected abnormal sample point i; Step 602: Calculate the covariance matrix of the centralized sample set according to the following formula: in, Represents a centralized sample set The transpose of x jm Represents the data of the mth dimension of the jth abnormal sample point around the abnormal sample point i, j = 1, 2, ... n, m = 1, 2, ... d, d represents the total number of dimensions; Step 603: Perform eigenvalue decomposition on G, select the three largest eigenvalues ​​A1, A2, and A3, and represent their eigenvectors with w1, w2, and w3; at the same time, make these three eigenvectors orthogonal to form a new three-dimensional feature space; Step 604, let W = (w1, w2, w3), by You can Projected into the three-dimensional feature space, a new abnormal sample set D'=(z1, z2, ..., z n’ ), where z n’ represents the n'th sample point in the new three-dimensional space, Represents the data of all dimensions of abnormal sample point i; In step 605, the Euclidean distance formula is used to calculate the Euclidean distance d(i', j') between the abnormal sample point j' and the abnormal sample point i'. The average distance MDT(D') of the abnormal samples is calculated. Assuming the density of the sample point i' is ρ(i'), the density expression formula can be obtained as follows: Where i' represents the i'th abnormal sample point in the abnormal sample set, j' represents the j'th abnormal sample point in the abnormal sample set, and n' represents the total number of samples in the abnormal sample set; Select K largest sample point densities ρ(i') as the initial cluster centers; According to the cluster center, at least K cluster clusters are output to complete the clustering, that is, to complete the classification of abnormal sample points.

13. The intelligent power data anomaly detection method according to claim 1, characterized in that: In step 7, the adaptive setting method is: Divide the time period into periods based on the equipment's daily operating time. Each period can be no more than 0.5 times the equipment's operating time. Collect power operation data for mm consecutive periods and calculate the mean, standard deviation, and confidence interval of the power operation data for mm periods: Among them, α represents 1-confidence level, and the confidence level is 95%, then α=1-0.95=0.05, It is called the z value, which is found by looking up the corresponding confidence level in the normal distribution table. represents the average value of the power operation data in mm consecutive cycles, and nn is the total number of power operation data in mm consecutive cycles; The calculated confidence interval is used as the dynamic alarm interval for the current cycle for alarm monitoring. If the number of abnormal power data is within the confidence interval, the lower limit of the dynamic alarm interval is adjusted upward by 1% of the original lower limit value, or the upper limit of the dynamic alarm interval is adjusted downward by 1% of the original upper limit value. If the number of abnormal power data exceeds the confidence interval, an alarm is triggered. After the alarm is triggered, the dynamic alarm interval is updated, the power operation data of the mm period is recollected, and the new confidence interval is calculated as the new dynamic alarm interval.

14. The intelligent power data anomaly detection system according to any one of claims 1 to 13, comprising a data acquisition module, a data cleaning module, a data annotation module, a calculation module, a storage module, an abnormal power data determination module, a clustering module, an alarm dynamic interval setting module, and a reminder module, characterized in that: The data acquisition module is used to collect direct data and transmit the direct data to the data cleaning module for data cleaning; The data cleaning module inputs the cleaned offline data into the data annotation module to annotate historical abnormal power data and historical normal power data; The data annotation module inputs the annotated data into the calculation module; The calculation module reduces the dimension of the input data and calculates the time series sample sequence; The storage module is connected to the data acquisition module and the calculation module respectively, and is used to store the directly collected data and the calculated time series sample sequence; The storage module inputs the data into the abnormal power data determination module; The abnormal power data determination module uses the time sample sequence of offline data to train a power data anomaly detection model. The power data anomaly detection model is then used to determine the time sample sequence of real-time data, filter out abnormal power data, and input it into the storage module for storage. The storage module inputs the determined abnormal power data into the clustering module, the alarm dynamic interval setting module and the reminder module; The clustering module clusters the input data to obtain the classification of abnormal power data; The alarm dynamic interval setting module adjusts the alarm dynamic interval according to the amount of input data, and inputs the updated alarm dynamic interval into the reminder module; The reminder module reminds the user based on the input alarm dynamic range and the number of abnormal power data; the reminder module also includes an alarm module and a status marking module, and the alarm module is connected to the status marking module; the alarm module uses an audible and visual alarm and a color warning light to remind the user.

15. The intelligent power data anomaly detection system according to claim 14, characterized in that: The intelligent power data anomaly detection system also includes an equipment labeling module, a main control system, and a backup control system; The backup control system is connected in parallel with the main control system. The equipment marking module is connected to the storage module, reminder module, main control system, and backup control system respectively. It is used to mark the key equipment in the system to be tested, obtain the power operation data of the key equipment, and process it; The marked key equipment is highlighted in the detection system; the position of the key equipment is located using the positioning system, and the detection data and results are transmitted to the cloud server. When abnormal data occurs in the key equipment or the detection is interrupted, the backup control system is enabled, and the cloud server transmits the data to the backup control system to continue the detection. At the same time, the information of the backup control system is regularly transmitted to the cloud server for storage. When the main control system is restored to use, the local data of the backup control system is automatically cleared.

Images