A method for detecting abnormal power consumption in power grid based on attention mechanism and residual network

Abstract

The present invention discloses a method for detecting abnormal power consumption in power grid based on attention mechanism and residual network, comprising the following steps: S1, data collection, using a large number of smart meters to collect users' massive power consumption information at high frequency, and collect a large amount of power data; S2, data preprocessing, S3, selecting training data set and test data set; S4, setting initial model parameters; S5, training a classification algorithm based on attention mechanism and residual network, using training samples to train the classification algorithm based on attention mechanism and residual network; S6, predicting power consumption data; S7, result analysis. The present invention realizes abnormal power consumption detection in power grid with high accuracy. In practical applications, the algorithm is very stable and has good processing capabilities for complex environments and data.

Description

Technical Field

[0001] The present invention relates to the technical field of electricity consumption detection, and in particular to a method for detecting abnormal electricity consumption in a power grid based on an attention mechanism and a residual network. Background Art

[0002] With the rapid development of new generation information technologies such as the Internet of Things and big data, various industries are undergoing major changes. In recent years, the country has attached great importance to the sustainable development of energy, which has led to a sharp increase in the demand for electricity in the society's industrial production and users' daily lives. However, the traditional power network structure is relatively simple and can only meet the basic electricity needs of some users. It mainly relies on non-renewable energy for power generation and cannot cope with the problem of high-speed consumption of electric energy in today's society. Therefore, at present, all countries are committed to developing the next generation of power systems with a variety of new energy sources such as distributed energy, mainly by integrating the two-way flow of information and energy to combine new systems of automation and intelligent management with existing power systems, so as to form a modern, information-based and intelligent power network.

[0003] Anomaly detection in smart grids includes a large number of data-driven abnormal power consumption detection technologies. Anomaly detection is also called outlier detection, which mainly detects abnormal patterns that do not conform to expected behaviors. Anomaly detection has been widely used in intrusion detection, fraud detection, industrial control systems and other fields. Therefore, it is also very important to study anomaly detection at present.

[0004] Anomaly detection is a method of detecting and classifying data in system equipment. It mainly determines whether an abnormal situation occurs based on the data of the detected object. Finally, the normal samples and abnormal samples in the data are classified, and the system equipment with abnormalities is found. Anomaly detection is mainly divided into two parts: feature extraction and classification. When an abnormality occurs in the system, it usually causes changes in the data. Anomaly detection mainly uses machine learning methods to extract data features, thereby mining the potential characteristics within the data, training and learning through these features, and finding abnormal data, thereby finding the system equipment with abnormalities.

[0005] The applicant, Shanghai Institute of Electric Power, with application publication number CN113011997B, proposed a method for detecting abnormal electricity consumption behavior of power grid users, including collecting and acquiring user electricity consumption data; preprocessing user electricity consumption data to obtain a training set; constructing an abnormal electricity consumption detection model based on AdaBoost ensemble learning; training the abnormal electricity consumption detection model using the training set; inputting user feature data into the trained abnormal electricity consumption detection model; analyzing and identifying abnormal electricity consumption behavior; the abnormal electricity consumption detection model outputs classification results; and using classification performance indicators to evaluate the classification results.

[0006] The applicant, Sichuan University, with authorization announcement number CN106296465B, proposed a method for detecting abnormal electricity consumption behavior in a smart grid. The method describes the user's electricity consumption behavior by the change in the amount of electricity used by each user's different electricity consumption business types over time, fully reflecting the user's electricity consumption characteristics; the users are clustered according to the similarity of their electricity consumption behaviors, and the deviation of abnormal electricity consumption behaviors from their normal behavior patterns is characterized by the mutual correlation of users' electricity consumption behaviors in the same user cluster and the correlation between users' electricity consumption behaviors and their historical electricity consumption behaviors, thereby ensuring the effectiveness of abnormal electricity consumption behavior detection; by analyzing the cumulative user electricity consumption behavior data within a fixed time interval, the method can avoid introducing too much noise in the analysis of massive real-time data while meeting the detection accuracy requirements, thereby making the method have better dynamic adaptability; by analyzing the cumulative user electricity consumption behavior data within a fixed time interval, the amount of original data is reduced.

[0007] The above method is innovative and can play a good role in power grid anomaly detection. However, the noise processing of power grid data still needs to be further improved, especially considering the inevitable noise interference in actual industrial applications and the simultaneous occurrence of load changes in the power system, resulting in the detection algorithm model The robustness and generalization ability are not strong enough.

[0008] The residual network is a convolutional neural network proposed by four scholars from Microsoft Research. It won the image classification and object recognition in the 2015 ImageNet Large Scale Visual Recognition Challenge (ILSVRC). The residual network is easy to optimize and can improve accuracy by increasing the depth. The residual block inside it uses skip connections to alleviate the gradient vanishing problem caused by increasing the depth in the deep neural network. The residual network was proposed mainly to solve the degradation problem in deep learning, that is, as the number of layers of the deep neural network increases, the performance of the model decreases.

[0009] The activation function is a function that runs on the neurons of the artificial neural network and is responsible for mapping the input of the neuron to the output.

[0010] The function used to calculate the loss is called the loss function. The quality of each prediction of the model is measured by the loss function.

[0011] Random forest is an integrated learning model, which refers to a classifier that uses multiple decision trees to train and predict samples. Summary of the invention

[0012] The present invention proposes a method for detecting abnormal power consumption in a power grid based on an attention mechanism and a residual network, which can detect abnormal power consumption in a power grid with high accuracy. In practical applications, the algorithm is very stable and has good processing capabilities for complex environments and data.

[0013] In order to achieve the above object, the present invention adopts the following technical solutions:

[0014] A method for detecting abnormal power consumption in a power grid based on an attention mechanism and a residual network comprises the following steps:

[0015] S1. Data collection: using a large number of smart meters to collect users' massive electricity consumption information at high frequency and collect a large amount of power data;

[0016] S2. Data preprocessing: pre-processing the collected data, such as filling missing values, removing outliers, reducing dimensions to eliminate redundancy, and normalizing the data to eliminate prediction errors caused by different dimensions;

[0017] S3, select training data set and test data set;

[0018] S4, setting the initial parameters of the model;

[0019] S5. Train the classification algorithm based on the attention mechanism and the residual network. Use the training samples to perform model training on the classification algorithm based on the attention mechanism and the residual network.

[0020] S6. Predict electricity consumption data, test the model using samples of the test data set, and predict abnormal electricity consumption data;

[0021] S7. Result analysis: evaluate and analyze the abnormal power consumption data predicted by S6.

[0022] Preferably, the classification algorithm based on the attention mechanism and the residual network in S5 includes the following steps:

[0023] S1. The input data is first input into the random forest, and the attention module is obtained by calculating the feature importance;

[0024] S2, the input features are weighted by the weight of the attention module;

[0025] S3, input to the convolution layer and pooling layer;

[0026] S4, input to 3 improved residual stacking blocks;

[0027] S5, input to the attention module;

[0028] S6, input to the Dropout layer and Softmax layer;

[0029] S7. Output the classification results.

[0030] Preferably, the method of training the attention module is: by inputting data into a random forest, calculating the importance of each feature in the data, and all the importance values ​​constitute an attention module for the next step of feature enhancement.

[0031] Preferably, the convolutional layer is the key component that makes the convolutional neural network (CNN) different from the traditional fully connected neural network; the convolution kernel operation in the convolutional layer has fewer parameters than the transformation matrix in the fully connected layer.

[0032] Preferably, the classification algorithm based on the attention mechanism and the residual network includes an attention module, a convolutional layer, batch normalization, an improved activation function, a residual module, a squeezing and excitation network structure, a cross entropy loss function, a Pooling layer global average pooling, and an improved residual module.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention uses historical data sets for automatic learning and establishes corresponding detection models. In the era of big data, the power data of users in smart grids has the characteristics of massive, high speed, and diversity. A large number of smart meters collect massive power consumption information of users at high frequency. The large amount of power data collected provides a reliable basis for data-driven abnormal power consumption detection technology. The present invention mainly analyzes and detects based on the big data information of user power consumption behavior; with innovative application, the attention mechanism and residual network are introduced into the abnormal power consumption detection of power grid, achieving high detection accuracy, strong anti-interference ability, strong robustness, and strong generalization ability.

[0035] The classification algorithm proposed in the present invention has the characteristics of high classification accuracy and fast training speed. The proposed model has the characteristics of strong anti-interference ability, strong robustness and strong generalization ability. In the task of abnormal power consumption detection in the power grid, the method proposed in the present invention can realize abnormal power consumption detection in the power grid with higher accuracy. In practical applications, the algorithm has very stable capabilities and has good processing capabilities for complex environments and data. It can timely and efficiently identify abnormal power consumption in the power grid, which plays an important role in improving enterprise production capacity, improving production efficiency and energy conservation and environmental protection in the park. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a method for detecting abnormal power consumption in a power grid based on an attention mechanism and a residual network proposed by the present invention;

[0037] Figure 2A flow chart of a classification algorithm for a method for detecting abnormal power consumption in a power grid based on an attention mechanism and a residual network proposed by the present invention;

[0038] Figure 3 Flowchart of two residual module improvement methods in the present invention. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0040] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0041] Reference Figure 1 :A method for detecting abnormal power consumption in power grid based on attention mechanism and residual network, comprising the following steps

[0042] S1. Data collection: using a large number of smart meters to collect users' massive electricity consumption information at high frequency and collect a large amount of power data;

[0043] S2. Data preprocessing: pre-processing the collected data, such as filling missing values, removing outliers, reducing dimensions to eliminate redundancy, and normalizing the data to eliminate prediction errors caused by different dimensions;

[0044] S3, select training data set and test data set;

[0045] S4, setting the initial parameters of the model;

[0046] S5. Train the classification algorithm based on the attention mechanism and the residual network. Use the training samples to perform model training on the classification algorithm based on the attention mechanism and the residual network.

[0047] S6. Predict electricity consumption data, test the model using samples of the test data set, and predict abnormal electricity consumption data;

[0048] S7. Result analysis: evaluate and analyze the abnormal power consumption data predicted by S6.

[0049] Reference Figure 2 , the classification algorithm based on attention mechanism and residual network in S5 includes the following steps:

[0050] S1. The input data is first input into the random forest, and the attention module is obtained by calculating the feature importance;

[0051] S2, the input features are weighted by the weight of the attention module;

[0052] S3, input to the convolution layer and pooling layer;

[0053] S4, input to 3 improved residual stacking blocks;

[0054] S5, input to the attention module;

[0055] S6, input to the Dropout layer and Softmax layer;

[0056] S7. Output the classification result.

[0057] The classification algorithm based on the attention mechanism and residual network includes (1) attention module, (2) convolution layer, (3) batch normalization, (4) improved activation function, (5) residual module, (6) squeeze and excitation network structure, (7) cross entropy loss function, (8) global average pooling of Pooling layer, and (9) improved residual module. Next, the design and improvement of each layer and module are introduced in detail.

[0058] (1) Attention Module

[0059] In the algorithm proposed in the present invention, the attention mechanism is introduced into the task of abnormal power consumption detection in the power grid, and combined with the improved residual network, feature enhancement and improved classification accuracy are achieved.

[0060] The way to train the attention module is to input the data into the random forest and calculate the importance of each feature in the data. All the importance values ​​form the attention module for the next step of feature enhancement.

[0061] In a random forest (RF), the dozens or even hundreds of trees generated are all drawn from the bag defined as B by autonomous sampling, and the unselected data is called out-of-bag data (OOB). Define C as all data, and C must include B. Define X n×p is a matrix data set with p features and n total data. Define y as a class label vector, which is a 1-dimensional label vector. The values ​​in the vector represent the class values ​​of the corresponding data in the data set. The feature values ​​in the data can be randomly arranged when input into the random forest, and the random forest can calculate the feature importance by obtaining the error corresponding to the rearranged feature input. In the random forest algorithm, each feature xj They all correspond to a set of feature permutation tests, and the feature permutation test is rearranged. In the out-of-bag data, the classification errors in various cases are compared by using the features of the original feature sorting and the rearranged features to measure the feature importance. In other words, if the classification error of the out-of-bag data becomes larger, it is because after the reordering, the discrimination of the features decreases and important feature information is replaced. Define T as the number of decision trees in the random forest, and the number of decision trees is the same as the number of out-of-bag data test sets established. Therefore, within a certain range, the more decision trees there are, the more accurate the degree of measuring feature importance. Define feature importance as J a , the formula is as follows:

[0062]

[0063] Among them, y i represents the label category corresponding to the i-th out-of-bag data, I(x) is the indicative function, and h k (i) is the function for predicting the label of sample i, which is a sample in dataset B k The samples in is the permutation feature x j The classification label after .

[0064] Therefore, by calculating the feature importance, we can obtain the importance of each feature in the sample. The importance is the contribution of all features in the sample. The importance value is used as the attention vector and weighted with the original feature to achieve the purpose of enhancing important features.

[0065] (2) Convolutional layer

[0066] The convolution layer is the key component that makes the convolution neural network (CNN) different from the traditional fully connected neural network. The convolution kernel operation in the convolution layer has fewer parameters than the transformation matrix in the fully connected layer. The convolution operation between the input feature map and the convolution kernel (by adding a bias term) can be expressed as:

[0067]

[0068] Where: x i is the i-th channel of the input feature map; y j is the jth channel of the output feature map; k is the convolution kernel; b is the bias term; M j is the set used to calculate the j-th channel of the output feature map.

[0069] (3) Batch Normalization

[0070] Batch normalization (BN) is a normalization method proposed to solve the problem of internal covariance shift. In the proposed classification algorithm, batch normalization is introduced, which not only speeds up the convergence of the model, but also, more importantly, alleviates the problem of "gradient diffusion" in deep networks to a certain extent, making the training of deep network models easier and more stable.

[0071] Batch normalization selects a small batch during deep learning training, and then calculates the mean and variance of the small batch data. After this process, the input of each layer of the neural network maintains the same distribution during training.

[0072] Different from the general normalization method, batch normalization is an operation embedded between layers in a deep neural network. The calculation process of BN is expressed as:

[0073]

[0074]

[0075]

[0076] y i =γz i +β

[0077] Where: x i and i Respectively represent the input and output features of the i-th observation in the batch; N batch is the number of samples in each batch in the classification task; γ and β are two trainable parameters that can adaptively learn a more appropriate feature distribution; ε is a constant close to zero.

[0078] (4) Improved activation function

[0079] As the most common activation function, ReLU solves the problems of S-shaped local gradient explosion and gradient disappearance, and accelerates the convergence of neural networks.

[0080] The ReLU algorithm is as follows:

[0081] y=max(0,x)

[0082] Where: x and y are the input and output of the ReLU activation function respectively.

[0083] When the input signal of the ReLU algorithm has oscillations, it will discard such vibration signals, which weakens the classification prediction ability of the model. In this regard, the method of the present invention uses LReLU as the activation function to solve the problem when the input signal has oscillations. The specific algorithm is as follows:

[0084]

[0085] Where: x and y are the input and output of the LReLU activation function respectively; a is determined based on practical experience. A large number of experiments have shown that the best effect is achieved when the value of a is in the range of 0 to 0.5.

[0086] (5) Basic principles of residual modules

[0087] The residual building block (RBB) is the core of Resnet. RBB is implemented by skipping convolutional layer blocks using shortcut connections, which avoids gradient explosion and vanishing, helps to construct deeper neural network structures, and improves the final performance of fault diagnosis.

[0088] The execution path of the convolutional layer block F(x) is "input x→BN layer→activation function ReLU→convolutional layer→BN layer→activation function ReLU→convolutional layer→output F(x)". When the input and output dimensions of the convolutional layer block are the same, the output value of the shortcut connection is the input value x. The final output result of the residual module is shown in the following formula:

[0089] y=F(x)+x

[0090] When the dimensions of the input and output are different, the quick connection needs to use a convolution layer with a convolution kernel size of 1x1 to match the dimensions of the output result, so as to obtain the output H(x) of the quick connection. The final output result is shown in the following formula:

[0091] y=F(x)+H(x)

[0092] (6) Squeeze and Incentive Network Structure

[0093] The present invention adopts a squeeze and excitation network structure (SENet), which can automatically obtain the importance of each channel and enhance the connection between channels, thereby achieving the purpose of improving model performance. The core of this structure is the two major operations of squeeze and excitation.

[0094] The Squeeze operation is to perform global pooling on the input feature map, compressing each feature map into a real number with a global receptive field. The specific algorithm is shown in the following formula:

[0095]

[0096] Among them, in the formula: x i Represents the input as the i-th feature map of size H×W.

[0097] The Exctation operation is mainly composed of 2 fully connected layers and 2 activation functions, which can help capture channel correlation and generate the weights of the corresponding channels. The algorithm is shown in the following formula:

[0098] y i =F ex (F sq (x i ),ω)=σ(ω2δ(ω1F sq (x i )))

[0099] Where: ω1 represents the calculation of the first fully connected layer; ω2 represents the calculation of the second fully connected layer; F(x) represents the output value after the Squeeze operation; δ represents the activation function ReLU; σ is the Sigmoid function. The specific algorithm is as follows:

[0100]

[0101] Where: x represents the output value after 2 full connection calculations.

[0102] (7) Cross Entropy Loss Function

[0103] The cross entropy loss function Softmax is generally used as the activation function of the final output layer. It fixes the output value of the neural network between (0, 1) to represent the probability of different events. The algorithm is shown in the following formula:

[0104]

[0105] Where: N class Represents the categories involved in the classification task, x j represents the jth output of the previous layer; y j represents the j-th prediction value of the neural network.

[0106] (8) Pooling layer global average pooling

[0107] Global average pooling (GAP) is an operation that averages feature maps. It can greatly reduce the parameters during neural network training and speed up the calculation of neural networks. It is a commonly used deep learning pooling operation.

[0108] (9) Improved residual module

[0109] The present invention proposes two methods for improving the residual module, such as Figure 3 shown.

[0110] The core part of the improvement is to add a branched SENet structure after the second convolutional layer of the residual module, generate the weights of the corresponding channels, and finally multiply them with the output results of the second convolutional layer; at the same time, LReLU is used as the activation function.

[0111] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for detecting abnormal power consumption in a power grid based on attention mechanism and residual network, characterized in that , including the following steps, S1. Data collection: using a large number of smart meters to collect users' massive electricity consumption information at high frequency and collect a large amount of power data; S2. Data preprocessing: pre-processing the collected data. Data preprocessing includes filling missing values, removing outliers, and reducing dimensions to eliminate redundancy. In order to eliminate the prediction error caused by different dimensions, the data will be normalized. S3, select training data set and test data set; S4, setting the initial parameters of the model; S5. Train the classification algorithm based on the attention mechanism and the residual network. Use the training samples to perform model training on the classification algorithm based on the attention mechanism and the residual network. S6. Predict electricity consumption data, test the model using samples of the test data set, and predict abnormal electricity consumption data; S7, result analysis, evaluation and analysis of the abnormal power consumption data predicted in S6; The classification algorithm based on attention mechanism and residual network in S5 includes the following steps: S51, the input data is first input into the random forest, and the attention module is obtained by calculating the feature importance; S52, the input feature is weighted by the weight of the attention module; S53, input to the convolution layer and pooling layer; S54, input to 3 improved residual stacking blocks; S55, input to the attention module; S56, input to the Dropout layer and Softmax layer; S57, output classification results; The classification algorithm based on attention mechanism and residual network includes attention module, convolution layer, batch normalization, improved activation function, residual module, squeezing and excitation network structure, cross entropy loss function, global average pooling of Pooling layer, and improved residual module.

2. According to claim 1, a method for detecting abnormal power consumption in a power grid based on an attention mechanism and a residual network is characterized in that: The way to train the attention module is: by inputting the data into the random forest, calculating the importance of each feature in the data, and all the importance values ​​​​compose the attention module for the next step of feature enhancement.

3. According to claim 1, a method for detecting abnormal power consumption in a power grid based on an attention mechanism and a residual network is characterized in that: Convolutional layers are the key component that makes convolutional neural networks different from traditional fully connected neural networks; the convolution kernel operation in a convolutional layer has fewer parameters than the transformation matrix in a fully connected layer.

Images