Energy consumption measurement data abnormity monitoring method and device and storage medium

Abstract

The invention discloses an energy consumption measurement data abnormity monitoring method and device and a storage medium. The monitoring method comprises the following steps: constructing a short-term monitoring model, and monitoring data exception in the early stage of energy consumption of a user according to the short-term monitoring model; and a long-term monitoring model is constructed, and the long-term monitoring model monitors data exceptions in the middle and later periods of energy consumption of the user for a period of time according to the existing energy consumption data. Based on the condition that the early-stage data of the power distribution system is less, the early-stage data abnormity of the power distribution system is monitored through the short-term monitoring model, and effective monitoring can be carried out even under the condition that the data is less; the long-term monitoring model is combined with available energy data to monitor data abnormity in long-term operation of the power distribution system, so that the reliability of the system is improved; and short-term monitoring and long-term monitoring are combined, so that the running state of the power distribution system can be comprehensively monitored.

Description

Technical Field

[0001] The present invention relates to the technical field of energy metering data monitoring, and in particular to an energy metering data abnormality monitoring method, device and storage medium. Background Art

[0002] There may be many reasons for the change of user's electricity consumption pattern, such as high-consuming appliances directly connected to external feeders, which makes the meter unable to record electricity usage, injecting tampered data into smart meters and transmitting them to the data center, and users moving without notifying the power center. These behaviors are abnormal behaviors because they will also cause changes in the user's electricity consumption pattern in the power center database.

[0003] Anomaly detection is particularly important in the field of smart grids, as it is directly related to their benefits and healthy operation. As a new type of power grid, smart grid integrates communication technology, multiple transmission and distribution systems, and complex sensors and monitoring equipment for power generation and distribution adjustment. The Advanced Metering Infrastructure (AMI) composed of multiple smart meters that can communicate provides an important guarantee for power management. AMI consists of three parts: data collection, communication transmission, and data reception management. At the data collection end, the smart meters installed by users replace traditional mechanical meters. Smart meters can record and monitor power consumption and related events (such as power outages). Communication transmission refers to the transmission media such as power line broadband, radio frequency networks, and public networks, which transmit the information collected by smart meters. Data reception management generally refers to the power center end and its integrated data storage, management, analysis and other functions. Through AMI, the data collected from smart meters are exchanged with the power center, which can serve energy management, such as fine electricity pricing, power generation and distribution. For a normal system, the frequency of abnormal samples is usually very low, and it is difficult to obtain abnormal data, and sometimes it is impossible to obtain abnormal data in a short period of time. This makes the detection accuracy of abnormal data low during monitoring. Summary of the invention

[0004] The main technical problem solved by the present invention is to provide a method for monitoring abnormal energy metering data, so as to solve the problem of detection accuracy of abnormal data.

[0005] In order to solve the above technical problems, a technical solution adopted by the present invention is to provide a method for monitoring abnormal energy metering data, comprising:

[0006] Construct a short-term monitoring model and monitor the data anomalies of the early stage of energy consumption of users (with less energy consumption data) according to the short-term monitoring model; and

[0007] Construct a long-term monitoring model. The long-term monitoring model monitors the abnormal data in the mid- and late-stage of users' energy consumption over a period of time based on the existing energy consumption data.

[0008] Furthermore, the short-term monitoring model includes an input layer, an output layer and a hidden layer. The layers are fully connected and have sufficient hidden layer nodes to fit any complex nonlinear mapping relationship. When the short-term monitoring model is used for multi-classification problems, there are multiple output layer nodes corresponding to multiple categories. When the short-term monitoring model is used for regression problems, the number of output layer nodes is 1, that is, only one monitoring value is output.

[0009] Furthermore, the monitoring process using the short-term monitoring model is as follows. Assume that there is D C samples, the input vector in each sample can be expressed as X = (x1, x2, ..., x n ), n represents the feature dimension of each sample; the output layer vector corresponding to the input vector is O = (o1, o2, ..., o l ), the expected output vector is d = (d1, d2, ..., d l ), l represents the number of neurons; the output vector of the hidden layer is Y = (y1, y2, ..., y m ), the weight vector between the input layer and the hidden layer can be expressed as V = (v1, v2, ..., v j , ..., v m ), m represents the number of hidden layers; the weight vector between the hidden layer and the output layer can be expressed as W = (w1, w2, ..., w k , ..., w l ); During the forward propagation process, the result of the kth neuron in the output layer can be expressed as:

[0010]

[0011] Where N k represents the output of sample N in the output layer neuron k; w jk represents the weight vector between the hidden layer neuron j and the output layer neuron;

[0012] The output of the jth neuron in the hidden layer can be expressed as:

[0013]

[0014] Where N j represents the output of sample N in the hidden layer neuron j; v ij represents the weight vector between input layer neuron i and hidden layer neuron j;

[0015] When the output of the short-term monitoring model is different from the expected output, the error between the two needs to be calculated; the error function usually uses the mean squared error (MSE), which can be expressed as:

[0016]

[0017] Among them, the variables in the error function are the weights w in each layer jk and v ij , the result of the loss function can be changed by adjusting the size of these two variables; if the updated E value cannot meet the predetermined accuracy, back propagation is performed to update the weights.

[0018] Furthermore, there are two data streams in the long-term monitoring model framework, namely, the historical normal data stream of each user and the new data stream that needs to be detected; in the historical normal data stream, two thresholds are generated using historical normal data, which are called the first threshold T1 and the second threshold T2, respectively, where T1 is generated by a Gaussian mixture model (GMM) and T2 is a preset value; in the new data stream, T1 and T2 judge the abnormal factors of the data that need to be detected, and binarize them to generate the first abnormal indicator BDI1 and the second abnormal indicator BDI2, respectively; BDI1 and BDI2 are sent to the "OR" mode module, and whether it is abnormal or not is judged according to its output.

[0019] Furthermore, the Gaussian mixture model is a probabilistic clustering model that combines a finite number of Gaussian linear combinations, mixing covariance structure information and potential Gaussian centers; as shown in the following formula, the form of GMM is

[0020]

[0021] Where x is a D-dimensional vector, π k is the mixing coefficient, each Gaussian component has its own mean center μ k and covariance ∑k, the mixing coefficient follows Constraints;

[0022] The optimal parameters determined by the optimal parameter determination algorithm are defined by the following formula:

[0023]

[0024] in represents the observed sample, is the log-likelihood estimate of the model, N is the number of sample data, and M is the model parameter θ MAP The number of parameters in .

[0025] Furthermore, for the model defined by the parameter Θ and the latent variable Z, the model parameters with latent variables can be estimated by the expectation maximization parameter algorithm, and the model parameters are estimated by iteratively executing the expectation step and the maximization likelihood estimation step;

[0026] Given an input vector X in a sample, n}, the log-likelihood estimate of the model with latent variables is given by,

[0027]

[0028] The maximization formula cannot be solved directly By introducing a new temporarily unknown distribution Q i (z i )get,

[0029]

[0030] The log(·) function is a concave function. According to Jensen’s inequality, a concave function has f(E(x))≥E(f(x)), so log∑ j α j x j ≥∑ j α j logx j , where α j ≥0,∑ j α j =1;

[0031] For GMM, the maximum log-likelihood estimate of its parameters is given by,

[0032]

[0033] Where Θ = {π, μ, ∑} is the GMM model parameter; the log-likelihood function The expanded representation is,

[0034]

[0035] In the first iteration of the expected execution step, the responsibility parameter γ(z nk ), the initial value is randomly generated, and the responsibility parameter is calculated using the randomly generated parameter value, where the responsibility parameter is defined as,

[0036]

[0037] The x in the sample vector n As observed data, the responsibility parameter is a posterior probability; then according to the current responsibility parameter, the parameters of the GMM are estimated separately by maximizing the likelihood estimation step

[0038]

[0039]

[0040]

[0041] Calculated After that, repeat the expected steps to calculate the updated responsibility parameter γ(z nk ), followed by the maximum likelihood estimation step, and the expectation step and the maximum likelihood estimation step are iteratively executed to iteratively update the parameters until convergence reaches the allowed tolerance.

[0042] Furthermore, the first abnormal factor DI1 and the second abnormal factor DI2 are used in both the historical data stream and the data stream to be checked.

[0043] Data sample x j The first outlier factor DIl of ∈X is defined as:

[0044]

[0045] where Θ = {π k , μ k ,∑ k} are the Gaussian mixture model parameters estimated by the EM algorithm; given Θ and data x j , the above formula is used to detect the possibility that the data belongs to the Gaussian mixture model;

[0046] Data sample x j The second abnormal factor DI2 of ∈X is defined as:

[0047]

[0048] Based on the understanding of power data anomalies, there are many ways to define anomaly factors; DI2 expressed in the form of L2 norm square can reflect the overall change in power consumption;

[0049] The first threshold T1 corresponding to DI1 is defined as:

[0050] T1=mean(DI1(X normal ))-α·std(DI1(X normal ))

[0051] The second threshold T2 corresponding to DI2 is defined as:

[0052]

[0053] In the above two formulas, mean(·) and std(·) calculate the mean and standard deviation respectively; αα and β are two important factors for adjusting the detection intensity, X normal represents a normal sample vector, T2 lower / upper Respectively represent the lower limit and upper limit of the second threshold.

[0054] Further, the first abnormal factor and the second abnormal factor are binarized according to T1 and T2 to obtain a first abnormal index BDI1 and a second abnormal index BDI2 respectively;

[0055]

[0056]

[0057] Among them, 1 represents potential abnormal data and 0 represents potential normal data; since each indicator will produce two results {0, 1}; therefore, DI1 and DI2 are combined into {(0, 0), (0, 1), (1, 0), (1, 1)}; through the binary OR operation, the output of BDIl and BDI2 is {0, 1, 1, 1}, that is, when any abnormal indicator is monitored as abnormal, the model gives the final monitoring of the abnormality.

[0058] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the method are implemented when the processor executes the computer program.

[0059] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described are implemented.

[0060] The beneficial effects of the present invention are as follows: In the present invention, based on the fact that there is less data in the early stage of the distribution system, a short-term monitoring model is used to monitor the data anomalies in the early stage of the distribution system, and effective monitoring can be performed even when there is less data. The long-term monitoring model combines the existing energy consumption data to monitor the data anomalies in the long-term operation of the distribution system, thereby improving the reliability of the system. Combining short-term monitoring and long-term monitoring, comprehensive monitoring of the operating status of the distribution system can be achieved. Short-term monitoring focuses on recent real-time data and can quickly detect instantaneous anomalies, while long-term monitoring, through historical data analysis, is more helpful in discovering potential problems in the long-term trend of the system. Through the combination of short-term and long-term monitoring, the monitoring of immediate and potential problems is taken into account, thereby improving the comprehensiveness, accuracy and reliability of system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a flow chart of a method for monitoring abnormal energy metering data according to an embodiment of the present invention;

[0062] Figure 2 is a schematic diagram of a short-term monitoring model according to an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of a specific framework of a long-term monitoring model according to an embodiment of the present invention;

[0064] Figure 4 is a flow chart of a training algorithm for a long-term monitoring model according to an embodiment of the present invention;

[0065] Figure 5 It is a flow chart of the detection algorithm of the long-term monitoring model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0067] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more related listed items.

[0068] Figure 1 An embodiment of the method for monitoring abnormal energy metering data of the present invention is shown, comprising:

[0069] Step S1: construct a short-term monitoring model, and monitor data anomalies in the early stage of user energy consumption when there is less energy consumption data according to the short-term monitoring model;

[0070] Step S2: construct a long-term monitoring model. The long-term monitoring model monitors the abnormal data of the user's energy consumption in the middle and late stages over a period of time based on the existing energy consumption data.

[0071] In the present invention, based on the fact that there is less data in the early stage of user energy consumption, a short-term monitoring model is used to monitor the data anomalies in the early stage of user energy consumption, and effective monitoring can be performed even when there is less data. The long-term monitoring model combines the existing energy consumption data to monitor the data anomalies in the long-term operation of user energy consumption, thereby improving the reliability of the system. Combining short-term monitoring and long-term monitoring, comprehensive monitoring of the user's energy consumption operation status can be achieved. Short-term monitoring focuses on recent real-time data and can quickly discover instantaneous anomalies, while long-term monitoring, through historical data analysis, is more helpful in discovering potential problems in the long-term trend of the system. Through the combination of short-term and long-term monitoring, both immediate and potential problems are monitored, thereby improving the comprehensiveness, accuracy and reliability of system operation.

[0072] The short-term monitoring model includes a three-layer structure with enough hidden layer nodes to fit any complex nonlinear mapping relationship. When the short-term monitoring model is used for multi-classification problems, there are multiple output layer nodes corresponding to multiple categories; when the short-term monitoring model is used for regression problems, the number of output layer nodes is 1, that is, only one monitoring value is output. Figure 2 As shown in the figure, the layers are fully connected. The number of neurons in the input layer corresponds to the parameter feature dimension of the input vector. The number of neurons in the hidden layer can be set arbitrarily, but if the number of neurons is too small, the generalization ability of the short-term monitoring model will be weakened, and if the number of neurons is too large, it will make it difficult for the short-term monitoring model to converge. Therefore, if the number of neurons in the input layer is n, the number of neurons in the output layer is l, the number of neurons in the hidden layer can be expressed as When monitoring using a short-term monitoring model, the state of each layer of neurons can only affect the neurons in the next layer during the forward propagation process. If the output layer cannot obtain the expected output value, the network will backpropagate its error signal. In this process, the gradient descent method is usually used to dynamically search for the weight vector value and minimize the loss function.

[0073] Figure 2 The monitoring process using the short-term monitoring model is shown. Assume that there is D c samples, the input vector in each sample can be expressed as X = (x1, x2, ..., x n ), n represents the feature dimension of each sample. The output layer vector corresponding to the input vector is O = (o1, o2, ..., o l ), the expected output vector is d = (d1, d2, ..., d1). The output vector of the hidden layer is Y = (y1, y2, ..., y m ), the weight vector between the input layer and the hidden layer can be expressed as V = (v1, v2, ..., v j , ..., v m), the weight vector between the hidden layer and the output layer can be expressed as W = (w1, w2, ..., w k , ..., w1). During the forward propagation process, the result of the kth neuron in the output layer can be expressed as:

[0074]

[0075] The output of the jth neuron in the hidden layer can be expressed as:

[0076]

[0077] In order to achieve better fitting results, the activation function f(·) in the above two equations uses a unipolar Sigmoid function, which has the property of continuous and differentiable, as shown below:

[0078]

[0079] f′(x)=f(x)[1-f(x)]

[0080] If the output of the short-term monitoring model is different from the expected output, the error between the two needs to be calculated. The error function usually uses the mean squared error (MSE), which can be expressed as:

[0081]

[0082] Among them, the variables in the error function are the weights w in each layer jk and v ij , the result of the loss function can be changed by adjusting the size of these two variables. If the updated E value cannot meet the predetermined accuracy, back propagation is performed to update the weights. In the back propagation process, the optimization goal is to minimize the error function E. Specifically, the weight w jk and v ij The adjustment in the negative gradient direction is proportional to the gradient descent of the error function E and can be expressed as follows:

[0083]

[0084]

[0085] Where η represents the learning rate, and its value range is (0, 1). According to the chain rule, the above formula can be rewritten as follows:

[0086]

[0087] If exists

[0088]

[0089] Then the formula Substitute Δw into the formula jk In this case, we can get:

[0090]

[0091] Similarly, for the weight v between the input layer and the hidden layer ij , using the chain rule to convert Δw jk Rewrite it as follows:

[0092]

[0093] If exists

[0094]

[0095] Among them, by taking partial derivatives of the output layer vector and the hidden layer vector respectively, we can get:

[0096]

[0097]

[0098] General and Substitution In this case, we can get:

[0099]

[0100] Finally, the formula Substitute Δv into the formula ij In the above equation, we can get v ij The calculation results are as follows:

[0101]

[0102] In the back propagation process, the formula Δw can be used jk and Δv ij The calculated weight values ​​are updated in the next round of learning as follows:

[0103] w jk (n+1)=w jk (n)+Δw jk

[0104] v ij (n+1)=v ij (n)+Δv ij

[0105] After the weight values ​​are updated using the above two formulas, the weight values ​​are applied to the forward propagation process to continue learning the short-term monitoring model. In the monitoring problem proposed by the present invention, the output of the short-term monitoring model is a specific monitoring value, so the dimension of the output layer vector is l = 1. Applying it to the above derivation process can obtain the weight update values ​​of each layer of the short-term monitoring model when performing short-term time series monitoring, thereby outputting the corresponding monitoring value.

[0106] For long-term monitoring, anomaly detection methods based on semi-supervised generative Gaussian mixture models are designed;

[0107] Specifically, the normal data x i ∈R n , i∈{1, 2, ..., m}, x i is an n-dimensional vector, which is the power consumption of a single user collected at a certain sampling rate, and m is the total number of users.

[0108] In the user's electricity consumption data, the user has a certain electricity consumption pattern every day. Corresponding to the daily electricity consumption pattern, the associated label information can be given to indicate whether the pattern is normal or abnormal. The present invention studies the daily electricity consumption patterns of users from multiple sources on a daily basis to detect abnormal behavior of electricity data. The detection frequency is once a day, and users with abnormal behavior are detected on a daily basis.

[0109] Based on the user's normal electricity consumption data, this paper proposes a long-term monitoring model. By modeling the user's normal electricity consumption pattern and introducing preset values, the abnormal electricity consumption pattern of the user is detected. The user's electricity consumption pattern is extracted and abnormal electricity consumption is detected on a daily basis. If abnormal behavior is detected, an alarm is issued. The proposed framework and model design details are described below.

[0110] For normal data, a long-term monitoring model is proposed, which utilizes the existing user energy consumption data to control the detection intensity.

[0111] The specific framework of the long-term monitoring model is as follows: Figure 3 As shown in the figure, the workflow for detecting outliers is described. In the long-term monitoring model framework, there are two data streams, namely the historical normal data stream of each user and the new data stream that needs to be detected. In the historical normal data stream, two thresholds are mainly generated using historical normal data, which are called the first threshold T1 and the second threshold T2, respectively, where T1 is generated by a Gaussian mixture model (GMM) and T2 is a preset value. In the new data stream, T1 and T2 judge the abnormal factors of the data to be detected, and binarize them to generate the first abnormal indicator BDI1 and the second abnormal indicator BDI2. BDI1 and BDI2 are sent to the "OR" mode module, and the abnormality is judged based on its output.

[0112] In this method, two abnormal indicators are generated for each user based on the power consumption. This method checks abnormal behavior with a detection frequency of once a day. In addition, the long-term monitoring model only needs normal data to detect whether the user's power consumption is abnormal in the future. Through the design of the framework, preset values ​​can be introduced to improve the detection performance and achieve the ability to control the detection intensity. The Gaussian mixture model, abnormal factors and abnormal indicators in the model framework are explained below.

[0113] The Gaussian mixture model is a probabilistic clustering model that combines a finite number of Gaussian linear combinations, mixing covariance structure information and potential Gaussian centers. As can be seen from the following formula, the form of GMM is:

[0114]

[0115] Where x is a D-dimensional vector, π k is the mixing coefficient, each Gaussian component has its own mean center μ k and covariance ∑k, the mixing coefficient follows constraints.

[0116] a) Gaussian mixture model free parameters

[0117] The number of Gaussian distributions k is a free parameter that can be selected based on practical experience or the optimal value can be determined using the optimal parameter determination algorithm. The optimal parameter determination algorithm is based on the model selection criterion of the likelihood function, which attempts to maximize the likelihood of the model while using the Occam factor to penalize its complexity. The optimal parameter determined by the optimal parameter determination algorithm is defined by the following formula,

[0118]

[0119] in represents the observed sample, is the log-likelihood estimate of the model, N is the number of sample data, and M is the model parameter θ MAP The number of parameters in .

[0120] As the amount of Gaussian mixture increases, the optimal parameter first decreases and reaches a minimum before increasing again. The optimal Gaussian mixture model is determined by the minimum optimal parameter. When the model is simple, the optimal parameter is dominated by the first half of the formula, that is, the quality of the model is mainly determined by the model's ability to describe the data. The better the model, the smaller the optimal parameter. As the complexity of the model increases, the optimal parameter model has a stronger fitting ability, but the generalization ability of models with too high complexity weakens. When the complexity of the model increases to the dominant optimal parameter standard, the value of the optimal parameter begins to increase. Therefore, the lower the optimal parameter, the better the model will perform in generalization.

[0121] b) Expectation-maximization parameter estimation

[0122] For the model defined by parameters Θ and latent variables Z, the model parameters with latent variables can be estimated by the expectation maximization parameter algorithm. Compared with the analytical solution, this is a numerical solution, which estimates the model parameters by iteratively executing the expectation step (hereinafter referred to as the E step) and the maximization likelihood estimation step (hereinafter referred to as the M step).

[0123] Given an input vector X in a sample, n}, the log-likelihood estimate of the model with latent variables is given by:

[0124]

[0125] The maximization formula cannot be solved directly By introducing a new temporarily unknown distribution Q i (z i )get,

[0126]

[0127] The log(·) function is a concave function. According to Jensen’s inequality, a concave function has f(E(x))≥E(f(x)), so log∑ j α j x j ≥∑ j α j logx j , where α j ≥0,∑ j α j =1. Using Jensen's inequality to scale the above equation, we can get:

[0128]

[0129] where ∑z i Q i (z i )=1, and when P(x i , z i ; Θ) / Q i (z i )=c, when c is a constant, the equality sign is satisfied. In this case, the log-likelihood estimate in the above formula has a lower bound. i , z i ; θ) = cQ i (z i ), and z on both sides of the equal sign i By summing, we can get

[0130]

[0131] Substitute the above formula into Q i(z i )=P(x i , z i ; Θ) / c, we can get:

[0132]

[0133] So the maximum log-likelihood estimate is It can be transformed into the following formula to maximize the lower bound of the log-likelihood estimate:

[0134]

[0135] Where Q i (z i )=P(z i |x i ; Θ), which is the posterior probability of the latent variable Z.

[0136] Iterate the expected step to calculate the log-likelihood estimate The maximization likelihood estimation step is to calculate Repeat the iterative execution of the expectation step and the maximization likelihood estimation step until Convergence. For GMM, the maximum log-likelihood estimate of its parameters is given by,

[0137]

[0138] Where Θ = {π, μ, ∑} is the GMM model parameter. Log-likelihood function The expanded representation is,

[0139]

[0140] In the first iteration of the expected execution step, the responsibility parameter γ(z nk ), the initial value is randomly generated, and the responsibility parameter is calculated using the randomly generated parameter value, where the responsibility parameter is defined as:

[0141]

[0142] The x in the sample vector n As observed data, the responsibility parameter is a posterior probability. Then, according to the current responsibility parameter, the parameters of GMM are estimated in M ​​steps respectively.

[0143]

[0144]

[0145]

[0146] Calculated After that, repeat step E to calculate the updated responsibility parameter γ(z nk ), followed by M steps, and the expectation step and the maximum likelihood estimation step are iteratively executed to iteratively update the parameters until convergence reaches the allowed tolerance.

[0147] Design of abnormal factors and abnormal indicators: Two abnormal factors DI1 and DI2 are used in both the historical data stream and the data stream to be checked. These two abnormal factors are explained below.

[0148] Data sample x j The first abnormal factor DI1 of ∈X is defined as:

[0149]

[0150] where Θ = {π k , μ k ,∑ k} is the Gaussian mixture model parameter estimated by the EM algorithm. In the proposed long-term monitoring model, the Gaussian mixture model plays an important role in modeling according to normal data. Given Θ and data x j , the above formula is used to detect the possibility that the data belongs to the Gaussian mixture model. The larger DI1 is, the greater the possibility that the data belongs to this Gaussian mixture model, and vice versa.

[0151] Data sample x j The second abnormal factor DI2 of ∈X is defined as:

[0152]

[0153] Based on the understanding of power data anomalies, there are many ways to define anomaly factors. Therefore, DI2 expressed in the form of L2 norm square can reflect the overall change in power consumption. Data sample x j A distance too close or too far from 0 may indicate a change in the power usage pattern. In addition, the number of abnormal factors may be increased. In order to merge the two abnormal factors, the present invention introduces the two abnormal factors into abnormality detection by combining binary operations through a binarization method.

[0154] The abnormal factor also directly affects the design of two thresholds in the historical data stream. The design and use of the first threshold T1 and the second threshold T2 are explained below.

[0155] The first threshold T1 corresponding to DI1 is defined as,

[0156] T1=mean(DI1(X normal ))-α(·std(DI1(X normal ))

[0157] The second threshold T2 corresponding to DI2 is defined as,

[0158]

[0159] In the above two equations, mean(·) and std(·) calculate the mean and standard deviation respectively. α and β are two important factors for adjusting the detection strength. As α increases, T1 decreases, which means that the data sample is more likely to be determined as belonging to the Gaussian mixture model. For β, as β increases, T2 upper -T2 lower The increase makes the data sample more likely to be determined as normal data from the perspective of abnormal factor DI2. T1 and T2 are the normal data X nomal Calculated without using outlier data.

[0160] In order to combine DI1 and DI2, the abnormal factors are firstly binarized according to T1 and T2 using the following formulas to obtain the first abnormality index BDI1 and the second abnormality index BDI2.

[0161]

[0162]

[0163] Among them, 1 represents potential abnormal data, and 0 represents potential normal data. Since each indicator will produce two results {0, 1}. Therefore, DI1 and DI2 are combined into {(0, 0), (0, 1), (1, 0), (1, 1)}. Through the binary OR operation, the output of BDI1 and BDI2 is {0, 1, 1, 1}, that is, when any abnormal indicator is monitored as abnormal, the model gives the final monitoring of the abnormality.

[0164] After explaining the specific definitions of two anomaly factors (DI1, DI2), two thresholds (T1, T2) and two anomaly indicators (BDI1, BDI2), their application in the framework is analyzed below.

[0165] The above explains that the semi-supervised generative Gaussian mixture model framework has two data streams: historical normal data stream and new data stream that needs to be detected. The two data streams correspond to two sets of algorithms: training algorithms such as Figure 4 As shown, and the detection algorithm is as Figure 5As shown. The first training algorithm implements model training in historical normal data streams, while the second detection algorithm is used to check whether there is abnormal data in new data streams. In the historical data stream training algorithm, the input is historical sampled normal data, and the output is two thresholds T1 and T2 that can detect abnormal samples. First, through steps 2 to 11, the GMM model is established using normal data. In steps 12 to 15, two abnormal factors DI1 and DI2 are calculated based on normal data, and then thresholds T1 and T2 are calculated based on these two factors. In the detection algorithm of the data stream to be tested, the input is the newly collected data, and the output is the judgment result of the newly collected data, normal or abnormal. In steps 2 to 3, the two abnormal factors DI1 and DI2 are also calculated first. Unlike in the training algorithm, these two abnormal factors do not calculate the thresholds T1 and T2, but judge DI1 and DI2 based on the previously obtained T1 and T2 to obtain the abnormal indicators BDI1 and BDI2, as shown in steps 4 to 13 of the algorithm. Finally, in steps 14 to 19, the abnormal indicators are used to obtain the final result through OR operation.

[0166] This model considers the case of obtaining only normal data, and creates a generative model as a benchmark through the Gaussian mixture model, which effectively utilizes normal data. Because the detection method is based on a semi-supervised mode, it avoids the data imbalance problem that occurs in anomaly detection based on machine learning. The proposed framework based on the Gaussian mixture model introduces more anomaly factors DI. Here, two anomaly factors are introduced in the framework, and the introduced anomaly factors can effectively control the detection strength of the model.

[0167] The above are only embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for monitoring abnormal energy metering data, characterized in that: include: Construct a short-term monitoring model and monitor the abnormal data of users' energy consumption in the early stage according to the short-term monitoring model; as well as Construct a long-term monitoring model. The long-term monitoring model monitors the abnormal data in the mid- and late-stage of users' energy consumption over a period of time based on the existing energy consumption data.

2. The method for monitoring abnormal energy metering data according to claim 1, characterized in that: The short-term monitoring model includes an input layer, an output layer and a hidden layer. The layers are fully connected and have enough hidden layer nodes to fit any complex nonlinear mapping relationship. When the short-term monitoring model is used for multi-classification problems, there are multiple output layer nodes corresponding to multiple categories. When the short-term monitoring model is used for regression problems, the number of nodes in the output layer is 1, and one monitoring value is output.

3. The method for monitoring abnormal energy metering data according to claim 2, characterized in that: The monitoring process using the short-term monitoring model is as follows: Assume that there is D C samples, and the input vector in each sample is represented by X=(x1,x2,…,x n ), n represents the feature dimension of each sample; the output layer vector corresponding to the input vector is O = (o1, o2, ..., o l ), the expected output vector is d=(d1,d2,…,d l ), l represents the number of neurons; the output vector of the hidden layer is Y = (y1, y2, ..., y m ), the weight vector between the input layer and the hidden layer is expressed as V = v1, v2, ..., v j ,…,v m ), m represents the number of hidden layers; the weight vector between the hidden layer and the output layer is expressed as W = (w1, w2, ..., w k ,…,w l ); During the forward propagation process, the result of the k-th neuron in the output layer is expressed as: Where N k represents the output of sample N in the output layer neuron k; w jk represents the weight vector between the hidden layer neuron j and the output layer neuron; The output of the jth neuron in the hidden layer is expressed as: Where N j represents the output of sample N in the hidden layer neuron j; v ij represents the weight vector between input layer neuron i and hidden layer neuron j; When the output of the short-term monitoring model is different from the expected output, the error between the two needs to be calculated; the error function uses the mean square error, which is expressed as: Among them, the variables in the error function are the weights w in each layer jk and v ij , the result of the loss function is changed by adjusting the size of these two variables; if the updated E value cannot meet the predetermined accuracy, back propagation is performed to update the weights.

4. The method for monitoring abnormal energy metering data according to claim 3, characterized in that: There are two data streams in the long-term monitoring model framework, namely, the historical normal data stream of each user and the new data stream that needs to be detected; in the historical normal data stream, two thresholds are generated using historical normal data, respectively called the first threshold T1 and the second threshold T2, wherein the first threshold T1 is generated by a Gaussian mixture model, and the second threshold T2 is a preset value; in the new data stream, the first threshold T1 and the second threshold T2 judge the abnormal factors of the data that needs to be detected, and binarize them to generate the first abnormal indicator BDI1 and the second abnormal indicator BDI2; judge whether the first abnormal indicator BDI1 and the second abnormal indicator BDI2 are abnormal or not.

5. The method for monitoring abnormal energy metering data according to claim 4, characterized in that: The Gaussian mixture model is a probabilistic clustering model that combines a limited number of Gaussian linear combinations, mixed covariance structure information and potential Gaussian centers; as shown in the following formula, the form of the Gaussian mixture model is: Where x is a D-dimensional vector, π k is the mixing coefficient, each Gaussian component has its own mean center μ k and covariance Σ k , the mixing coefficient follows 0≤π k ≤1 constraint; The optimal parameters determined by the optimal parameter determination algorithm are defined by the following formula: in, represents the observed sample, is the log-likelihood estimate of the model, N is the number of sample data, and M is the model parameter θ MAP The number of parameters in .

6. The method for monitoring abnormal energy metering data according to claim 5, characterized in that: For the model defined by the parameter Θ and the latent variable Z, the model parameters with latent variables are estimated by the expectation maximization parameter algorithm, and the model parameters are estimated by iteratively executing the expectation step and the maximum likelihood estimation step; Given an input vector X in a sample, n }, the log-likelihood estimate of the model with latent variables is given by: The maximization formula cannot be solved directly By introducing a new temporarily unknown distribution Q i (z i )get, The log(·) function is a concave function. According to Jensen’s inequality, a concave function has f E(x))≥E(f(x)), so log∑ j α j x j ≥∑ j α j logx j , where α j ≥0,∑ j α j =1; For the Gaussian mixture model, the maximum log-likelihood estimate of its parameters is given by: Where Θ = {π, μ, Σ} is the Gaussian mixture model parameter; the log-likelihood function The expanded representation is: In the first iteration of the expected execution step, the responsibility parameter γ(z nk ), the initial value is randomly generated, and the responsibility parameter is calculated using the randomly generated parameter value, where the responsibility parameter is defined as: The x in the sample vector n As observed data, the responsibility parameter is a posterior probability; then according to the current responsibility parameter, the parameters of the Gaussian mixture model are estimated separately by maximizing the likelihood estimation step Calculated After that, repeat the expected steps to calculate the updated responsibility parameter γ(z nk ), followed by the maximum likelihood estimation step, and the expectation step and the maximum likelihood estimation step are iteratively executed to iteratively update the parameters until convergence reaches the allowed tolerance.

7. The method for monitoring abnormal energy metering data according to claim 6, characterized in that: The first abnormal factor DI1 and the second abnormal factor DI2 are used in both the historical data stream and the data stream to be checked. Data sample x j The first abnormal factor DI1 of ∈X is defined as: where Θ = {π k ,μ k ,Σ k } are the Gaussian mixture model parameters estimated by the EM algorithm; given Θ and data x j , the above formula is used to detect the possibility that the data belongs to the Gaussian mixture model; Data sample x j The second abnormal factor DI2 of ∈X is defined as: DI2, expressed in the form of L2 norm squared, reflects the overall change in electricity consumption; The first threshold T1 corresponding to DI1 is defined as: T1=mean DI1(X normal ))-α·std(DI1(X normal )) The second threshold T2 corresponding to DI2 is defined as: In the above two formulas, mean(·) and std(·) calculate the mean and standard deviation respectively; α and β are two important factors for adjusting the detection intensity, T2 lower / upper Respectively represent the lower and upper limits of the second threshold, X normal represents a normal sample vector.

8. The method for monitoring abnormal energy metering data according to claim 7, characterized in that: Binarize the two abnormal factors according to T1 and T2 to obtain the first abnormal index BDI1 and the second abnormal index BDI2; Among them, 1 represents potential abnormal data and 0 represents potential normal data. Since each indicator will produce two results {0,1}, DI1 and DI2 are combined into {(0,0), (0,1), (1,0), (1,1)}. Through the binary OR operation, the output of BDI1 and BDI 2 is {0,1,1,1}, that is, when any abnormal indicator is monitored as abnormal, the model gives the final monitoring of the abnormality.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images