Power grid voltage data anomaly detection method, device and equipment based on VAE-Bi-LSTM-SAM and storage medium
By using the VAE-Bi-LSTM-SAM model in the detection of grid voltage data abnormality, combining the self-attention mechanism and bidirectional long and short-term memory network, the problems of high computational complexity and poor feature capture in the existing technology are solved, and the problem of More accurate and robust abnormality detection effects.
Patent Information
- Application Number
- CN202410967431.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing grid voltage timing data abnormality detection methods have high computational complexity when processing high-dimensional data. The traditional VAE-LSTM model performs poorly when capturing complex features and local correlations, making it difficult to accurately identify outliers in grid voltage data.
Using the VAE-Bi-LSTM-SAM method, the feature extraction and reconstruction capabilities of the model are enhanced through self-attention mechanism (SAM), combined with the bidirectional long and short-term memory network (Bi-LSTM), the front and back dependencies of time series data are captured, and Use the Logcosh loss function to improve the robustness of the model.
It improves the accuracy and robustness of abnormal detection of grid voltage data, can better capture the correlation and complex timing relationships between multivariables, and enhances the model's ability to identify abnormal data and reconstruction performance.
Smart Images

Figure CN118897973A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power systems, and in particular to a method, device, equipment and storage medium for detecting abnormality in power grid voltage data based on VAE-Bi-LSTM-SAM. Background Art
[0002] With the rapid development of my country's economy and the continuous advancement of smart grid construction, the amount of data generated during the operation of the power grid has exploded. However, during the operation of the power system, due to various reasons such as equipment failure, load fluctuation, system oscillation, etc., the power grid operation data deviates from the expected measurement value, generating abnormal data. These abnormal data may mislead the monitoring, dispatching and management of the power grid, and even lead to power grid accidents. In order to ensure the safe and stable operation of the power grid, it is of great significance to timely discover and process abnormal data. However, in a power grid that generates a large amount of high-dimensional data, it is difficult to manually label the data because the data itself is noisy and it is impossible to clearly define normal and abnormal. In order to solve this problem, unsupervised learning methods have become an important solution for abnormal data detection in power grids.
[0003] In recent years, unsupervised learning methods based on deep learning have been widely used and studied in the field of abnormal data detection. The methods can be mainly divided into two categories: clustering-based methods and reconstruction-based methods. In power grid systems, real-time monitoring and analysis of large amounts of multidimensional time series data (such as voltage, phase angle, current, power, temperature, etc.) are essential to ensure stable operation of the power grid. However, clustering-based anomaly detection methods face the challenge of high computational complexity when processing such high-dimensional data. These methods usually require calculating the distance between data points to determine the cluster center, which becomes very time-consuming in high-dimensional data sets.
[0004] The reconstruction-based method is to first train the model, then reconstruct the data through the trained model to obtain the reconstruction error, and detect abnormal data by comparing whether the error between the reconstructed value and the actual value exceeds the threshold. Since there are often some noise data in the grid voltage time series data, the variational autoencoder VAE can tolerate the existence of some abnormal data during training, and the long short-term memory network LSTM can capture the long-term dependency of the time series. Therefore, the combination of variational autoencoder VAE and long short-term memory network LSTM is often used for anomaly detection of time series data. However, the time series of grid voltage data usually has complex characteristics and interrelationships. The traditional VAE-LSTM model may have limitations in capturing these complex characteristics, resulting in limited recognition and modeling capabilities for abnormal grid data. In particular, the performance of traditional models is not ideal when dealing with local correlations or capturing key time points such as grid emergencies. These limitations indicate that the existing models still need to be further improved to more accurately detect anomalies in grid voltage time series data.
[0005] Chinese patent document CN 116011329 A discloses a method, device and system for detecting anomalies in the topology of a low-voltage distribution network based on VAE-LSTM. By collecting historical normal and abnormal low-voltage time series data of users in the same substation area and performing normalization processing, a VAE-LSTM model is constructed, wherein VAE is responsible for extracting local features of voltage time series data in a short window, and LSTM is responsible for extracting long-term dependencies, and then model training and optimization are performed to finally achieve anomaly detection. Chinese patent document CN 117763144 A discloses a method and terminal for log anomaly detection. The method first obtains original log data from each data node of a smart grid, and performs classification analysis and feature extraction, converts unstructured log data into structured data, and extracts key features. Then, a LSTM prediction model is trained using a small amount of labeled log data, and the model is used to perform probability prediction on a large amount of unlabeled log data, thereby generating a large amount of labeled data. Finally, these labeled data are mixed with the labeled data to train an LSTM model that introduces a self-attention mechanism for final log anomaly detection. In the above two abnormal data detection methods, the LSTM model is used. However, due to the unidirectional structure of the LSTM model, it is impossible to obtain the backward time information in the time series data, and neither of them pays attention to the local correlation of multiple variables in the data. Summary of the invention
[0006] In order to solve the deficiencies in the prior art, the present invention proposes a method, device, equipment and storage medium for detecting anomaly in power grid voltage data based on VAE-Bi-LSTM-SAM.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] In a first aspect of the present invention, a method for detecting anomaly in power grid voltage data based on VAE-Bi-LSTM-SAM is disclosed. The method comprises the following steps:
[0009] S1. Collect grid voltage time series data to build a data set, preprocess the data in the data set, and divide the preprocessed data set into a training set and a test set.
[0010] S2. Standardize the grid voltage time series data in the training set and the test set, and perform window sliding slicing on the standardized grid voltage time series data according to the time step to form a multivariate time series window data X t .
[0011] S3. Establish an abnormal data detection model based on VAE-Bi-LSTM-SAM, multivariate time series window data X t As input to the anomaly detection model.
[0012] S4. Use the training set to train the VAE-Bi-LSTM-SAM-based anomaly data detection model to obtain a trained anomaly data detection model.
[0013] S5. Use the trained abnormal data detection model to perform anomaly detection on the grid voltage time series data in the test set.
[0014] Preferably, in step S1, the collecting of grid voltage time series data to construct a data set, preprocessing the data in the data set, and dividing the preprocessed data set into a training set and a test set include:
[0015] S11. Collect grid voltage time series data of the bus to be detected in the power grid within a set time period, and construct a data set using the collected grid voltage time series data; the grid voltage time series data includes bus voltage historical time series data and phase angle historical time series data, and the grid voltage time series data is collected once every hour.
[0016] The grid voltage time series data x r for:
[0017] x r ={x1,x2,...,x t ,...,x m},x t ∈R k (1);
[0018] In formula (1), x r represents the grid voltage time series data in the data set; m represents the total number of moments of the grid voltage time series data in the data set; each element xt It is a two-tuple, representing the voltage and phase angle values at time t; R represents the real number domain; k represents the characteristic dimension of the input data.
[0019] S12. Preprocess the grid voltage time series data in the data set, remove duplicate data, and complete missing data.
[0020] S13. Divide the preprocessed data set into a training set and a test set in proportion.
[0021] Preferably, in step S2, the grid voltage time series data in the training set and the test set are standardized using formula (2):
[0022]
[0023] In formula (2), Standardization is the standardized data; x i is a data point in the grid voltage time series data; μ is the average value of the grid voltage time series data in the training set; σ is the standard deviation of the grid voltage time series data in the training set.
[0024] According to a preferred embodiment of the present invention, in step S2, the multivariate time series window data X t The expression is as follows:
[0025] X t =[x t-T ,x t-T+1 ,...,x j ,...,x t-1 ]∈R k×T (3);
[0026] In formula (3), each element x j is a two-tuple, representing the voltage and phase angle values at time j; R represents the real number domain; k represents the characteristic dimension of the input data; T represents the length of the window data; X t It is the multivariate time series window data, including the multivariate power grid time series data from time tT to time t-1.
[0027] Preferably, according to the present invention, in step S3, the abnormal data detection model based on VAE-Bi-LSTM-SAM includes: an input layer, a first self-attention mechanism layer, a VAE encoder, a latent variable layer, a second self-attention mechanism layer and a VAE decoder; the VAE encoder and the VAE decoder are both composed of two layers of Bi-LSTM.
[0028] Multivariate time series window data X tAs the input of the abnormal data detection model, it is transmitted to the first self-attention mechanism layer through the input layer; the first self-attention mechanism layer processes the multivariate time series window data X t Extract key features to obtain the attention matrix A, and use the attention matrix A to analyze the grid voltage time series data X t Weighted, the enhanced grid voltage time series data x is obtained A ; Enhanced grid voltage time series data x A Input VAE encoder, VAE encoder output mean μ(x A ) and variance σ(x A ); the latent variable layer is based on the mean μ(x A ) and variance σ(x A ), generating a latent variable z; the second self-attention mechanism layer weights the latent variable z to obtain a weighted latent variable z'; the VAE decoder maps the latent variable z' back to the original data space to generate reconstructed data x'.
[0029] The first self-attention mechanism layer uses formula (4) to extract the attention matrix A, and uses formula (5) to input the multivariate time series window data X t Weighted, the enhanced grid voltage time series data x is obtained A :
[0030] A=SAM(X t ) (4);
[0031] In formula (4), SAM represents the input multivariate time series window data X t The operation of applying the self-attention mechanism; A represents the attention matrix, and the elements in the matrix A are the weights.
[0032] x A =A·X t (5);
[0033] In formula (5), x A Represents the enhanced grid voltage time series data, which is the multivariate time series window data X t After weighting, A represents the attention matrix, · represents matrix multiplication, X t Represents multivariate time series window data.
[0034] Preferably, in step S4, the abnormal data detection model based on VAE-Bi-LSTM-SAM is trained using the training set to obtain the trained abnormal data detection model, including:
[0035] The grid voltage time series data in the training set is used as the input of the abnormal data detection model. The VAE-Bi-LSTM-SAM-based abnormal data detection model is trained using the minimum reconstruction error algorithm to obtain the trained VAE-Bi-LSTM-SAM-based abnormal data detection model.
[0036] The objective function of the abnormal data detection model based on VAE-Bi-LSTM-SAM is shown in formula (11):
[0037]
[0038] In formula (11), L(φ,θ;x) is the next term of Lb, φ and θ represent the parameters of the corresponding encoder and decoder networks respectively; x represents the data, KL represents the divergence term, which is used to penalize the difference between the latent variable distribution output by the encoder network and the prior distribution; q φ and P θ They represent the encoder and decoder networks of the abnormal data detection model based on VAE-Bi-LSTM-SAM respectively; q φ (z|x) is a posterior probability of deriving z from data x, P θ (z) is the prior probability of z; E qφ(z|x) is the loss function for reconstructing the data.
[0039] Preferably, in step S5, using the trained abnormal data detection model to perform abnormality detection on the grid voltage time series data in the test set includes:
[0040] S51. Input the data in the test set into the trained VAE-Bi-LSTM-SAM-based abnormal data detection model. The abnormal data detection model uses the Logcosh loss function shown in formula (12) to construct the reconstruction error:
[0041]
[0042] In formula (12), E(y, f(x)) represents the error function, which is used to measure the reconstruction error between the original data y and the reconstructed data f(x); log cosh() represents the logarithm of the hyperbolic cosine of the error value; y is the real data; f(x) is the reconstructed data, which is the predicted value of the model.
[0043] S52. Compare the reconstruction error with the optimal threshold, and determine whether the data is abnormal based on the comparison result: if the reconstruction error is greater than the optimal threshold, it is determined to be abnormal data and the abnormality detection result is output; if the reconstruction error is not greater than the optimal threshold, it is determined to be normal data.
[0044] In step S52, the optimal threshold is determined by the silhouette coefficient: the silhouette coefficient of each sample point is determined using formula (13), and the optimal threshold is determined according to the distribution of the silhouette coefficient:
[0045]
[0046] In formula (13), S(i) represents the silhouette coefficient; b(i) represents the average distance from sample i to all samples in the nearest cluster; a(i) represents the average distance from sample i to other samples in its cluster; max{a(i), b(i)} represents the maximum value between a(i) and b(i); the range of the silhouette coefficient is [-1, 1], and the closer the value is to 1, the better the sample clustering effect.
[0047] In the second aspect of the present invention, a VAE-Bi-LSTM-SAM-based power grid voltage data anomaly detection device is disclosed. The device includes: a data acquisition and preprocessing module, a standardization processing module, an abnormal data detection model building module, a model training module and an abnormal data detection module.
[0048] The data acquisition and preprocessing module is used to acquire grid voltage time series data to construct a data set, preprocess the data in the data set, and divide the preprocessed data set into a training set and a test set.
[0049] The standardization processing module is used to perform standardization processing on the grid voltage time series data in the training set and the test set, and perform window sliding slicing processing on the standardized grid voltage time series data according to the time step to form a multivariate time series window data X t .
[0050] The abnormal data detection model building module is used to establish an abnormal data detection model based on VAE-Bi-LSTM-SAM. t As input to the anomaly detection model.
[0051] The model training module is used to train the VAE-Bi-LSTM-SAM-based abnormal data detection model using the training set to obtain a trained abnormal data detection model.
[0052] The abnormal data detection module is used to perform abnormality detection on the power grid voltage time series data in the test set using the trained abnormal data detection model.
[0053] In a third aspect of the present invention, an electronic device is disclosed, comprising: at least one processor; and a memory, wherein the memory stores instructions, and when the instructions are executed by the at least one processor, the at least one processor executes the VAE-Bi-LSTM-SAM-based power grid voltage data anomaly detection method.
[0054] In a fourth aspect of the present invention, a machine-readable storage medium is disclosed, which stores executable instructions, and when the instructions are executed, the machine executes the VAE-Bi-LSTM-SAM-based power grid voltage data anomaly detection method.
[0055] Compared with the prior art, the advantages of the present invention are:
[0056] (1) The present invention is based on an unsupervised deep learning method, using a hybrid model combining a variational autoencoder (VAE) and a bidirectional long short-term memory network (Bi-LSTM), and adding a self-attention mechanism (SAM) to the hybrid model to achieve accurate detection of abnormal grid voltage data. Replacing the feedforward neural network in the VAE with a Bi-LSTM can consider the front-to-back dependencies of time series data, improve the generalization ability of the model, and enable it to better adapt to different data distributions and characteristics. Adding the SAM self-attention mechanism to the hybrid model can dynamically adjust the degree of attention to different variables according to the data content, so that the model can better capture the correlation and interaction between variables and enhance the model's ability to identify abnormal data. In addition, the present invention adds the Logcosh loss function to the reconstruction loss instead of the default selection of the L2 loss function to reduce the impact of abnormal data on model training and improve the robustness of the model.
[0057] (2) The present invention replaces the original feedforward neural network with two Bi-LSTM layers in the encoder and decoder parts of the VAE, wherein the Bi-LSTM is equivalent to being composed of two LSTMs, which can obtain timing information forward and backward, thereby better capturing the complex timing relationships in multivariate time series data. After the input data and before entering the encoder, adding SAM can dynamically learn the dependencies between multiple variables, and can also better capture long-distance dependencies and important information in the time series by dynamically adding attention weights. SAM is also added after the intermediate variables and before the decoder. By reconstructing the original input sequence, the model can be prompted to learn the key features and potential structures in the data. In short, adding the SAM attention mechanism to the VAE-Bi-LSTM model can improve the model's ability to detect abnormal data in multivariate time series, while enhancing the model's performance in data representation and reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 It is a method flow chart of the power grid voltage data anomaly detection method based on VAE-Bi-LSTM-SAM in the present invention;
[0059] Figure 2 Schematic diagram of an abnormal data detection model based on VAE-Bi-LSTM-SAM in the present invention;
[0060] Figure 3 It is a curve chart of the anomaly detection results of multivariate time series data of a certain bus. DETAILED DESCRIPTION
[0061] The present disclosure is further described below with reference to the accompanying drawings and embodiments:
[0062] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.
[0063] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0064] In the absence of conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.
[0065] Example 1
[0066] like Figure 1 The power grid voltage data anomaly detection method based on VAE-Bi-LSTM-SAM shown in the figure comprises the following steps:
[0067] S1. Collect grid voltage time series data to construct a data set, preprocess the data in the data set, and divide the preprocessed data set into a training set and a test set.
[0068] In the step S1, the collecting of grid voltage time series data to construct a data set, preprocessing the data in the data set, and dividing the preprocessed data set into a training set and a test set include:
[0069] S11, collecting grid voltage time series data of the bus to be detected in the grid within a set time period, and constructing a data set using the collected grid voltage time series data. The grid voltage time series data includes bus voltage historical time series data and phase angle historical time series data, and the grid voltage time series data is measured once every hour.
[0070] S12. Preprocess the grid voltage time series data in the data set, and divide the preprocessed data into a training set and a test set according to a preset ratio. The data in the training set is normal data, and a small amount of abnormal data is allowed to exist. The presence of a small amount of abnormal data will not affect the accuracy of the model. The test set data includes normal data and abnormal data. The training set is used to train the model, that is, the normal data is used to train the model, so that the normal data can be reconstructed during the test, while the abnormal data cannot be reconstructed, resulting in a high error, so that the abnormal data can be detected.
[0071] The original collected grid voltage time series data X is preprocessed. First, the existing methods such as duplicate data removal and missing data completion are used to preprocess the grid voltage time series data X. The preprocessed data set is divided into a training set and a test set in a ratio of 8:2 to achieve data division. Among them, the grid voltage time series data x in the data set r for:
[0072] x r ={x1,x2,...,x t ,...,x m},x t ∈R k (1);
[0073] In formula (1), x r represents the grid voltage time series data in the data set; m represents the total number of moments of the grid voltage time series data in the data set; each element x t It is a two-tuple, representing the voltage and phase angle values at time t; R represents the real number domain; k represents the characteristic dimension of the input data.
[0074] S2. Standardize the grid voltage time series data in the training set and the test set respectively, and perform window sliding slicing on the standardized training set and the test set according to the time step to form a multivariate time series window data X. t .
[0075] Formula (2) is used to standardize the grid voltage time series data in the training set and the test set:
[0076]
[0077] In formula (2), Standardization is the standardized data; x i is a data point in the grid voltage time series data; μ is the average value of the grid voltage time series data in the training set; σ is the standard deviation of the grid voltage time series data in the training set.
[0078] When using the abnormal data detection model to predict the grid voltage time series data in the test set, it is necessary to keep the model processing methods consistent on the training set and the test set, otherwise the model will perform poorly on the test set. Therefore, the mean and standard deviation of the training set are used to standardize the data of the test set to ensure that the performance of the model on the test set is consistent with that on the training set. This ensures that the model processes new data in a consistent manner and improves the generalization ability of the model.
[0079] Multivariate time series window data X t As the input of the anomaly detection model, the multivariate time series window data X t The expression is as follows:
[0080] X t =[x t-T ,x t-T+1 ,...,x j ,...,x t-1 ]∈R k×T (3);
[0081] In formula (3), each element x j is a two-tuple, representing the voltage and phase angle values at time j; R represents the real number domain; k represents the characteristic dimension of the input data; T represents the length of the window data; X t It is the multivariate time series window data, including the multivariate power grid time series data from time tT to time t-1.
[0082] S3. Establish an abnormal data detection model based on VAE-Bi-LSTM-SAM.
[0083] The grid voltage time series data after preprocessing and standardization in the data set is used as the input of the abnormal data detection model, and the abnormal data detection model is used to perform anomaly detection on the grid voltage time series data.
[0084] like Figure 2 As shown, the abnormal data detection model includes: an input layer, a first self-attention mechanism layer, a VAE encoder, a latent variable layer, a second self-attention mechanism layer and a VAE decoder.
[0085] exist Figure 1 In the example, the SAM between the input and the encoder is the first self-attention mechanism layer, and the SAM between the sampled latent variables and the decoder is the second self-attention mechanism layer.
[0086] (1) Input layer
[0087] The preprocessed grid voltage time series data in the dataset is transmitted to the first self-attention mechanism layer through the input layer.
[0088] (2) First self-attention mechanism layer
[0089] Take n continuous multivariate time series window data X t As the input of the abnormal data detection model, the input data is input into the first self-attention mechanism layer SAM layer, the first self-attention mechanism layer ( Figure 2 The SAM on the left side of the figure is used to extract key features, obtain the attention matrix, and use the attention matrix to weight the grid voltage time series data to obtain the enhanced grid voltage time series data. The first self-attention mechanism layer is used to strengthen the relationship between multiple variables and weight the grid voltage data to extract key features.
[0090] The first self-attention mechanism layer dynamically assigns attention weights by calculating the correlation between different parts of the input data, thereby weighting the key features in the input data. In this process, the model will simultaneously identify important features in the data and weight different features according to the importance of these features, thereby effectively focusing on key information. The final output of the first self-attention mechanism layer is the weighted data.
[0091] Specifically, after the grid voltage time series data is input into the first self-attention mechanism layer, the first self-attention mechanism layer extracts the attention matrix A using formula (4):
[0092] A=SAM(X t ) (4);
[0093] In formula (4), SAM represents the input multivariate time series window data X t The operation of applying the self-attention mechanism; A represents the attention matrix, and the elements in the matrix A are the weights.
[0094] Based on the attention matrix A, the input multivariate time series window data X is calculated using formula (5) t Weighted, the enhanced grid voltage time series data x is obtained A :
[0095] x A =A·X t (5);
[0096] In formula (5), x A Represents the enhanced grid voltage time series data, which is the multivariate time series window data X t After weighting, A represents the attention matrix, · represents matrix multiplication, Xt Represents multivariate time series window data.
[0097] After the first self-attention mechanism layer, x A It can contain more information about the relationship between input data and can be used as input to the VAE encoder.
[0098] (3) VAE Encoder
[0099] Multivariate time series window data X extracted using VAE model t The local features of the grid voltage time series data x are enhanced by the VAE encoder. A Mapped to latent variable z. The principle of the VAE model is to first use the VAE encoder to reduce the data dimension to the latent space, and then use the VAE decoder to reconstruct the original data. The VAE encoder can help map the input data to the latent variable space, thereby extracting the local features of the data in the window. These local features can represent some key information in the window data.
[0100] The VAE encoder learns two distributions: the prior distribution P of the latent variable z θ (z) and the enhanced grid voltage time series data x for a given latent variable z A The posterior distribution q φ (z|x A ).
[0101] In this embodiment, given the enhanced grid voltage time series data x of the latent variable z A The posterior distribution q φ (z|x A ) can be any distribution, here we assume it is a normal distribution, the expression is shown in formula (6):
[0102] q φ (z|x A )~N(μ(x A ),σ(x A ))(6);
[0103] In formula (6), q φ is the encoder network, φ represents the encoder parameters, N(μ(x A ),σ(x A )) is a normal distribution, μ(x A ) and σ(x A ) is the encoder network q φ The mean and variance of the output.
[0104] Prior distribution P of latent variable z θ(z) is a preset distribution, usually assumed to be a standard normal distribution. It is not learned by the encoder, but serves as a fixed reference point to guide the learning of the posterior distribution. The input data of the VAE encoder, i.e., the enhanced grid voltage time series data x A Generate the posterior distribution q through the VAE encoder φ After (z|x), the latent variable layer samples the latent variable z from the posterior distribution.
[0105] Since the posterior distribution is usually assumed to be a Gaussian distribution, the sampling process involves drawing samples from the Gaussian distribution of the posterior distribution. This is usually achieved through the reparameterization technique to maintain the differentiability of the sampling process. The reparameterization technique transforms the sampling process into sampling from a standard Gaussian distribution. The objective function of the VAE model contains a KL divergence term, which is used to measure the difference between the variational posterior distribution and the prior distribution. Minimizing the KL divergence term means making the posterior distribution q φ (z|x) is as close as possible to the prior distribution P θ (z), which helps maintain the diversity of latent representations while ensuring that the data generated by the model is similar to real data.
[0106] (4) Latent variable layer
[0107] First, in order to keep the posterior distribution of the latent variable z consistent with the prior distribution P θ (z) is similar, using the difference between the posterior distribution and the prior distribution using the KL divergence, where the KL divergence is: KL(q φ (z|x A )||P θ (z)), the latent variable z and the prior distribution P can be reduced by minimizing the KL divergence θ (z) to ensure that the learned latent variable z can be closer to the prior distribution.
[0108] Secondly, we introduce a reparameterization technique to generate a noise vector ε, which is a random vector that follows a standard normal distribution. The dimension of the noise vector ε is the same as the latent variable z, and the output μ(x A ) and σ(x A ) is input into the latent variable layer, and the latent variable layer generates the latent variable z using formula (7):
[0109] z=μ(x A )+σ(x A )×ε (7);
[0110] In formula (7), z represents the latent variable, μ(x A ) and σ(x A ) is the encoder network q φThe mean and variance of the output; ε is the noise vector, which is a random vector that follows a standard normal distribution. The encoder network is a VAE encoder, and the mean and variance of the encoder network output are used to parameterize the posterior distribution of the latent variable z.
[0111] (5) Second self-attention mechanism layer
[0112] The VAE encoder inputs the enhanced grid voltage timing data x A After mapping to the latent variable z, a second self-attention mechanism layer is added after the latent variable layer ( Figure 2 The second self-attention mechanism layer weights the extracted latent variable z to obtain the weighted latent variable z'. The weighting method used by the second self-attention mechanism layer is the same as that of the first self-attention mechanism layer.
[0113] (6) VAE Decoder
[0114] According to the conditional probability distribution P θ (x A |z') decoding to obtain the reconstructed grid voltage time series data, where the conditional probability distribution P θ (x A |z') is learned by the decoder network through training, P θ is the decoder network, and θ represents the parameters of the decoder. The latent variable z' is mapped back to the original data space through the VAE decoder to generate the reconstructed data x'.
[0115] During the training of the VAE decoder, the parameters θ of the decoder network are optimized by minimizing the reconstruction error and KL divergence so that the reconstructed data x' is as close to the original data x as possible. A .
[0116] Further, such as Figure 2 As shown in the figure, the encoder and decoder in the VAE model are composed of two layers of Bi-LSTM. The Bi-LSTM layer is called a bidirectional LSTM. Each Bi-LSTM layer is composed of two LSTMs, one LSTM that processes information forward and one LSTM that processes information backward. Each Bi-LSTM layer takes the output of the previous layer as input and outputs it to the next layer. The output of each Bi-LSTM layer is used to calculate the mean and variance. Using the Bi-LSTM layer can better capture the bidirectional dependencies in time series data.
[0117] LSTM consists of three gates: forget gate, input gate, and output gate. In addition to the three gates, LSTM also has a memory cell to remember the information of the previous moment. At time t, LSTM has three inputs, including the input value x of the network at the current moment. t , the output h at the previous momentt-1 and the cell state at the previous moment. The first step in LSTM is to use the forget gate f t The forget gate f decides what information in the cell state to discard. t The calculation method is:
[0118] f t =σ(W f ·[h t-1 ,x t ]+b f ) (8);
[0119] In formula (8), f t represents the forget gate, σ(·) represents the sigmoid function, W t is the weight matrix of the forget gate, h t-1 is the hidden information of the previous moment, x t is the information input at the current moment, b f is the forget gate bias, and [·,·] represents the connection between vectors.
[0120] The second step in LSTM is the input gate, which determines what new information is added to the cell state. The calculation process is as follows:
[0121] i t =σ(W i ·[h t-1 ,x t ]+b i )
[0122]
[0123] In formula (9), i t represents the input gate, σ(·) represents the sigmoid function, W i is the weight matrix of the input gate, h t-1 is the hidden information of the previous moment, x t is the information input at the current moment, b i is the input gate bias; It is obtained by further updating the information in the input gate using the tanh function, where W C is the memory gate weight matrix, b C is the memory gate bias, tanh(·) is the hyperbolic tangent function, and the symbol · indicates that the corresponding components of the vector are multiplied to obtain a new vector; f t represents the forget gate, in C t Where C t-1 is the cell state at the previous moment, C t is the cell state at the current moment.
[0124] The third step in LSTM is the output gate, which determines what information to output. This output is based on the C t , the calculation process is as follows:
[0125] o t =σ(W o [h t-1 ,x t ]+b o )
[0126] h t =o t *tanh(C t ) (10);
[0127] In formula (10), o t represents the output gate, σ(·) represents the sigmoid function, W o is the weight matrix of the output gate, h t-1 is the hidden information of the previous moment, x t is the information input at the current moment, b o is the output gate bias, h t is the hidden information at this moment, tanh(·) is the hyperbolic tangent function, and the symbol · indicates that the corresponding components of the vector are multiplied to obtain a new vector; C t is the cell state at the current moment.
[0128] S4. Use the training set to train the abnormal data detection model to obtain a trained abnormal data detection model.
[0129] In step S4, the grid voltage time series data in the training set is used as the input of the abnormal data detection model, and the abnormal data detection model is trained using the objective function of minimizing the reconstruction error and KL divergence until the abnormal data detection model parameters reach the target value, and the training of the abnormal data detection model based on VAE-Bi-LSTM-SAM is completed. The parameters of the abnormal data detection model include weights and bias terms.
[0130] The objective function of the anomaly data detection model based on VAE-Bi-LSTM-SAM is to minimize the evidence lower term ELBO, called Lb lower term, expressed as:
[0131]
[0132] In formula (11), L(φ,θ;x) is the next term of Lb, φ and θ represent the parameters of the corresponding encoder and decoder networks respectively; x represents the data, KL represents the divergence term, which is used to penalize the difference between the latent variable distribution output by the encoder network and the prior distribution; q φ and P θThey represent the encoder and decoder networks of the abnormal data detection model based on VAE-Bi-LSTM-SAM respectively; q φ (z|x) is a posterior probability of deriving z from data x, P θ (z) is the prior probability of z; E qφ(z|x) is the loss function for reconstructing the data.
[0133] S5. Input the data in the test set into the trained abnormal data detection model based on VAE-Bi-LSTM-SAM, and use the trained abnormal data detection model to perform abnormality detection on the power grid voltage time series data in the test set.
[0134] In step S5, the data in the test set is input into the trained abnormal data detection model based on VAE-Bi-LSTM-SAM, and the abnormal data detection model is used to perform abnormality detection on the grid voltage time series data in the test set, including:
[0135] S51. Input the data in the test set into the trained VAE-Bi-LSTM-SAM-based abnormal data detection model. The abnormal data detection model uses the Logcosh loss function shown in formula (12) to construct the reconstruction error:
[0136]
[0137] In formula (12), E(y, f(x)) represents the error function, which is used to measure the reconstruction error between the original data y and the reconstructed data f(x); log cosh() represents the logarithm of the hyperbolic cosine of the error value; y is the real data; f(x) is the reconstructed data, which is the predicted value of the model.
[0138] S52. Compare the reconstruction error with the optimal threshold, and determine whether the data is abnormal based on the comparison result: if the reconstruction error is greater than the optimal threshold, it is determined to be abnormal data and the abnormality detection result is output; if the reconstruction error is not greater than the optimal threshold, it is determined to be normal data.
[0139] For smaller errors, its performance is similar to the mean square error (MSE), which helps the smoothness and generalization ability of the training process; for larger errors, its performance is similar to the mean absolute error (MAE), which can alleviate the impact of outliers on model training. The Logcosh loss function is smoother than the L2 norm when reconstructing errors, which can effectively reduce the impact of outliers on model training. The Logcosh loss function is more suitable for processing data with noise or outliers while maintaining a certain degree of smoothness and robustness.
[0140] The reconstruction error is used as the anomaly score, and an optimal threshold is determined. The time step with a reconstruction error greater than the threshold is considered an anomaly. Figure 3 The reconstruction error distribution of the test set is shown in , where the marked parts are the anomalies. Figure 3 In the figure, the horizontal axis represents the sample index, which indicates the sequential number of the input data; the vertical axis represents the error / reconstruction value, which represents the reconstruction error of each sample, that is, the degree of difference between each sample after the VAE model reconstructs it and the original sample.
[0141] In step S52, the optimal threshold is determined by the silhouette coefficient: the silhouette coefficient of each sample point is determined by formula (13), and the optimal threshold is determined according to the distribution of the silhouette coefficient. According to the distribution of the silhouette coefficient, samples with significantly low silhouette coefficients are identified, and these samples are judged to be outliers, and the optimal threshold is finally determined.
[0142] The silhouette coefficient is introduced when determining the optimal threshold. It is known that the abnormal points in the data are outliers. Therefore, when using the silhouette coefficient for anomaly detection, if the silhouette coefficient of some samples is much lower than that of other samples, it means that these samples may be outliers or abnormal points, because they are very different from the samples in their clusters and have low similarity with other clusters. Therefore, an optimal threshold can be determined based on the distribution of the silhouette coefficient to determine which samples may be abnormal points. The silhouette coefficient formula is:
[0143]
[0144] In formula (13), S(i) represents the silhouette coefficient; b(i) represents the average distance from sample i to all samples in the nearest cluster; a(i) represents the average distance from sample i to other samples in its cluster; max{a(i), b(i)} represents the maximum value between a(i) and b(i). The range of the silhouette coefficient is [-1, 1]. The closer the value is to 1, the better the sample clustering effect is.
[0145] In summary, in the present invention, two layers of Bi-LSTM layers are used in the encoder and decoder parts of VAE to replace the original feedforward neural network, which can realize the simultaneous consideration of the forward and backward information of the time series data, and can more effectively capture the long-term dependencies in the sequence data. In order to strengthen the relationship between multiple data, the present invention also adds SAM before the encoder and the decoder. Adding SAM after the input data and before the encoder allows the encoder to strengthen the correlation between multiple variables when processing the input multivariate time series data, which helps to capture global dependencies. Adding SAM after the latent variables and before the decoder allows the decoder to focus on the previously generated part when generating data, and also helps to improve the model's understanding of the global dependencies of the reconstructed data, thereby improving the quality of data reconstruction.
[0146] Example 2
[0147] This embodiment provides a power grid voltage data anomaly detection device based on VAE-Bi-LSTM-SAM, which includes: a data acquisition and preprocessing module, a standardization processing module, an abnormal data detection model construction module, a model training module and an abnormal data detection module.
[0148] The data acquisition and preprocessing module is used to acquire grid voltage time series data to construct a data set, preprocess the data in the data set, and divide the preprocessed data set into a training set and a test set.
[0149] The standardization processing module is used to perform standardization processing on the grid voltage time series data in the training set and the test set, and perform window sliding slicing processing on the standardized grid voltage time series data according to the time step to form a multivariate time series window data X t .
[0150] The abnormal data detection model building module is used to establish an abnormal data detection model based on VAE-Bi-LSTM-SAM. t As input to the anomaly detection model.
[0151] The model training module is used to train the VAE-Bi-LSTM-SAM-based abnormal data detection model using the training set to obtain a trained abnormal data detection model.
[0152] The abnormal data detection module is used to perform abnormality detection on the power grid voltage time series data in the test set using the trained abnormal data detection model.
[0153] Example 3
[0154] This embodiment also provides an electronic device, including: at least one processor; and a memory, wherein the memory stores instructions, and when the instructions are executed by the at least one processor, the at least one processor executes the power grid voltage data anomaly detection method based on VAE-Bi-LSTM-SAM as described above.
[0155] In this embodiment, the electronic device may include, but is not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronic devices, and the like.
[0156] Example 4
[0157] This embodiment also provides a machine-readable storage medium storing executable instructions, which, when executed, enable the machine to execute the above-mentioned VAE-Bi-LSTM-SAM-based power grid voltage data anomaly detection method.
[0158] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.
[0159] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute part of this specification.
[0160] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or a cloud via a communication network.
[0161] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0162] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0163] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0165] The above-described embodiments are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the scope of protection determined by the claims of the present invention.
Claims
1. A method for detecting abnormality of power grid voltage data based on VAE-Bi-LSTM-SAM, characterized in that: The method comprises the following steps: S1, collecting grid voltage time series data to build a data set, preprocessing the data in the data set, and dividing the preprocessed data set into a training set and a test set; S2. Standardize the grid voltage time series data in the training set and the test set, and perform window sliding slicing on the standardized grid voltage time series data according to the time step to form a multivariate time series window data X t ; S3. Establish an abnormal data detection model based on VAE-Bi-LSTM-SAM, multivariate time series window data X t As input to anomaly detection models; S4. Use the training set to train the VAE-Bi-LSTM-SAM-based abnormal data detection model to obtain a trained abnormal data detection model; S5. Use the trained abnormal data detection model to perform anomaly detection on the grid voltage time series data in the test set.
2. The method for detecting anomaly of power grid voltage data based on VAE-Bi-LSTM-SAM according to claim 1 is characterized in that: In the step S1, the collecting of grid voltage time series data to construct a data set, preprocessing the data in the data set, and dividing the preprocessed data set into a training set and a test set include: S11, collecting grid voltage time series data of a bus to be detected in a grid within a set time period, and constructing a data set using the collected grid voltage time series data; the grid voltage time series data includes bus voltage historical time series data and phase angle historical time series data, and the grid voltage time series data is collected once every hour; The grid voltage time series data x r for: x r {x1,x2,...,x t ,…,x m },x t ∈R k (1)? In formula (1), x r represents the grid voltage time series data in the data set; m represents the total number of moments of the grid voltage time series data in the data set; each element x t is a two-tuple, representing the voltage and phase angle values at time t; R represents the real number domain; k represents the characteristic dimension of the input data; S12, preprocessing the grid voltage time series data in the data set, removing duplicate data, and filling in missing data; S13. Divide the preprocessed data set into a training set and a test set in proportion.
3. The method for detecting abnormality of power grid voltage data based on VAE-Bi-LSTM-SAM according to claim 1, characterized in that: In step S2, the grid voltage time series data in the training set and the test set are standardized using formula (2): In formula (2), Standardization is the standardized data; x i is a data point in the grid voltage time series data; μ is the average value of the grid voltage time series data in the training set; σ is the standard deviation of the grid voltage time series data in the training set.
4. The method for detecting anomaly of power grid voltage data based on VAE-Bi-LSTM-SAM according to claim 1, characterized in that: In step S2, the multivariate time series window data X t The expression is as follows: X t =[x t-T ,x t-T+1 ,...,x j ,...,x t-1 ]∈R k×T (3); In formula (3), each element x j is a two-tuple, representing the voltage and phase angle values at time j; R represents the real number domain; k represents the characteristic dimension of the input data; T represents the length of the window data; X t It is the multivariate time series window data, including the multivariate power grid time series data from time tT to time t-1.
5. The method for detecting abnormality of power grid voltage data based on VAE-Bi-LSTM-SAM according to claim 1, characterized in that: In step S3, the abnormal data detection model based on VAE-Bi-LSTM-SAM includes: an input layer, a first self-attention mechanism layer, a VAE encoder, a latent variable layer, a second self-attention mechanism layer and a VAE decoder; the VAE encoder and the VAE decoder are both composed of two layers of Bi-LSTM; Multivariate time series window data X t As the input of the abnormal data detection model, it is transmitted to the first self-attention mechanism layer through the input layer; the first self-attention mechanism layer processes the multivariate time series window data X t Extract key features to obtain the attention matrix A, and use the attention matrix A to analyze the grid voltage time series data X t Weighted, the enhanced grid voltage time series data x is obtained A ; Enhanced grid voltage time series data x A Input VAE encoder, VAE encoder output mean μ(x A ) and variance σ(x A ); the latent variable layer is based on the mean μ(x A ) and variance σ(x A ), generating a latent variable z; the second self-attention mechanism layer weights the latent variable z to obtain a weighted latent variable z'; the VAE decoder maps the latent variable z' back to the original data space to generate reconstructed data x'; The first self-attention mechanism layer uses formula (4) to extract the attention matrix A, and uses formula (5) to input the multivariate time series window data X t Weighted, the enhanced grid voltage time series data x is obtained A : A=SAM(X t ) (4); In formula (4), SAM represents the input multivariate time series window data X t The operation of applying the self-attention mechanism; A represents the attention matrix, and the elements in the matrix A are weights; x A =A·X t (5); In formula (5), x A Represents the enhanced grid voltage time series data, which is the multivariate time series window data X t After weighting, A represents the attention matrix, · represents matrix multiplication, X t Represents multivariate time series window data.
6. The method for detecting abnormality of power grid voltage data based on VAE-Bi-LSTM-SAM according to claim 1, characterized in that: In step S4, the abnormal data detection model based on VAE-Bi-LSTM-SAM is trained using the training set to obtain a trained abnormal data detection model, including: The grid voltage time series data in the training set is used as the input of the abnormal data detection model, and the VAE-Bi-LSTM-SAM-based abnormal data detection model is trained using the minimum reconstruction error algorithm to obtain a trained VAE-Bi-LSTM-SAM-based abnormal data detection model. The objective function of the abnormal data detection model based on VAE-Bi-LSTM-SAM is shown in formula (11): In formula (11), L(φ,θ;x) is the next term of Lb, φ and θ represent the parameters of the corresponding encoder and decoder networks respectively; x represents the data, KL represents the divergence term, which is used to penalize the difference between the latent variable distribution output by the encoder network and the prior distribution; q φ and P θ They represent the encoder and decoder networks of the abnormal data detection model based on VAE-Bi-LSTM-SAM respectively; q φ (z|x) is a posterior probability of deriving z from data x, P θ (z) is the prior probability of z; is the loss function for reconstructing the data.
7. The method for detecting abnormality of power grid voltage data based on VAE-Bi-LSTM-SAM according to claim 1, characterized in that: In step S5, the trained abnormal data detection model is used to perform abnormality detection on the grid voltage time series data in the test set, including: S51. Input the data in the test set into the trained VAE-Bi-LSTM-SAM-based abnormal data detection model. The abnormal data detection model uses the Logcosh loss function shown in formula (12) to construct the reconstruction error: In formula (12), E(y, f(x)) represents the error function, which is used to measure the reconstruction error between the original data y and the reconstructed data f(x); log cosh() represents the logarithm of the hyperbolic cosine of the error value; y is the real data; f(x) is the reconstructed data, which is the predicted value of the model; S52, comparing the reconstruction error with the optimal threshold, and determining whether the data is abnormal according to the comparison result: if the reconstruction error is greater than the optimal threshold, it is determined to be abnormal data, and the abnormality detection result is output; if the reconstruction error is not greater than the optimal threshold, it is determined to be normal data; In step S52, the optimal threshold is determined by the silhouette coefficient: the silhouette coefficient of each sample point is determined using formula (13), and the optimal threshold is determined according to the distribution of the silhouette coefficient: In formula (13), S(i) represents the silhouette coefficient; b(i) represents the average distance from sample i to all samples in the nearest cluster; a(i) represents the average distance from sample i to other samples in its cluster; max{a(i), b(i)} represents the maximum value between a(i) and b(i); the range of the silhouette coefficient is [-1, 1], and the closer the value is to 1, the better the sample clustering effect.
8. A power grid voltage data anomaly detection device based on VAE-Bi-LSTM-SAM, characterized in that: The device comprises: a data acquisition and preprocessing module, a standardization processing module, an abnormal data detection model building module, a model training module and an abnormal data detection module; The data acquisition and preprocessing module is used to collect grid voltage time series data to construct a data set, preprocess the data in the data set, and divide the preprocessed data set into a training set and a test set; The standardization processing module is used to perform standardization processing on the grid voltage time series data in the training set and the test set, and perform window sliding slicing processing on the standardized grid voltage time series data according to the time step to form a multivariate time series window data X t ; The abnormal data detection model building module is used to establish an abnormal data detection model based on VAE-Bi-LSTM-SAM. t As input to anomaly detection models; The model training module is used to train the VAE-Bi-LSTM-SAM-based abnormal data detection model using the training set to obtain a trained abnormal data detection model; The abnormal data detection module is used to perform abnormality detection on the power grid voltage time series data in the test set using the trained abnormal data detection model.
9. An electronic device, characterized in that: include: at least one processor; and a memory storing instructions, which, when executed by the at least one processor, enable the at least one processor to execute the VAE-Bi-LSTM-SAM-based power grid voltage data anomaly detection method as described in any one of claims 1 to 7.
10. A machine-readable storage medium, characterized in that: It stores executable instructions, which, when executed, enable the machine to perform the VAE-Bi-LSTM-SAM-based power grid voltage data anomaly detection method as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Optical module fault prediction method based on variational auto-encoder and long and short term memory network hybrid model
CN115833937A
Low-voltage distribution network topological structure anomaly detection method, device and system based on VAE-LSTM
CN116011329A
Medium and low pressure gas pressure regulator anomaly detection method based on LSTM-AE-DT model
CN116108885A
Electric vehicle charging pile occupation prediction method based on cloud computing double-flow feature interaction
CN116933931A
Log anomaly detection method and terminal
CN117763144A
Cited By
Energy storage EMS abnormal user behavior detection method based on sparse attention mechanism
CN119202766A
Magnetic anomaly detection method based on variational auto-encoder and bidirectional gating circulation unit hybrid model
CN119577652A
Water pipeline operation state anomaly detection method combining reconstruction and prediction model
CN119807970A
Internal threat detection method and system based on user behavior representation technology
CN120068090A
Internet of Things system and device based on fifth generation mobile communication technology, and medium
CN120342910A
