Electrical load abnormal fluctuation detection method based on Transform architecture
Through the Transformer architecture, the abnormal fluctuation detection method of power load is solved, and the problems of large-scale, high-dimensional power load data processing are large-scale, high-dimensional power load data processing are large-scale, high-dimensional, and poor real-time performance are achieved, and efficient abnormal detection is achieved and time capture is captured. The global correlation and long-sequence processing capabilities in the sequence improve the accuracy of abnormal detection of power load data.
Patent Information
- Application Number
- CN202411091157.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-27
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The traditional power load data abnormality detection method is large in processing large-scale and high-dimensional data, has low accuracy and poor real-time performance, which is easy to misjudgment the noise as abnormal, and lacks the context information behind the changes in power load, resulting in The abnormality of meaning is ignored.
Using the Transformer architecture to detect abnormal fluctuations of electricity loads based on the Transformer architecture, the electricity load data is obtained through preprocessing and sequence encoding, and a multi-layer Transformer encoder model is constructed, and the global correlation in the time series is captured using the self-attention mechanism, and the entire process is completed. The connection layer is mapped to the exception score for judgment.
It improves the accuracy and efficiency of abnormal detection of power load data, and can learn information representation from complex time dynamics in unsupervised learning, maintain high performance and accuracy, especially when processing long sequences, and exhibit high detection capabilities.
Smart Images
Figure CN119202961A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of power load detection, and in particular to a method for detecting abnormal fluctuations in power load based on a Transformer architecture. Background Art
[0002] With the rapid development of smart grid technology and the increasing complexity of power systems, anomaly detection of power load data has become a key link to ensure the stable operation of the power grid and improve energy efficiency. Traditional methods for detecting anomalies in power load data include statistical threshold comparison and rule-based expert systems. The statistical threshold comparison method is mainly based on historical data, and uses statistical methods (such as the 3σ principle, quartile method, etc.) to set a reasonable threshold range. For example, according to the mean and standard deviation of historical data, the normal fluctuation range of power load is set to (μ-3σ, μ+3σ), where μ is the mean of historical data and σ is the standard deviation. Although traditional anomaly detection methods can meet basic detection needs to a certain extent, they often face problems such as large computational complexity, low accuracy, and poor real-time performance when processing large-scale, high-dimensional power load data.
[0003] Traditional load anomaly detection methods assume that power load data obeys a certain probability distribution, but the actual situation may not fully conform to this assumption, which may lead to misjudgment of outliers; traditional load anomaly detection methods also have the following problems: they are sensitive to noise and can easily misjudge noise as anomalies; the selection of appropriate thresholds or other parameters has a great impact on algorithm performance and requires experience or experiments to determine, which increases the complexity and uncertainty of the algorithm; judging anomalies based on numerical comparisons lacks consideration of the contextual information behind power load changes, which may lead to meaningful anomalies being ignored. Summary of the invention
[0004] The purpose of the present invention is to overcome the defects of the prior art methods for detecting abnormal loads in power grids, which easily lead to misjudgment of abnormal values or neglect of some abnormal values, resulting in low accuracy, poor real-time performance, and affecting the stable operation of the power grid, and to provide a method for detecting abnormal fluctuations in power load based on the Transformer architecture.
[0005] The purpose of the present invention is achieved through the following technical solutions: A method for detecting abnormal fluctuations in power load based on a Transformer architecture comprises the following steps: Step 1: Obtain power load data and preprocess the power load data; Step 2: sequence encoding the preprocessed power load data to convert the power load data into a vector representation; Step 3: construct an abnormal transformation model based on the Transformer architecture, including constructing a multi-layer Transformer encoder and mapping the output of the multi-layer Transformer encoder to an abnormality score; Step 4: Based on the calculated anomaly score, determine whether the load data at each time point is abnormal.
[0006] Preferably, in step 1, the power load data is time series data, and the power load data includes load values at multiple time points, which is expressed as: L = {l 1 ,l 2 ,...l T}, Among them l T Represents the power load at the tth time point.
[0007] Preferably, in step 1, the preprocessing of the power load data is specifically performed as follows: Clean the power load data to remove missing values and obviously erroneous outliers in the power load data. Use interpolation, mean filling or median filling methods to fill missing values. Use statistical methods to identify and process obviously erroneous outliers. The power load data after data cleaning is standardized and converted into a standard scale to eliminate the dimensional differences between different load levels. The standardized announcement is as follows: Where μ is the mean of the load data over a period of time, and σ is the standard deviation of the load data over a period of time; The power load data is also subjected to feature extraction, and the load change rate at adjacent time points, the average value and standard deviation within the sliding window are calculated.
[0008] Preferably, the step 2 is specifically as follows: The embedding layer is used to convert the power load data at each time point into a vector representation of a fixed size. Assuming the output dimension of the embedding layer is d, the embedded sequence can be expressed as: E={e 1 ,e 2 ,...e T}, where e T is the vector representation of the power load data at the Tth time point after being transformed by the embedding layer, e T ∈R d , R represents the set of real numbers, and d is the dimension of the embedding vector.
[0009] Preferably, in step 3, constructing a multi-layer Transformer encoder is specifically as follows: The multi-layer Transformer encoder includes multiple stacked self-attention layers, which include an attention mechanism, a position encoding, and a feedforward neural network. The self-attention mechanism is: by calculating the matrix of query, key, and value, and applying the softmax function to obtain the attention weight, the weighted fusion of information at different positions in the sequence is achieved; Q=W q E K=W k E V=W v E Where Q represents query, K represents key, V represents value, and W represents q , W k , W v is the weight matrix learned by the model; where d k is the dimension of the key; The position coding method includes absolute position coding and relative position coding; The feedforward neural network includes two linear transformations and an activation function.
[0010] Preferably, in the step 3, mapping the output of the multi-layer Transformer encoder to the anomaly score is specifically as follows: mapping the output of the Transformer encoder to the anomaly score using a fully connected layer, the fully connected layer receiving the representation of the last time point output by the encoder, and outputting one or more anomaly scores; FFN(Z)=ReLU(W 1 Z+b 1 )W 2 +b 2 Where FFN represents a feed-forward neural network, Z is the output of the self-attention mechanism, and W 1 , W 2 is the weight learned by the model, b 1 , b 2 It is the bias of model learning and ReLU is the activation function.
[0011] Preferably, the abnormal fluctuation detection method of power load based on Transformer architecture also optimizes the model parameters and constructs a loss function, which includes a mean square error or a mean absolute error loss function; the model is trained by a gradient descent optimization algorithm to optimize the model parameters with the goal of minimizing the loss function.
[0012] Preferably, the step 4 is specifically as follows: Set an anomaly threshold and compare the anomaly score with the preset threshold. If the anomaly score exceeds the threshold, the time point is marked as an anomaly, otherwise it is marked as normal.
[0013] Preferably, the abnormal threshold is determined by cross-validation, empirical rules or a method based on statistical distribution.
[0014] The beneficial effects of the present invention are: (1) Through the self-attention mechanism, the present invention can capture the global correlation between elements in the time series, which is an important factor in discovering abnormal patterns.
[0015] (2) The present invention can learn information representation from complex temporal dynamics in unsupervised learning tasks and derive a distinguishable criterion to detect anomalies.
[0016] (3) Due to the characteristics of the Transformer architecture, the present invention has advantages in processing long sequences and can maintain high performance and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION
[0018] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.
[0019] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.
[0020] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions. A method for detecting abnormal fluctuations in power load based on Transformer architecture, such as Figure 1 As shown, the following steps are included: Step 1: Obtain power load data and preprocess the power load data; Step 2: sequence encoding the preprocessed power load data to convert the power load data into a vector representation; Step 3: construct an abnormal transformation model based on the Transformer architecture, including constructing a multi-layer Transformer encoder and mapping the output of the multi-layer Transformer encoder to an abnormality score; Step 4: Based on the calculated anomaly score, determine whether the load data at each time point is abnormal.
[0021] In this embodiment, the anomaly transformation model based on the Transformer architecture is the Anomaly Transformer deep learning model. The Anomaly Transformer model is a deep learning architecture specifically designed for anomaly detection. It combines the sequence modeling capability of the Transformer with the anomaly detection algorithm to accurately identify abnormal patterns in time series data such as power load data.
[0022] The Anomaly Transformer model usually consists of two main parts: an encoder and a decoder. The encoder is responsible for receiving the input time series data and capturing the dependencies and patterns in the data through the self-attention mechanism. The decoder generates a reconstructed time series data based on the output of the encoder. During the training process, the Anomaly Transformer model optimizes its parameters by minimizing the difference between the reconstructed data and the original data. In this way, the model is able to learn the characteristics and patterns of normal data. In the testing phase, when the model receives new time series data, it generates a reconstructed data sequence and compares it with the original data. If the difference between the two exceeds a certain threshold, the data is considered to contain anomalies. By introducing the self-attention mechanism and position encoding technology, the Anomaly Transformer model can capture the temporal dependencies and spatial correlations in the power load data, thereby more accurately extracting the abnormal features in the load data. Through training on the training sample set and testing on the test sample set, this method can more accurately extract the abnormal features in the load data and improve the accuracy and efficiency of anomaly detection of power load data.
[0023] In step 1, the power load data is time series data, and the power load data includes load values at multiple time points, which is expressed as: L = {l 1 ,l 2 ,...l T}, Among them l T Represents the power load at the tth time point.
[0024] The original power load data may have missing values, outliers, noise and other problems, which will affect the accuracy of anomaly detection. Data preprocessing aims to improve data quality and provide a reliable data basis for subsequent analysis. Therefore, in step 1, the power load data is preprocessed as follows: Clean the power load data to remove missing values and obviously erroneous outliers in the power load data. Use interpolation, mean filling or median filling methods to fill missing values. Use statistical methods to identify and process obviously erroneous outliers. The power load data after data cleaning is standardized and converted into a standard scale to eliminate the dimensional differences between different load levels. The standardized announcement is as follows: Where μ is the mean of the load data over a period of time, and σ is the standard deviation of the load data over a period of time; The power load data is also extracted to calculate the load change rate at adjacent time points, the average value and standard deviation within the sliding window. The purpose of feature extraction is to carry out feature engineering and enrich the input information of the model.
[0025] The step 2 is specifically as follows: Using an embedding layer to convert the power load data at each time point into a fixed-size vector representation helps capture patterns and trends in the data and provide effective input for anomaly detection. Assuming the output dimension of the embedding layer is d, the embedded sequence can be expressed as: E={e 1 ,e 2 ,...e T}, where e T is the vector representation of the power load data at the Tth time point after being transformed by the embedding layer, e T ∈R d , R represents the set of real numbers, and d is the dimension of the embedding vector.
[0026] In step 3, constructing a multi-layer Transformer encoder is specifically as follows: The multi-layer Transformer encoder includes multiple stacked self-attention layers, which include an attention mechanism, a position encoding, and a feedforward neural network. The self-attention mechanism is: by calculating the matrix of query, key, and value, and applying the softmax function to obtain the attention weight, the weighted fusion of information at different positions in the sequence is achieved; Q=W q E K=W k E V=W v E Where Q represents query, K represents key, V represents value, and W represents q , W k , W v is the weight matrix learned by the model; where d k is the dimension of the key; The position encoding method includes absolute position encoding and relative position encoding; since the Transformer model itself does not have position information, position encoding needs to be added to distinguish data at different positions in the sequence.
[0027] The feedforward neural network includes two linear transformations and an activation function.
[0028] The Transformer model can capture the global correlation between different time points in the input sequence through the self-attention mechanism, which is crucial for anomaly detection. The multi-layer Transformer encoder can further extract the deep features of the input sequence by stacking multiple self-attention layers and feedforward neural network layers.
[0029] In step 3, the output of the multi-layer Transformer encoder is mapped to the anomaly score as follows: Map the output of the Transformer encoder to anomaly scores using a fully connected layer, which receives the representation of the last time point output by the encoder and outputs one or more anomaly scores; FFN(Z)=ReLU(W 1 Z+b 1 )W 2 +b 2 Where FFN represents a feed-forward neural network, Z is the output of the self-attention mechanism, and W 1 , W 2 , b 1 , b 2 are the weights and biases learned by the model, and ReLU is the activation function.
[0030] The abnormal fluctuation detection method of power load based on Transformer architecture also optimizes the model parameters and constructs the loss function, which includes the mean square error or mean absolute error loss function; the model is trained through the gradient descent optimization algorithm to optimize the model parameters with the goal of minimizing the loss function.
[0031] The step 4 is specifically as follows: An abnormality threshold is set, and the abnormality score is compared with the preset threshold. If the abnormality score exceeds the threshold, the time point is marked as abnormal, otherwise it is marked as normal. The abnormality threshold is determined by cross-validation, empirical rules, or methods based on statistical distribution.
[0032] Assume that the final output of the Transformer encoder is O∈R T×d , maps O to anomaly score vector S∈R through a fully connected layer T ,Right now: S=W s Mean(O)+b s Where W s , b s are the final learned weights and biases of the model, and Mean(O) is the average of the last dimension of O.
[0033] The specific model parameters are as follows: In order to verify the performance of the algorithm in detecting abnormal power load of high-voltage non-residential users, the actual power load data of 10 high-voltage non-residential users in Zhejiang Province in 2024 were used. These data sets have been manually labeled to ensure their authenticity and accuracy. When collecting data, a sampling interval of 15 minutes was set, so 96 data points could be collected every day. In order to comprehensively evaluate the performance of the algorithm, three indicators, accuracy, recall rate and F1-score, were selected. Accuracy (precision rate) measures the proportion of samples that are truly abnormal among all samples predicted to be abnormal, and the calculation formula is: Among them, TP (True Positives) represents true positives, that is, the number of samples that are actually abnormal and predicted to be abnormal; FP (False Positives) represents false positives, that is, the number of samples that are actually normal but predicted to be abnormal.
[0034] The recall rate measures the proportion of samples that are correctly predicted to be abnormal among all samples that are actually abnormal. The calculation formula is: Among them, FN (False Negatives) represents false negatives, that is, the number of samples that are actually abnormal but predicted to be normal. F1-score is an indicator that combines precision and recall, aiming to provide a more balanced evaluation. It is the harmonic mean of precision and recall, and the calculation formula is: These indicators help to more comprehensively and accurately evaluate the performance of the algorithm in predicting abnormal power loads of high-voltage non-residential users.
[0035] In order to evaluate the performance of different methods in detecting abnormal load fluctuations, the experiment used a variety of methods such as 3σ threshold method, LSTM (Long Short-Term Memory Network) and Anomaly Transformer, and conducted comparative tests on the Zhejiang Province High Voltage Non-Residential Dataset. The results are as follows: As can be seen from the above table, Anomaly Transformer shows significant advantages in abnormal load fluctuation detection. It models time series data through the powerful Transformer self-attention mechanism, achieving high precision (0.879) and high recall rate (0.866), as well as excellent F1-score (0.881), showing high accuracy in identifying abnormal events and comprehensive detection effects. It is an effective method in the task of detecting abnormal power load fluctuations.
[0036] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.
[0037] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for detecting abnormal fluctuations in power load based on Transformer architecture, characterized in that: The following steps are involved: Step 1: Obtain power load data and preprocess the power load data; Step 2, performing sequence encoding on the preprocessed power load data to convert the power load data into a vector representation; Step 3: construct an anomaly transformation model based on the Transformer architecture, including constructing a multi-layer Transformer encoder and mapping the output of the multi-layer Transformer encoder to anomaly scores; Step 4: Based on the calculated anomaly score, determine whether the load data at each time point is abnormal.
2. According to claim 1, a method for detecting abnormal fluctuations in power load based on a Transformer architecture is characterized in that: In step 1, the power load data is time series data, and the power load data includes load values at multiple time points, which can be expressed as: L = {l1, l2, ...l T }, Among them l T Represents the power load at the tth time point.
3. According to claim 1, a method for detecting abnormal fluctuations in power load based on a Transformer architecture is characterized in that: In step 1, the preprocessing of the power load data is specifically as follows: Clean the power load data to remove missing values and obviously erroneous outliers in the power load data. Use interpolation, mean filling or median filling methods to fill missing values. Use statistical methods to identify and process obviously erroneous outliers. The power load data after data cleaning is standardized and converted into a standard scale to eliminate the dimensional differences between different load levels. The standardized announcement is as follows: Where μ is the mean of the load data over a period of time, and σ is the standard deviation of the load data over a period of time; The power load data is also subjected to feature extraction, and the load change rate at adjacent time points, the average value and standard deviation within the sliding window are calculated.
4. According to claim 1, a method for detecting abnormal fluctuations in power load based on a Transformer architecture is characterized in that: The step 2 is specifically as follows: The embedding layer is used to convert the power load data at each time point into a vector representation of a fixed size. Assuming the output dimension of the embedding layer is d, the embedded sequence can be expressed as: E}{e1,e2,...e T }, where e T is the vector representation of the power load data at the Tth time point after being transformed by the embedding layer, e T ∈R d , R represents the set of real numbers, and d is the dimension of the embedding vector.
5. According to claim 1, a method for detecting abnormal fluctuations in power load based on a Transformer architecture is characterized in that: In step 3, constructing a multi-layer Transformer encoder is specifically as follows: The multi-layer Transformer encoder consists of multiple stacked self-attention layers, which include attention mechanisms, position encodings, and feedforward neural networks. The self-attention mechanism is: by calculating the matrix of query, key and value and applying the softmax function to obtain the attention weight, the weighted fusion of the information at different positions in the sequence is achieved; Q=W q E W=W k E V / W v E Where Q represents query, K represents key, V represents value, and W represents q , W k , W v is the weight matrix learned by the model; where d k is the dimension of the key; The position coding method includes absolute position coding and relative position coding; The feedforward neural network includes two linear transformations and an activation function.
6. According to claim 5, a method for detecting abnormal fluctuations in power load based on a Transformer architecture is characterized in that: In step 3, the output of the multi-layer Transformer encoder is mapped to the anomaly score as follows: Map the output of the Transformer encoder to anomaly scores using a fully connected layer, which receives the representation of the last time point output by the encoder and outputs one or more anomaly scores; FFN(Z)=ReLU(W1Z+b1)W2+b2 Where FFN represents a feedforward neural network, Z is the output of the self-attention mechanism, W1 and W2 are the weights learned by the model, b1 and b2 are the biases learned by the model, and ReLU is the activation function.
7. The method for detecting abnormal fluctuation of power load based on Transformer architecture according to claim 6 is characterized in that: The model parameters are also optimized and a loss function is constructed. The loss function includes a mean square error or a mean absolute error loss function. The model is trained using a gradient descent optimization algorithm to optimize the model parameters with the goal of minimizing the loss function.
8. The method for detecting abnormal fluctuation of power load based on Transformer architecture according to claim 1 is characterized in that: The step 4 is specifically as follows: Set an anomaly threshold and compare the anomaly score with the preset threshold. If the anomaly score exceeds the threshold, the time point is marked as an anomaly, otherwise it is marked as normal.
9. The method for detecting abnormal fluctuation of power load based on Transformer architecture according to claim 8 is characterized in that: The abnormal threshold is determined by cross-validation, empirical rules or a method based on statistical distribution.