Power load data anomaly detection method based on generative adversarial transformer
By constructing an improved generative adversarial Transformer model and combining multi-stage mapping and unsupervised learning techniques, the problem of extracting relevant features in power load data anomaly detection using the Transformer model is solved, achieving more efficient anomaly detection and recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING INST OF TECH
- Filing Date
- 2023-05-29
- Publication Date
- 2026-06-09
AI Technical Summary
Existing Transformer models struggle to extract correlation features from data before and after load anomalies, resulting in low anomaly detection accuracy.
An improved generative adversarial Transformer-based model for detecting anomalies in power load data is constructed. By introducing a multi-stage mapping and training method, and employing a multi-head attention layer and unsupervised learning techniques in the adversarial generative network, the model extracts anomaly features from the load data and combines a focus score scoring mechanism for anomaly diagnosis.
It significantly improves the accuracy and efficiency of power load data anomaly detection, enabling better extraction of load data anomaly characteristics and enhancing the accuracy of power system planning and energy consumption analysis.
Smart Images
Figure CN116738204B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for detecting anomalies in power load data based on generative adversarial Transformer, belonging to the field of power load detection technology. Background Technology
[0002] The power load data structure suffers from diverse and heterogeneous characteristics and large data volume. During data acquisition and transmission, it is highly susceptible to factors such as system failures, equipment malfunctions, transmission errors, and measurement errors. This results in the inclusion of subtle anomalies in the power load data, including extreme value anomalies and fluctuation anomalies. For example... Figure 1 (a) Extreme load anomaly curves are characterized by extreme load data anomalies, with load peaks, troughs, or breaks occurring at a certain moment or within a short period, disrupting the curve's similarity and periodicity. For example... Figure 1 (b) The abnormal load curve is characterized by a large number of spikes and frequent fluctuations in a short period of time, which is significantly different from the normal load fluctuation pattern.
[0003] These abnormal data will have immeasurable negative impacts on system planning, load forecasting, and energy consumption analysis. Therefore, detecting and identifying abnormal power load data can: firstly, promptly discover and quickly diagnose faults and user electricity theft; and secondly, prevent the further spread of abnormal load data, thus preventing power system planning from making erroneous decisions based on abnormal data.
[0004] Existing Transformer models perform well in time series processing tasks. Their masked multi-head attention layer ensures that the decoder only focuses on the feature mapping of data before time t to data at time t, without introducing data information after time t. Although it can make some inferences about time trends, it is difficult to extract the correlation features of data before and after the time of load anomalies or time periods, resulting in low anomaly detection accuracy.
[0005] The above-mentioned issues should be considered and resolved in the process of power load data anomaly detection based on generative adversarial Transformer. Summary of the Invention
[0006] The purpose of this invention is to provide a power load data anomaly detection method based on generative adversarial Transformer to solve the problem in the prior art that it is difficult to extract the correlation features of data before and after the load anomaly time or period, and the anomaly detection accuracy needs to be improved.
[0007] The technical solution of this invention is:
[0008] A method for detecting anomalies in power load data based on generative adversarial transformation includes the following steps:
[0009] S1. Obtain historical power load data and divide the historical power load data into a training sample set and a test sample set according to a set ratio;
[0010] S2. After normalizing the training samples in the training sample set, the processed training sample set is obtained.
[0011] S3. Construct an anomaly detection model for power load data based on an improved generative adversarial Transformer. Use the processed training sample set to train the anomaly detection model for power load data based on an improved generative adversarial Transformer to obtain the trained anomaly detection model for power load data based on an improved generative adversarial Transformer.
[0012] S4. Input the test sample set into the trained power load data anomaly detection model based on the improved generative adversarial Transformer to obtain the load data prediction results.
[0013] S5. Perform anomaly diagnosis on the load data prediction results according to the anomaly detection mechanism to determine whether the data is abnormal.
[0014] Furthermore, in step S3, the constructed power load data anomaly detection model based on the improved generative adversarial Transformer includes a generator network module and a discriminator network module. The generator network module includes a first generator and a second generator, and the discriminator network module includes a first discriminator and a second discriminator.
[0015] First generator: The focus score is position-encoded to obtain the first position code information. The focus score and the first position code information are concatenated to obtain the input matrix I1. The mapping data is obtained from the input matrix I1.
[0016] The second generator: After position encoding the power load data sequence W, it obtains the second position encoding information. It then concatenates the power load data sequence with the second position encoding information to obtain the input matrix I2, and uses the input matrix I2 to obtain the remapped data.
[0017] First discriminator: Input mapping data and remapping data The reconstructed load sequence O1 is obtained;
[0018] Second discriminator: Input remapping data The output during the model mapping phase is the first-phase reconstruction sequence. The output during the model training and remapping phases is the reconstruction sequence for the second phase. As a result of load data prediction.
[0019] Furthermore, in the first generator, the mapping data is obtained from the input matrix I1. The specific expression is:
[0020]
[0021] in, This represents the output data of the input matrix I1 after passing through the first multi-head attention layer MH1(I1,I1,I1). Represents the input matrix I1 and the output data After the first residual connection and normalization layer The output data, Indicates output data and the first feedforward network layer The output passes through a second residual connection and a normalization layer. The output data is the mapping data.
[0022] Furthermore, in the second generator, the remapping data is obtained from the input matrix I2. The specific expression is:
[0023]
[0024] in, This represents the output data of the input matrix I2 after passing through the second multi-head attention layer MH2(I2,I2,I2). Represents the input matrix I2 and the output data After the third residual connection and normalization layer The output data, Indicates output data and the third multi-head attention layer The output passes through the fourth residual connection and normalization layer. The output data is the remapped data, and the third multi-head attention layer. The inputs are the output data. and mapping data
[0025] Furthermore, in the first discriminator, the input mapping data I13 and remapping data I23 are used to obtain the specific expression for the reconstructed load sequence O1:
[0026]
[0027] in, This indicates the second feedforward network layer, and Sigmoid1 indicates the first activation function layer used to activate the output in the range [0,1].
[0028] Furthermore, in the second discriminator, the input remapping data The output during the model mapping phase is the first-phase reconstruction sequence. The specific expression is
[0029]
[0030] Wherein, FFN3(I23) represents the third feedforward network layer, and Sigmoid2 represents the second activation function layer used to activate the output in the range [0,1].
[0031] The output during the model training and remapping phases is the reconstruction sequence for the second phase. The specific expression is
[0032]
[0033] in, This indicates the third feedforward network layer, and Sigmoid2 indicates the second activation function layer used to activate the output in the range [0,1].
[0034] Further, in step S3, the processed training sample set is used to train the improved generative adversarial Transformer-based power load data anomaly detection model, specifically,
[0035] S31. In the first stage, namely the model mapping stage, an improved power load data anomaly detection model based on generative adversarial Transformer is used to generate a first-stage reconstruction sequence that is consistent with the distribution of the input power load data sequence.
[0036] S32. In the second stage, namely the model training and remapping stage, the power load data anomaly detection model based on the improved generative adversarial Transformer is trained adversarially and then remapped to obtain the second stage reconstruction sequence, which serves as the load data prediction result.
[0037] Further, in step S31, the improved generative adversarial Transformer-based power load data anomaly detection model is used to generate a first-stage reconstruction sequence consistent with the distribution of the input power load data sequence. Specifically,
[0038] S311. After position encoding the focus score, obtain the first position encoding information. Concatenate the focus score and the first position encoding information to obtain the input matrix I1, and input it into the first generator. The first generator outputs the mapping data.
[0039] S312. After the power load data sequence W is position-encoded, the second position-encoded information is obtained. The focus score is concatenated with the second position-encoded information to obtain the input matrix I2, which is then input into the second generator. At the same time, the mapping data is... As input to the third multi-head attention layer of the second generator, the second generator outputs remapped data.
[0040] S313, Mapping data With remapping data Both are used as inputs to the first discriminator, which then derives the reconstructed load sequence O1.
[0041] S314, Remap the data As input to the second discriminator, the second discriminator outputs the first-stage reconstructed sequence.
[0042] Furthermore, in step S32, after adversarial training is completed on the power load data anomaly detection model based on the improved generative adversarial Transformer, remapping is performed to obtain the second-stage reconstruction sequence, specifically,
[0043] S321. Calculate the reconstruction loss of the first discriminator in the first stage. L1 (1) =||O1-W||2
[0044] S322, The reconstruction loss of the first discriminator in the first stage. As the focus score for the second stage;
[0045] S323. After obtaining the third position encoding information by position encoding of the focus score of the second stage, the focus score of the second stage and the third position encoding information are concatenated and input into the first generator. The first generator outputs the mapping data of the second stage to the third multi-head attention layer of the second generator.
[0046] S324, Simultaneously reconstruct the first-stage sequence After obtaining the fourth positional encoding information through positional encoding, the first-stage reconstructed sequence is concatenated. The data, along with the fourth position encoding information, is input to the second generator, which then outputs the remapping data for the second stage to the second discriminator.
[0047] S325, The second discriminator outputs the second-stage reconstruction sequence.
[0048] Further, in step S5, the load data prediction results are subjected to anomaly diagnosis based on the anomaly detection mechanism to determine whether they are abnormal data. Specifically, the anomaly detection mechanism introduces anomaly scores and anomaly diagnostic labels to determine abnormal data. The anomaly score st and the anomaly diagnostic label y are respectively:
[0049]
[0050] Where O1 represents the reconstructed load sequence, This indicates the first-stage reconstruction sequence. This represents the second-stage reconstruction sequence, i.e., the load data prediction result, POT(s). t ) represents the threshold, y t =1 indicates an anomaly label for the data at time t, y t =0 indicates the normal label for the data at time t.
[0051] The beneficial effects of this invention are:
[0052] I. This power load data anomaly detection method based on Generative Adversarial Transformer (GAP) constructs an improved GAP model for power load data anomaly detection, introduces multi-stage mapping and training methods, integrates a focus score scoring mechanism, and reconstructs load sequences in stages. This allows for better extraction of load data anomaly features, improves the accuracy of power load data anomaly detection, effectively enhances the detection efficiency of power load anomaly data, and improves the quality of power load data. This has significant implications for power system planning, load forecasting, and energy consumption analysis.
[0053] Second, in this invention, the second generator of the power load data anomaly detection model based on the improved generative adversarial Transformer adopts a multi-head attention layer instead of a masked multi-head attention layer to ensure that the discriminator network module, as a decoder, not only focuses on the feature mapping of the data before time t to the data at time t, but also introduces the data information after time t to extract the correlation features of the load anomaly data before or after time t; at the same time, the feedforward network is removed to balance the global and local dependencies of the model inference.
[0054] Third, this power load data anomaly detection method based on generative adversarial Transformer constructs an improved power load data anomaly detection model based on generative adversarial Transformer. It combines Transformer with adversarial generative networks using unsupervised learning techniques. By analyzing the periodicity, autocorrelation, and trend features between a set of sequences, it can focus on the duration of anomalies and the degree of mutation when extracting features, thus significantly improving the anomaly detection accuracy and generalization ability, and achieving rapid and accurate identification of abnormal data. Attached Figure Description
[0055] Figure 1 These are schematic diagrams illustrating abnormal data in power load data, where (a) is a schematic diagram illustrating extreme value anomalies in power load data, and (b) is a schematic diagram illustrating fluctuation anomalies in power load data.
[0056] Figure 2 This is a flowchart illustrating the power load data anomaly detection method based on generative adversarial Transformer in an embodiment of the present invention.
[0057] Figure 3 This is an illustrative diagram illustrating the power load data anomaly detection model based on the improved generative adversarial Transformer in the embodiment.
[0058] Figure 4 This is a schematic diagram illustrating the generation of a reconstructed dataset and anomaly scores based on historical power load data in the embodiment.
[0059] Figure 5 yes Figure 4 A schematic diagram showing the reconstructed dataset and anomaly scores of extreme anomaly one at point a and extreme anomaly two at point b, where (a) is... Figure 4 A schematic diagram of the reconstructed dataset and anomaly score of the extreme anomaly at point a, and (b) is... Figure 4 A schematic diagram of the reconstructed dataset and anomaly score of extreme anomaly 2 at point b;
[0060] Figure 6 This is a schematic diagram illustrating the generation of a reconstructed dataset and anomaly scores based on electricity load data in the embodiment.
[0061] Figure 7 yes Figure 6 A schematic diagram showing the reconstructed dataset and anomaly scores for extreme value anomalies at point a and fluctuation anomalies at point b, where (a) is... Figure 6 A schematic diagram of the reconstructed dataset and anomaly score for the extreme value anomaly at point a, and (b) is... Figure 6 A schematic diagram of the reconstructed dataset and anomaly score for the fluctuation anomaly at point b. Detailed Implementation
[0062] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0063] Example
[0064] A method for detecting anomalies in power load data based on generative adversarial Transformer, such as Figure 2 This includes the following steps:
[0065] S1. Obtain historical power load data and divide the historical power load data into a training sample set and a test sample set according to a set ratio;
[0066] In step S1, typical historical power system load data is acquired. The acquisition frequency can be once every 30 seconds. The dimension size of the dataset can be defined as 8000*2. The acquired historical power load data is divided into a training sample set and a test sample set, with the ratio of training sample set to validation sample set data volume being 7:3.
[0067] S2. After normalizing the training samples in the training sample set, the processed training sample set is obtained.
[0068] In step S2, the training samples are normalized using the min-max normalization method. Both the input training samples and the generated data from multiple scenarios are scaled proportionally and converted to dimensionless normalized values in the range [0,1].
[0069]
[0070] Where min and max are the minimum and maximum value vectors in the time series; ε' is a constant vector, x * t Let x be the training sample data at time t. t The value after normalization.
[0071] S3. Construct an anomaly detection model for power load data based on an improved generative adversarial Transformer. Use the processed training sample set to train the anomaly detection model for power load data based on an improved generative adversarial Transformer to obtain the trained anomaly detection model for power load data based on an improved generative adversarial Transformer.
[0072] In step S3, such as Figure 3 The constructed power load data anomaly detection model based on the improved generative adversarial Transformer includes a generator network module and a discriminator network module. The generator network module acts as an encoder, and the discriminator network module acts as a decoder. The generator network module includes a first generator and a second generator, and the discriminator network module includes a first discriminator and a second discriminator.
[0073] First generator: The focus score is position-encoded to obtain the first position code information. The focus score and the first position code information are concatenated to obtain the input matrix I1. The mapping data is obtained from the input matrix I1. In the first stage, the initial value of the focus score is... The reconstruction loss of the first discriminator in the first stage As the focus score for the second stage;
[0074] In the first generator, the mapping data is obtained from the input matrix I1. The specific expression is:
[0075]
[0076] in, This represents the output data of the input matrix I1 after passing through the first multi-head attention layer MH1(I1,I1,I1). Represents the input matrix I1 and the output data After the first residual connection and the normalization layer LayerNorm1(I1,I1) 1 Output data, Indicates output data and the first feedforward network layer The output passes through a second residual connection and a normalization layer. The output data is the mapping data.
[0077] The second generator: After position encoding the power load data sequence W, it obtains the second position encoding information. It then concatenates the power load data sequence with the second position encoding information to obtain the input matrix I2, and uses the input matrix I2 to obtain the remapped data.
[0078] In the second generator, the remapped data is obtained from the input matrix I2. The specific expression is:
[0079]
[0080] in, This represents the output data of the input matrix I2 after passing through the second multi-head attention layer MH2(I2,I2,I2). Represents the input matrix I2 and the output data After the third residual connection and normalization layer The output data, Indicates output data and the third multi-head attention layer The output passes through the fourth residual connection and normalization layer. The output data is the remapped data, and the third multi-head attention layer. The inputs are the output data. and mapping data
[0081] First discriminator: Input mapping data and remapping data The reconstructed load sequence O1 is obtained;
[0082] In the first discriminator, the input mapping data and remapping data The specific expression for obtaining the reconstructed load sequence O1 is:
[0083]
[0084] in, This indicates the second feedforward network layer, and Sigmoid1 indicates the first activation function layer used to activate the output in the range [0,1].
[0085] Second discriminator: Input remapping data The output during the model mapping phase is the first-phase reconstruction sequence. The output during the model training and remapping phases is the reconstruction sequence for the second phase. As a result of load data prediction.
[0086] In the second discriminator, the input remapping data The output during the model mapping phase is the first-phase reconstruction sequence. The specific expression is
[0087]
[0088] in, This indicates the third feedforward network layer, and Sigmoid2 indicates the second activation function layer used to activate the output in the range [0,1].
[0089] The output during the model training and remapping phases is the reconstruction sequence for the second phase. The specific expression is
[0090]
[0091] in, This indicates the third feedforward network layer, and Sigmoid2 indicates the second activation function layer used to activate the output in the range [0,1].
[0092] In step S3, the processed training sample set is used to train the improved generative adversarial Transformer-based power load data anomaly detection model. Specifically,
[0093] S31. In the first stage, namely the model mapping stage, an improved power load data anomaly detection model based on generative adversarial Transformer is used to generate a first-stage reconstruction sequence that is consistent with the distribution of the input power load data sequence.
[0094] In step S31, an improved generative adversarial Transformer-based power load data anomaly detection model is used to generate a first-stage reconstruction sequence that is consistent with the distribution of the input power load data sequence. Specifically,
[0095] S311. After position encoding the focus score, obtain the first position encoding information. Concatenate the focus score and the first position encoding information to obtain the input matrix I1, and input it into the first generator. The first generator outputs the mapping data.
[0096] S312. After the power load data sequence W is position-encoded, the second position-encoded information is obtained. The focus score is concatenated with the second position-encoded information to obtain the input matrix I2, which is then input into the second generator. At the same time, the mapping data is... As input to the third multi-head attention layer of the second generator, the second generator outputs remapped data.
[0097] S313, Mapping data With remapping data Both are used as inputs to the first discriminator, which then derives the reconstructed load sequence O1.
[0098] S314, Remap the data As input to the second discriminator, the second discriminator outputs the first-stage reconstructed sequence.
[0099] S32. In the second stage, namely the model training and remapping stage, the power load data anomaly detection model based on the improved generative adversarial Transformer is trained adversarially and then remapped to obtain the second stage reconstruction sequence, which serves as the load data prediction result.
[0100] In step S32, the power load data anomaly detection model based on the improved generative adversarial Transformer is trained adversarially and then remapped to obtain the second-stage reconstruction sequence. Specifically,
[0101] S321. Calculate the reconstruction loss of the first discriminator in the first stage. L1 (1) =||O1-W||2
[0102] S322, The reconstruction loss of the first discriminator in the first stage. As the focus score for the second stage;
[0103] S323. After obtaining the third position encoding information by position encoding of the focus score of the second stage, the focus score of the second stage and the third position encoding information are concatenated and input into the first generator. The first generator outputs the mapping data of the second stage to the third multi-head attention layer of the second generator.
[0104] S324, Simultaneously reconstruct the first-stage sequence After obtaining the fourth positional encoding information through positional encoding, the first-stage reconstructed sequence is concatenated. The data, along with the fourth position encoding information, is input to the second generator, which then outputs the remapping data for the second stage to the second discriminator.
[0105] S325, The second discriminator outputs the second-stage reconstruction sequence.
[0106] In step S3, the improved generative adversarial Transformer-based power load data anomaly detection model is trained using a multi-stage mapping approach. The goal of the first stage is to generate a first-stage reconstructed sequence consistent with the distribution of the input load data. The output of the first-stage mapping is used as the input for the second stage. In the second stage, the reconstruction loss of the first discriminator is used as the focus score, prompting the generator network module to focus on subsequences exhibiting abnormal trends.
[0107] In step S3, the first stage is the model mapping stage, which involves mapping the focus scores. After location encoding, the first location encoding information is obtained. The focus score is concatenated with the first location encoding information to obtain input matrix I1, which is then input into the first generator. The outputs of the first and second generators serve as input to the first discriminator, which generates the reconstructed load sequence O1. Simultaneously, the historical power load data sequence W is location encoded to obtain the second location encoding information. The focus score is concatenated with the second location encoding information to obtain input matrix I2, which is then input into the second generator. The output of the first generator serves as part of the input to the third multi-head attention layer of the second generator, and the output of the second generator serves as the input to the second discriminator. At this point, the output of the second discriminator corresponds to the reconstructed sequence of the first stage.
[0108] In step S3, the second stage is the model training and remapping stage. Based on the mapping in the first stage, the model undergoes adversarial training before remapping. The reconstruction loss of the first discriminator from the first stage is then used... As the second-stage focus score, the second-stage focus score is positionally encoded to obtain third-stage positional encoding information. This third-stage focus score is then concatenated with the third-stage positional encoding information and input into the first generator. Simultaneously, the first-stage reconstructed sequence is... After obtaining the fourth positional encoding information through positional encoding, the sequence is concatenated with the first-stage reconstructed sequence. After being encoded with the fourth position information, it is input again into the second generator. At this time, the output of the second discriminator is the reconstructed sequence of the second stage. As a result of load data prediction.
[0109] S4. Input the test sample set into the trained power load data anomaly detection model based on the improved generative adversarial Transformer to obtain the load data prediction results.
[0110] S5. Perform anomaly diagnosis on the load data prediction results according to the anomaly detection mechanism to determine whether the data is abnormal.
[0111] In step S5, the load data prediction results are analyzed using an anomaly detection mechanism to determine whether they are abnormal data. Specifically, the anomaly detection mechanism introduces an anomaly score and an anomaly diagnostic label to identify abnormal data. The anomaly score st and the anomaly diagnostic label y are respectively:
[0112]
[0113] Where O1 represents the reconstructed load sequence, This indicates the first-stage reconstruction sequence. This represents the second-stage reconstruction sequence, i.e., the load data prediction result, POT(s). t ) represents the threshold, y t =1 indicates an anomaly label for the data at time t, y t =0 indicates the normal label for the data at time t.
[0114] In step S4, in order to accurately identify outlier data points in the input load data, the input load data is compared with the first-stage reconstruction sequence processed by the model. To compare the results, anomaly scores and anomaly diagnostic labels are introduced to detect anomalous data. The anomaly score is defined as the sum of the L2 norms of the two-stage reconstructed sequence and the input data; a higher anomaly score indicates a greater anomalous deviation in the load data at that point. The anomaly label is defined as the anomaly score at a specific time point, determined by the anomaly score exceeding a set threshold. The peak exceedance (POT) method is used to automatically and dynamically select the threshold.
[0115] In the improved generative adversarial Transformer-based power load data anomaly detection model, the reconstruction loss function of the first discriminator and the second discriminator in the first stage is L1. (1) and L2 (1) for: In the second stage, the reconstruction loss function of the first and second discriminators is L1. (2) and L2(2) for: Therefore, the overall loss functions L1 and L2 of the power load data anomaly detection model based on the improved generative adversarial Transformer are: Where n is the number of times the model is trained, and ε is the hyperparameter.
[0116] This power load data anomaly detection method based on Generative Adversarial Transformer (GAP) constructs an improved GAP model for power load data anomaly detection, introduces multi-stage mapping and training methods, integrates a focus score scoring mechanism, and reconstructs load sequences in stages. This helps the model better extract anomaly features from load data, improves the accuracy of power load data anomaly detection, effectively enhances the detection efficiency of power load anomaly data, and improves the quality of power load data. It has significant positive implications for power system planning, load forecasting, and energy consumption analysis.
[0117] In this invention, the second generator of the power load data anomaly detection model based on the improved generative adversarial Transformer adopts a multi-head attention layer instead of a masked multi-head attention layer to ensure that the discriminator network module, as a decoder, not only focuses on the feature mapping of the data before time t to the data at time t, but also introduces the data information after time t to extract the correlation features of the load anomaly data before or after time t; at the same time, the feedforward network is removed to balance the global and local dependencies of the model inference.
[0118] This power load data anomaly detection method based on generative adversarial Transformer (GAP) constructs an improved GAP model for power load data anomaly detection. By combining the Transformer with adversarial generative networks using unsupervised learning techniques, it can analyze the periodicity, autocorrelation, and trend features between a set of sequences. When extracting features, it can focus on both the duration of anomalies and the degree of mutation, significantly improving the anomaly detection accuracy and generalization ability, and achieving rapid and accurate identification of abnormal data.
[0119] The experimental verification of this power load data anomaly detection method based on generative adversarial Transformer in the embodiment is as follows:
[0120] The power load data anomaly detection method based on generative adversarial transformation in this embodiment uses historical power supply load data and power consumption load data of a certain power supply area. The data collection time interval is 30 seconds, the time span is 90 days, and the abnormal data covers two typical features: fluctuation anomaly and extreme value anomaly.
[0121] When training the power load data anomaly detection model based on the improved generative adversarial Transformer, the hyperparameter ε is set to 1, and the number of training iterations is set to 500. This model is then applied to a power load data anomaly detection scenario. The anomaly detection results for power supply load data and power consumption load data in the embodiment are as follows: Figure 4 , Figure 5 , Figure 6 and Figure 7 .
[0122] Figure 4 This is a schematic diagram illustrating the generation of a reconstructed dataset and anomaly scores based on historical power load data in the embodiment. Figure 5 yes Figure 4 A schematic diagram showing the reconstructed dataset and anomaly scores for extreme anomaly one at point a and extreme anomaly two at point b. Figure 5 (a) is Figure 4 A schematic diagram of the reconstructed dataset and anomaly score for extreme anomaly 1 at point a. Figure 5 (b) is Figure 4 A schematic diagram showing the reconstructed dataset and anomaly score of the second extreme anomaly at point b. Analysis Figure 4 and Figure 5 The power supply load data, model reconstruction sequence, and anomaly score results show that the GAN-Transformer model in the embodiment can accurately detect and identify extreme value anomalies in the power supply load data.
[0123] Figure 6 This is a schematic diagram illustrating the generation of a reconstructed dataset and anomaly scores based on electricity load data in the embodiment. Figure 7 yes Figure 6 A schematic diagram showing the reconstructed dataset and anomaly scores for extreme value anomalies at point a and fluctuation anomalies at point b. Figure 7 (a) is Figure 6 A schematic diagram of the reconstructed dataset and anomaly scores for the extreme value anomaly at point a. Figure 7 Yes Figure 6 A schematic diagram showing the reconstructed dataset and anomaly score for the fluctuation anomaly at point b. Analysis Figure 6 and Figure 7 The reconstructed data generated by the second discriminator can effectively capture the variation characteristics of a given power load curve. Whether from the power load data and model reconstruction sequence or the anomaly score results, it can be concluded that the GAN-Transformer model in this embodiment can accurately detect and identify extreme value anomalies and fluctuation anomalies in the power load data.
[0124] Using a validation sample set, the output power supply load and power consumption load reconstructed data were compared with existing control models such as OmniAnomaly, LSTM-NDT, MAD-GAN, DAGMM, USAD, and Transformer in anomaly detection experiments. Precision, recall, and F1 score were used to comprehensively evaluate the data anomaly detection performance of the generative adversarial Transformer model. The results are shown in Tables 1 and 2.
[0125] Table 1. Anomaly detection results of power supply load data using the method in this embodiment and existing methods.
[0126] Model Pre Rec F1 Training time s / time OmniAnomaly 0.4735 0.6634 0.5521 10.54 LSTM-NDT 0.8454 0.9846 0.9097 3.11 MAD-GAN 0.8519 0.9599 0.9026 10.30 DAGMM 0.8243 0.9999 0.9037 7.00 USAD 0.8409 0.9999 0.9135 7.90 Transformer 0.7469 0.8635 0.8010 6.00 Generative Adversarial Transformer 0.9492 0.9384 0.9437 2.30
[0127] Table 1 shows the anomaly detection results of the power load data. The anomaly detection model based on the improved Generative Adversarial Transformer (GAP) of the embodiment method achieves a detection precision of 0.9492, which is 10% to 20% higher than other existing control models. The recall of the GAP model based on the improved GAP is 0.9384, slightly lower than the control models LSTM-NDT, MAD-GAN, DAGMM, and USAD, but higher than OmniAnomaly and the Transformer model. This indicates that the GAP model of the embodiment has a lower recall rate for anomaly data, and consequently, a lower false positive rate.
[0128] Table 2. Anomaly detection results of electricity load data using the method in this embodiment and existing methods.
[0129] Model Pre Rec F1 Training time s / time OmniAnomaly 0.7965 0.8523 0.8234 10.54 LSTM-NDT 0.5013 0.6551 0.5680 3.11 MAD-GAN 0.8967 0.8677 0.8820 12.30 DAGMM 0.8682 0.9799 0.9206 7.00 USAD 0.9261 0.9199 0.9229 7.90 Transformer 0.7652 0.8699 0.8141 6.00 Generative Adversarial Transformer 0.9762 0.9264 0.9506 2.30
[0130] Table 2 shows that the power load data anomaly detection results of the embodiment method have a detection precision of 0.9762 based on the improved generative adversarial Transformer, which is still superior to other existing control models. At this time, the recall of the power load data anomaly detection model based on the improved generative adversarial Transformer is 0.9264, which is similar to other existing control models.
[0131] Based on a comprehensive analysis of precision and recall results, Table 1 shows that the F1 score of the power load data anomaly detection model based on the improved generative adversarial Transformer in the example method is 0.9437, and Table 2 shows that the F1 score of the same model is 0.9506, both significantly higher than the existing control model and closest to 1. Analysis of the training time of different models shows that, with the same number of training iterations and a 70% reduction in training time, the anomaly detection accuracy of the power load data anomaly detection model based on the improved generative adversarial Transformer in the example method is significantly improved, indicating superior overall model performance.
[0132] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for detecting anomalies in power load data based on generative adversarial transformation, characterized in that: Includes the following steps, S1. Obtain historical power load data and divide the historical power load data into a training sample set and a test sample set according to a set ratio; S2. After normalizing the training samples in the training sample set, the processed training sample set is obtained. S3. Construct an anomaly detection model for power load data based on an improved generative adversarial Transformer. Use the processed training sample set to train the anomaly detection model for power load data based on an improved generative adversarial Transformer to obtain the trained anomaly detection model for power load data based on an improved generative adversarial Transformer. In step S3, the constructed power load data anomaly detection model based on the improved generative adversarial Transformer includes a generator network module and a discriminator network module. The generator network module includes a first generator and a second generator, and the discriminator network module includes a first discriminator and a second discriminator. First generator: The focus score is position-encoded to obtain the first position-encoded information, and the focus score and the first position-encoded information are concatenated to obtain the input matrix. And from the input matrix Obtain mapping data ; In the first generator, the input matrix... Obtain mapping data The specific expression is: ， in, This indicates that the input matrix I1 has passed through the first multi-head attention layer. The output data, Represents the input matrix and output data After the first residual connection and normalization layer The output data, Indicates output data and the first feedforward network layer The output passes through a second residual connection and a normalization layer. The output data is the mapping data; The second generator: After position encoding the power load data sequence W, it obtains the second position encoding information. It then concatenates the power load data sequence with the second position encoding information to obtain the input matrix I2, and uses the input matrix I2 to obtain the remapped data. ; In the second generator, the input matrix... Obtain remapping data The specific expression is: ， in, Represents the input matrix After the second multi-head attention layer The output data, Represents the input matrix and output data After the third residual connection and normalization layer The output data, Indicates output data and the third multi-head attention layer The output passes through the fourth residual connection and normalization layer. The output data is the remapped data, and the third multi-head attention layer. The inputs are the output data. and mapping data ; First discriminator: Input mapping data and remapping data The reconstructed load sequence is obtained. The specific expression is: ， in, This indicates the second feedforward network layer. This represents the first activation function layer used to activate outputs in the range [0,1]. Second discriminator: Input remapping data The output of the model mapping stage is the first-stage reconstruction sequence. The output during the model training and remapping phase is the reconstruction sequence for the second phase. This serves as the result of load data prediction. In the second discriminator, the input remapping data The output of the model mapping stage is the first-stage reconstruction sequence. The specific expression is ， in, This indicates the third feedforward network layer. This represents the second activation function layer used to activate outputs in the range [0,1]. The output during the model training and remapping phases is the reconstruction sequence for the second phase. The specific expression is ， in, This indicates the third feedforward network layer. This represents the second activation function layer used to activate outputs in the range [0,1]. S4. Input the test sample set into the trained power load data anomaly detection model based on the improved generative adversarial Transformer to obtain the load data prediction results. S5. Perform anomaly diagnosis on the load data prediction results according to the anomaly detection mechanism to determine whether the data is abnormal.
2. The power load data anomaly detection method based on generative adversarial transformation as described in claim 1, characterized in that: In step S3, the processed training sample set is used to train the improved generative adversarial Transformer-based power load data anomaly detection model. Specifically, S31. In the first stage, namely the model mapping stage, an improved power load data anomaly detection model based on generative adversarial Transformer is used to generate a first-stage reconstruction sequence that is consistent with the distribution of the input power load data sequence. S32. In the second stage, namely the model training and remapping stage, the power load data anomaly detection model based on the improved generative adversarial Transformer is trained adversarially and then remapped to obtain the second stage reconstruction sequence, which serves as the load data prediction result.
3. The power load data anomaly detection method based on generative adversarial transformation as described in claim 2, characterized in that: In step S31, an improved generative adversarial Transformer-based power load data anomaly detection model is used to generate a first-stage reconstruction sequence that is consistent with the distribution of the input power load data sequence. Specifically, S311. After position encoding the focus score, obtain the first position encoding information. Concatenate the focus score and the first position encoding information to obtain the input matrix I1, and input it into the first generator. The first generator outputs the mapping data. ; S312. After the power load data sequence W is position-encoded, the second position-encoded information is obtained. The focus score is concatenated with the second position-encoded information to obtain the input matrix I2, which is then input into the second generator. At the same time, the mapping data is... As input to the third multi-head attention layer of the second generator, the second generator outputs remapped data. ; S313, Mapping data With remapping data Both serve as inputs to the first discriminator, from which the reconstructed load sequence is derived. ; S314, Remap the data As input to the second discriminator, the second discriminator outputs the first-stage reconstructed sequence. .
4. The power load data anomaly detection method based on generative adversarial transformation as described in claim 3, characterized in that: In step S32, the power load data anomaly detection model based on the improved generative adversarial Transformer is trained adversarially and then remapped to obtain the second-stage reconstruction sequence. Specifically, S321. Calculate the reconstruction loss of the first discriminator in the first stage. ; S322, The reconstruction loss of the first discriminator in the first stage. As the focus score for the second stage; S323. After obtaining the third position encoding information by position encoding of the focus score of the second stage, the focus score of the second stage and the third position encoding information are concatenated and input into the first generator. The first generator outputs the mapping data of the second stage to the third multi-head attention layer of the second generator. S324, Simultaneously reconstruct the first-stage sequence After obtaining the fourth positional encoding information through positional encoding, the sequence is concatenated with the first-stage reconstructed sequence. The data, along with the fourth position encoding information, is input to the second generator, which then outputs the remapping data for the second stage to the second discriminator. S325, The second discriminator outputs the second-stage reconstruction sequence. .
5. The power load data anomaly detection method based on generative adversarial transformation as described in claim 4, characterized in that: In step S5, the load data prediction results are analyzed using an anomaly detection mechanism to determine whether they are abnormal data. Specifically, the anomaly detection mechanism introduces an anomaly score and an anomaly diagnostic label to identify abnormal data. The anomaly score st and the anomaly diagnostic label y are respectively: ， in, Indicates the reconfigured load sequence. This indicates the first-stage reconstruction sequence. This indicates the second-stage reconstruction sequence, i.e., the load data prediction result. Indicates the threshold. The label representing the anomaly in the data at time t. This represents the normal label for the data at time t.