Power time series data anomaly detection method, device and medium based on large model

Abstract

The present invention belongs to the field of data analysis technology, and relates to a method, device and medium for detecting anomalies of power time series data based on a large model, wherein the method comprises the following steps: inputting the power time series data to be detected into a trained power time series data anomaly detection model, and outputting the detection result; the power time series data anomaly detection model comprises a wavelet decomposition module, an encoder module and a decoder module connected in sequence, wherein the wavelet decomposition module first performs wavelet decomposition on the power time series data, and then converts each component sequence into a feature representation of equal length through multi-head self-attention; the decoder module calculates the dynamic time warping distance between the output of the encoder module and each historical candidate mode in a differentiable manner, and calculates the contribution score of each historical candidate mode accordingly, and aggregates to obtain the feature representation of the power time series data at each period scale, and outputs it through an inverse wavelet transform. Compared with the prior art, the present invention can improve the accuracy of the anomaly detection results of power time series data.

Description

Technical Field

[0001] The present invention belongs to the technical field of data analysis, and in particular relates to a method, device and medium for detecting anomalies in power time series data based on a large model. Background Art

[0002] Time series data anomaly detection is a big data analysis method widely used in energy, finance, medical and other fields. For example, power companies can perform anomaly detection on power time series load data to provide advance warning and post-event investigation of abnormal events and changes in electricity consumption. This is of great significance for power grid companies to reduce costs and increase efficiency as well as improve the transmission efficiency of distribution networks.

[0003] The development of machine learning has brought significant progress in the research of time series data anomaly detection methods. However, in the process of using big data technology for data analysis, machine learning methods still have shortcomings such as poor interpretability, single data source, and low completeness of user pattern representation. With the continuous deepening of the digital reform of power companies, the requirements for mining and utilization of massive power data are getting higher and higher. The current methods are often based on the assumption of independent and identically distributed data for machine learning. The analysis of power consumption in different periods and different companies is not detailed enough, and the mining of changing power consumption data features is not flexible enough. Compared with other time series data, the scale of power data is larger, the time domain information contained is richer, and the time span included is longer, which also puts higher requirements on the model's ability to capture long-distance dependencies.

[0004] In order to accurately model dynamic time series dependencies and adapt to long-distance and distributed time series data, pre-trained large models have begun to play a role in the field of time series prediction. Existing research on large models for time series prediction mainly focuses on semantic alignment and modal adaptation. For example, Yu uses time series data as contextual prompts for large models and fine-tunes the large models by designing a variety of prompt templates; Koa et al. designed a prompt framework called "summary-explanation-prediction" and used multi-agent technology to improve the model's understanding of time series data; Zhou et al. started from the data itself and directly provided massive time series data to the large model for pre-training, and then fine-tuned it on each downstream task. However, the existing large time series models still have limitations in solving the problem of anomaly detection of power time series data. Current work still remains at the level of using the language analysis capabilities of large models, lacking understanding of the information of the time series data itself, that is, lacking detailed analysis and full understanding of the trend, periodicity, seasonality and other components of the time series data; and essentially still relying on the semantic understanding ability of the large model, there is inevitable information loss in the alignment and adaptation process in the application, which may lead to poor performance of time series data prediction and anomaly detection. Therefore, how to develop a time series data anomaly detection method that is highly accurate and adaptable to the characteristics of power data is crucial for the in-depth digital reform of power companies. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a method, device and medium for detecting anomalies in power time series data based on a large model, so as to improve the accuracy and reliability of the results of detecting anomalies in power time series data.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a method for detecting anomalies in power time series data based on a large model, comprising the following steps:

[0008] Obtain the power time series data to be detected, input the trained power time series data anomaly detection model, and output the detection result; wherein the training process of the power time series data anomaly detection model includes a pre-training stage and a fine-tuning stage, wherein the pre-training stage performs self-supervised training based on a pre-acquired general time series data set, and creates a memory storage unit in the pre-training stage for storing historical candidate modes, wherein the historical candidate modes are generated based on general time series data in the general time series data set whose cosine similarity is higher than a set threshold; and the fine-tuning stage further performs model parameter fine-tuning through the pre-acquired power time series data set;

[0009] The power time series data anomaly detection model includes a wavelet decomposition module, an encoder module and a decoder module connected in sequence. The wavelet decomposition module first performs wavelet decomposition on the power time series data to obtain component sequences of different period scales, and then converts each component sequence into a feature representation of equal length through multi-head self-attention, and divides it into partially overlapping slices and inputs it into the encoder module for encoding; the decoder module uses an improved dynamic time warping algorithm to differentiably calculate the dynamic time warping distance between the output of the encoder module and each historical alternative mode in the memory storage unit, and calculates the contribution score of each historical alternative mode based on this, aggregates to obtain the feature representation of the power time series data at each period scale, and obtains the final output through inverse wavelet transform.

[0010] Furthermore, the power time series data is decomposed by wavelet decomposition using Dobesi wavelet.

[0011] Furthermore, the specific process of converting each component sequence into a feature representation of equal length is as follows:

[0012] Calculate the attention weight of each component sequence at each time step, perform weighted summation on the component sequence at each time step, and obtain the aggregated representation of each component sequence;

[0013] The aggregate representation of each component sequence is projected to the specified output dimension through a linear layer to complete the length alignment.

[0014] Furthermore, the component sequence At time step The attention weight The calculation formula is as follows:

[0015] ,

[0016] component sequence Aggregate representation of The calculation formula is as follows:

[0017] ,

[0018] in, is a component sequence At time step The characteristic vector of is a trainable weight vector, is the length of the component sequence.

[0019] Furthermore, the calculation formula of the dynamic time warping distance between the output of the encoder module and each historical candidate mode is as follows:

[0020] ,

[0021] ,

[0022] in, Output for encoder module With historical alternative mode The dynamic time warping distance between , , each historical alternative mode , is the number of historical alternative models, Indicates the time, is the length of the time series, Output for encoder module With historical alternative mode The dimension of is the smoothing parameter, for The iterative calculation formula is: for and The Euclidean distance of .

[0023] Furthermore, the characteristic representation of power time series data at each cycle scale is as follows:

[0024] ,

[0025] ,

[0026] in, is the number of historical alternative models, Alternative mode for history contribution score.

[0027] Furthermore, the memory storage unit is dynamically updated according to the occurrence frequency of the historical candidate modes stored therein.

[0028] Furthermore, the clustering values ​​of the common time series data whose cosine similarity matrix is ​​higher than a set threshold are calculated and stored in the memory storage unit as historical candidate modes.

[0029] The present invention also provides an electronic device, comprising a memory, a processor, and a program stored in the memory, wherein the processor implements the above method when executing the program.

[0030] The present invention also provides a computer-readable storage medium on which a computer program is stored, and the program implements the above method when executed by a processor.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The present invention proposes a method for detecting anomalies in power time series data based on a large model, and outputs detection results through a trained power time series data anomaly detection model. The training process of the model includes a pre-training stage and a fine-tuning stage. In the pre-training stage, a memory storage unit for storing historical alternative modes is created. The historical alternative modes are generated based on general time series data with cosine similarity higher than a set threshold in a general time series data set. The memory storage unit helps to accumulate more vector patterns, thereby having stronger generalization capabilities when facing new data. The power time series data anomaly detection model includes a wavelet decomposition module, an encoder module and a decoder module connected in sequence, wherein the wavelet decomposition module first performs wavelet decomposition on the power time series data to obtain component sequences of different period scales, which can improve the model's ability to understand time series information; then, each component sequence is converted into a feature representation of equal length through multi-head self-attention, and is divided into partially overlapping slices and input into the encoder module for encoding, which can solve the problem of insufficient information density of the original time series data points; the decoder module improves the dynamic time warping algorithm, and can calculate the dynamic time warping distance between the output of the encoder module and each historical candidate mode in the memory storage unit in a differentiable way, which helps to transfer gradients in training and accelerate model training; finally, the contribution score of each historical candidate mode is calculated according to the dynamic time warping distance, and the feature representation of the power time series data at each period scale is obtained by aggregation, and the final output is obtained by inverse wavelet transform. The above method can accurately and efficiently realize power time series data anomaly detection.

[0033] 2. In the present invention, the memory storage unit is dynamically updated according to the frequency of occurrence of the historical alternative modes stored therein, which can more effectively utilize storage resources, capture dynamic changes in the environment, and ensure the timeliness and effectiveness of the memory storage unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a structural diagram of the power time series data anomaly detection model;

[0035] Figure 2 Schematic diagram of the training process of the power time series data anomaly detection model. DETAILED DESCRIPTION

[0036] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments. Example

[0037] This embodiment provides a method for detecting anomalies in power time series data based on a large model, which performs bidirectional adaptive migration of power time series data and a large language model, inputs the acquired power time series data to be detected into a trained power time series data anomaly detection model, and outputs the detection results.

[0038] like Figure 2 As shown, the training process of the power time series data anomaly detection model includes a pre-training stage and a parameter fine-tuning stage, wherein the pre-training stage refers to the next token prediction method in the large language model training process, and uses a partial masking method to perform self-supervised training based on a pre-acquired general time series data set. In the pre-training stage, this embodiment creates a memory storage unit for storing valuable information during the training process, that is, during the training process, the cosine similarity between each input vector is calculated, and the most frequently occurring classic patterns in the training process are retained in the process of continuous updating and iteration, and used in subsequent prediction steps. Specifically, assuming there is a batch of input vectors }, and The calculation formula of cosine similarity is:

[0039] ,

[0040] Construct a cosine similarity matrix based on the cosine similarities of all vectors , its Line Elements of a column That is .

[0041] For each input vector , find the similarity with its cosine higher than a certain threshold A vector of:

[0042] ,

[0043] Store these high similarity vectors (or their representative representations) as classic patterns in the memory storage unit:

[0044] ,

[0045] in, for The clustering value is calculated by the following formula:

[0046] ,

[0047] At the end of each training phase, the historical candidate patterns retained in the memory storage unit are updated according to their frequency of occurrence. This allows the high-frequency patterns to be screened out and used as a candidate pool for subsequent training and prediction.

[0048] After the pre-training phase, the predefined data model can be obtained. Entering the parameter fine-tuning phase, the model parameters are fine-tuned using professional time series data sets for specific downstream tasks, a task-oriented training environment is established, and the performance of the model on downstream tasks is optimized.

[0049] The power time series data anomaly detection model includes a wavelet decomposition module, an encoder module, and a decoder module connected in sequence. In order to make up for the lack of understanding of time series information by the large model, decomposing the time series data into components with obvious periodic properties is a natural and effective method. Figure 1 As shown, the input power time series data is first decomposed by the wavelet decomposition module to obtain component sequences of different period scales. This embodiment uses Daubechies Wavelet to decompose the input sequence Decompose. Assume the input sequence is ,filter and The length is ,but:

[0050] The first low frequency component : ,

[0051] The first high frequency component : .

[0052] The first low-frequency component Repeat the above process as the input sequence until the required number of decompositions is obtained. ,in , that is, the input sequence length is , the dimension of each vector is , Represents the input sequence The new feature sequence is obtained by decomposition. The processing steps for low-frequency and high-frequency components are then consistent. To simplify the description, Take this as an example to illustrate.

[0053] First, the attention weight of each time step is calculated by the attention score function :

[0054] ,

[0055] in, is a component sequence At time step The feature vector of is a trainable weight vector.

[0056] The feature vector of each time step According to the attention weight Perform weighted summation to obtain the aggregated representation :

[0057] ,

[0058] Then, through a linear layer Project to the specified output dimension :

[0059] ,

[0060] in are the weights of the linear layer, is the bias vector. Finally, by learning the weight vector To adjust the importance of each time step, the variable length sequence can be mapped into a fixed length representation, thereby unifying the input in the subsequent network.

[0061] After completing the wavelet decomposition and length alignment of the original input sequence, the input is split into partially overlapping blocks using patch splitting, and the input encoder module encodes them into word vector representations of equal length. Treating a time series as a word vector of a pre-trained model can solve the problem of insufficient information density of the original time series data points.

[0062] In order to enable the model to further understand concepts such as historical patterns, periodic properties, and distribution changes in time series data, this embodiment uses a differentiable dynamic time warping algorithm in the decoder module to calculate the dynamic time warping distance between the output of the encoder module and each historical alternative mode in the memory storage unit, and then calculates the contribution score of each historical alternative mode through a normalization method, and uses an attention mechanism to aggregate and obtain the feature representation of the power time series data at each periodic scale, and finally outputs it through an inverse wavelet transform.

[0063] Assume that the output sequence of the encoder is , the historical memory module is represented as , is the number of historical alternative modes, where each historical alternative mode is a sequence .

[0064] First, the dynamic time warping distance between the output of the encoder module and each historical candidate mode is approximately differentiable to solve the problem that the DTW formula is not differentiable and cannot transfer gradients during training:

[0065] ,

[0066] ,

[0067] in, Output for encoder module With historical alternative mode The dynamic time warping distance between is the length of the time series, The dimension of the encoder module output, historical alternative mode With encoder module output The shape and dimensions are consistent. is the smoothing parameter, for and The Euclidean distance of . is the final calculation result of the dynamic time warping distance, through Iterate the calculation until , .

[0068] Then, the normalized weight calculation method is used to obtain the contribution score of each historical pattern to the forecast sequence:

[0069] ,

[0070] These normalized contribution scores are used as attention weights to weight and aggregate the data of each historical mode to obtain the feature representation of the power time series data at each cycle scale:

[0071] ,

[0072] Finally, according to the wavelet transform and The sequence combination order and patch segmentation method are used to synthesize the output frequency domain sequences into prediction vector outputs through inverse wavelet transform, and the gradient is calculated through L1 and L2 loss functions and then back-propagated to train the model parameters.

[0073] The above method obtains time series frequency domain information through wavelet decomposition, aggregates it into equal-length sequences using the self-attention mechanism, introduces a self-supervised memory update module to obtain typical historical patterns, and calculates the attention coefficient using the differentiable dynamic time warping algorithm to obtain the final sequence representation, surpassing the limitations of the previous large model's ability to understand time series prediction.

[0074] In order to verify the effectiveness of the above method, this embodiment conducts experiments on power time series data prediction and power time series data anomaly detection. Table 1 shows the experimental results of the power data set ETT, which consists of two hourly data sets (ETTh) and two 15-minute data sets (ETTm). Each of them contains the load characteristics of seven types of petroleum and power transformers from July 2016 to July 2018. The selected baselines include GPT4TS, Informer, Reformer, NonstationaryTransformer, and the evaluation indicators include mean absolute error MAE and mean square error MSE. It can be seen from Table 1 that the method proposed in this embodiment (Ours) has the smallest average value of the mean absolute error MAE and the average value of the mean square error MSE on the four data sets, indicating that the degree of fit to the target has a high degree of accuracy.

[0075] Table 1 Experimental results of power time series data prediction

[0076]

[0077] Table 2 shows the results of anomaly detection of power time series data, using F1-score as the evaluation index. As can be seen from Table 2, the method (Ours) proposed in this embodiment has the highest F1-score average value, indicating that the recognition of the target has a high accuracy.

[0078] Table 2 Anomaly detection of power time series data

[0079]

[0080] Example 2

[0081] This embodiment provides an electronic device, including a memory and a processor, wherein the processor is used to execute a program stored in the memory, wherein the program includes a number of instructions and can execute all or part of the steps of the method described in Embodiment 1. The memory includes a computer-readable storage medium, which can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0082] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A method for detecting anomalies in power time series data based on a large model, characterized in that: The following steps are involved: Obtain the power time series data to be detected, input the trained power time series data anomaly detection model, and output the detection result; wherein the training process of the power time series data anomaly detection model includes a pre-training stage and a fine-tuning stage, wherein the pre-training stage performs self-supervised training based on a pre-acquired general time series data set, and creates a memory storage unit in the pre-training stage for storing historical candidate modes, wherein the historical candidate modes are generated based on general time series data in the general time series data set whose cosine similarity is higher than a set threshold; and the fine-tuning stage further performs model parameter fine-tuning through the pre-acquired power time series data set; The power time series data anomaly detection model includes a wavelet decomposition module, an encoder module and a decoder module connected in sequence. The wavelet decomposition module first performs wavelet decomposition on the power time series data to obtain component sequences of different period scales, and then converts each component sequence into a feature representation of equal length through multi-head self-attention, and divides it into partially overlapping slices and inputs it into the encoder module for encoding; the decoder module uses an improved dynamic time warping algorithm to differentiably calculate the dynamic time warping distance between the output of the encoder module and each historical candidate mode in the memory storage unit, and calculates the contribution score of each historical candidate mode based on this, aggregates to obtain the feature representation of the power time series data at each period scale, and obtains the final output through inverse wavelet transform; The calculation formula of the dynamic time warping distance between the output of the encoder module and each historical candidate mode is as follows: ， ， in, Output for encoder module With historical alternative mode The dynamic time warping distance between , , each historical alternative mode , is the number of historical alternative models, Indicates the time, is the length of the time series, Output for encoder module and historical alternative models The dimension of is the smoothing parameter, for The iterative calculation formula is: for and The Euclidean distance of The characteristics of power time series data at each cycle scale are as follows: ， ， in, is the number of historical alternative models, Alternative mode for history contribution score.

2. According to claim 1, a method for detecting anomalies in power time series data based on a large model is characterized in that: The power time series data is decomposed by Dobesi wavelet.

3. The method for detecting anomalies in power time series data based on a large model according to claim 1, characterized in that: The specific process of converting each component sequence into a feature representation of equal length is as follows: Calculate the attention weight of each component sequence at each time step, perform weighted summation on the component sequence at each time step, and obtain the aggregated representation of each component sequence; The aggregate representation of each component sequence is projected to the specified output dimension through a linear layer to complete the length alignment.

4. The method for detecting anomalies in power time series data based on a large model according to claim 3, characterized in that: component sequence At time step The attention weight The calculation formula is as follows: ， component sequence Aggregate representation of The calculation formula is as follows: ， in, is a component sequence At time step The characteristic vector of is a trainable weight vector, is the length of the component sequence.

5. The method for detecting anomalies in power time series data based on a large model according to claim 1, characterized in that: The memory storage unit is dynamically updated according to the occurrence frequency of the historical candidate patterns stored therein.

6. The method for detecting anomalies in power time series data based on a large model according to claim 1, characterized in that: The clustering values ​​of the common time series data whose cosine similarity matrix is ​​higher than a set threshold are calculated and stored in the memory storage unit as historical candidate modes.

7. An electronic device comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images