A power data anomaly detection method based on federated learning

Abstract

The application discloses a power data anomaly detection method based on federal learning, and steps are as follows: (1) fault detection data preprocessing: a multi-dimensional time series client is established; data preprocessing is carried out; a time window is constructed; (2) based on a time domain convolution neural network and a self-attention mechanism, a local model is established: model input initialization; single-layer fusion encoder operation; decoding operation; a model loss function is constructed; abnormal scoring is carried out; local model parameters are updated; (3) a server end adopts a federal learning model to realize joint training: a model loss function is constructed; local model parameters are updated; after the local model parameters are updated through the above process, the local model parameters are uploaded to a terminal server, and a global model is generated by aggregation. The application can simultaneously consider data local dependency and data global dependency, can better extract time sequence reconstruction information in a time sequence, and thus improves the effectiveness of multi-dimensional time series anomaly detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data recognition technology, power system and its automation, and particularly relates to a power data anomaly detection method based on federated learning. BACKGROUND

[0002] The real-time operation data of the power system has the potential ability to reflect the current operation state and future development trend of the power system. For the traditional power anomaly detection method, it must collect the historical power information of the electrical appliances to establish the reference power information sample. However, in order to establish an accurate reference power information sample, the historical power information collected by the traditional power anomaly detection method must consider the influence of season, time and temperature on each electrical appliance, resulting in a long and tedious process of collecting the historical power information of the electrical appliances.

[0003] With the rapid development of the intelligentization of the power system, the scale of embedding various sensors in the power system is continuously expanding, which makes the data collected by the perception layer more detailed and the data to be processed sharply increases. According to incomplete statistics, the power grid business data collected by a single city every day can reach PB level. Overall, the real-time operation data of the power system has the characteristics of multiple data collection devices, high collection frequency, large data size and complex data types. The data collected is typical time series data. Making full use of the power time series data and using appropriate technology for anomaly detection can timely find the faults existing in the power system and provide decision-making and auxiliary support for efficient and safe operation of the power system.

[0004] The prior art close to the present application includes a patent with the application number 20130416182.6, which discloses a power anomaly fluctuation detection and prediction calculation method, which is based on mutation point analysis and hidden Markov model integrated algorithm for real-time detection of power energy consumption, and can predict and calculate the reasonable interval range of the power energy consumption interval at future time; a patent with the application number 202110976109.9 discloses a power distribution network abnormal event identification method and system based on power disturbance data, which extracts four or more features of power disturbance data, and realizes rapid identification, classification and tracing of abnormal events through comparison between the features; a patent with the application number 201910604895.2 discloses a large-scale power anomaly data detection method and system based on machine learning, which uses a multi-layer perception neural network model to realize effective detection of abnormal data. The above patents discuss the reasons for power data anomalies, including power fluctuations caused by various events, sharp increase in data volume, and faster data transmission and processing speed, and propose detection methods for different types of data such as voltage and current, power factor, and switching signal. However, the existing fault detection methods directly use the acquired data for detection without joint learning between different enterprises.

[0005] Furthermore, unsupervised multidimensional time series anomaly detection methods can be broadly categorized into two types: reconstruction-based anomaly detection methods and prediction-based anomaly detection methods. Reconstruction-based anomaly detection methods cannot simultaneously and effectively learn the representations between indicators and temporal dependencies, resulting in insufficient reconstruction of multidimensional time series data, and the reconstruction error has an insignificant effect as anomaly scoring. Prediction-based anomaly detection methods, due to the unpredictability of complex multidimensional time series, suffer from a high false positive rate.

[0006] Therefore, there is an urgent need for a new method for detecting power data anomalies to overcome the shortcomings of existing technologies. Summary of the Invention

[0007] Purpose of the invention: The purpose of this invention is to provide a power data anomaly detection method based on federated learning. By introducing a self-attention mechanism into a temporal convolutional network, the method simultaneously captures local and global features of the data, and constructs an information fusion module to fuse these two types of features. After the client-side data is learned, the parameters are uploaded to the server, and the server then distributes the model to each local terminal, capturing more comprehensive temporal reconstruction information, thereby achieving more effective multidimensional time series anomaly detection.

[0008] Technical solution: The present invention provides a power data anomaly detection method based on federated learning, comprising the following steps:

[0009] (1) Fault detection data preprocessing: Establish a multi-dimensional time series client; data preprocessing; construct time windows.

[0010] (1.1) Establish a multidimensional time series client.

[0011] A multidimensional time series is a timestamp sequence of length T formed by sampling multidimensional time variables at equal time intervals, where X = (x1, x2, ..., x...). t Time observation point x t The data is multidimensional data obtained at timestamp t, x t ∈R m ,t∈{1,2,…,T},m is the dimension of the data, and the multidimensional data reflects the device status information from different levels; in the client, given a time series X, the corresponding abnormal label sequence needs to be given. At this point, the client for multidimensional time series anomaly detection is established.

[0012] (1.2) Data preprocessing.

[0013] For multidimensional time series data from clients, maximum and minimum value standardization is used to map the data to the range [0, 1]. The standardization function is as follows:

[0014]

[0015] where x t is the collected feature parameter of the power distribution station at time t, x max is the maximum value in the collected sample parameters, x min is the minimum value in the collected sample parameters, is the standardized result of the collected feature parameter of the power distribution station at time t, and a is a very small constant vector set to avoid a denominator of 0.

[0016] (1.3) Construct a time window.

[0017] Considering the dependence of the time observation point x t and the historical time point, a time window with a length of N is constructed:

[0018] w t = (x t-N+1 , x t-N+2 ,..., x t ) (2)

[0019] In the formula, x t is the collected feature parameter of the power distribution station at time t, x t-N+1 is the starting position of the time window.

[0020] In order to avoid the influence of short time window on the normal pattern reconstruction of the model sequence, for the time observation point x t , since there is not enough information to judge its abnormal degree, it is regarded as a normal time point, and the multi-dimensional time sequence X is converted into a sliding window sequence W as the model input.

[0021] (2) Based on the time domain convolution neural network and the self-attention mechanism, a local model is established: model input initialization; single-layer fusion encoder operation; decoding operation; construction of model loss function; abnormal score; update of local model parameters.

[0022] (2.1) Model input initialization.

[0023] The stacking of multi-layer fusion encoders is conducive to learning deeper potential time sequence correlation; assuming that there are L layers of fusion encoders, the input time sequence window w ∈ R m , the calculation process of the lth layer and the initial input of the model are formalized as follows:

[0024]

[0025] In the formula, are respectively two inputs of the fusion encoder, representing the input of the (l+1)th layer fusion encoder, where l ∈ {1, 2,..., L}, dmodel dimension of the feature space vector, is is the value after fusion encoder processing, FusionEncoder represents the fusion encoder, and the LayerNorm function represents normalization, is the value of the fusion encoder input decoder feedforward network, Convld represents convolution operation, and FeedForward represents feedforward network.

[0026] (2.2) Single-layer fusion encoder operation.

[0027] The time sliding window W is taken as 2 identical inputs Z1, Z2 of the model, at this time, there are 2 inputs and 2 outputs for the fusion encoder; the attention weight of each time point compared with other time points is obtained through matrix multiplication by the time domain convolutional neural network and the self-attention mechanism; the input of the lth layer encoder is obtained by formula (3), and then information extraction is performed by the time domain convolutional network and the self-attention, and information fusion of the extracted features is performed by the cross-stitch unit; the specific formula is shown in (4):

[0028]

[0029] In the formula, Q, K, and V represent query, key value, and value in self-attention calculation, respectively, and are obtained by linear transformation of the input Z2 and 3 linear transformation matrices W Q , W K , and W V ; M represents the self-attention matrix; the Softmax function represents normalization processing, and the TemporalBlock function represents time domain convolution operation; I1 is the hidden layer representation after adding the self-attention matrix information by the time domain convolution, I2 is the hidden layer representation after processing by the self-attention module by the time domain convolution, O1 and O2 are the results of Z1 and Z2 after the single-layer fusion encoder, and γ 11 , γ 12 , γ 21 , and γ 22 are weight parameters.

[0030] (2.3) Decoding operation.

[0031] The decoder decodes the two hidden layer features output by the Lth layer fusion encoder by two decoding parts; the first decoding part is an inverse time domain convolution layer, which is realized by replacing the dilated causal convolution in the time domain convolution residual block with a transposed convolution, and the second decoding part is composed of a single-layer feedforward neural network layer and a sigmoid function; the cross-stitch unit fuses the two kinds of feature representations which focus on extracting local data dependence and global data correlation.

[0032] (2.4) Constructing a model loss function.

[0033] The reconstruction error loss is as follows: the model loss function is as follows:

[0034] α×||w-C1||+β×||w-C2|| (5)

[0035] In the formula, α and β are hyperparameters, and α+β=1, C1 and C2 are the final output results of the decoder.

[0036] (2.5) Performing anomaly scoring.

[0037] The anomaly score is calculated by formula (5), and the higher the score, the greater the possibility of anomaly at the time point.

[0038] (2.6) Updating local model parameters.

[0039] The loss function of the kth participant can be obtained from formula (5), and then the model parameters θ of each local model are updated according to the Adam optimizer, as shown in formula (6), and formula (7) is the parameter update formula.

[0040]

[0041]

[0042] In the formula, m is the first moment estimation of the gradient, i.e., the mean of the gradient, v is the second moment estimation of the gradient, i.e., the biased variance of the gradient, g is the gradient, t represents the current iteration number of learning, ⊙ is a kind of multiplication of homothetic elements; γ1, γ2∈[0,1) is a set of hyperparameters and are the mean and biased variance of the corrected gradient; η is the learning rate; after updating the local model parameters through the above process, they are uploaded to the terminal server to generate a global model by aggregation.

[0043] (3) The server side adopts a federated learning model to realize joint training: constructing a model loss function; updating local model parameters; after updating the local model parameters through the above process, they are uploaded to the terminal server to generate a global model by aggregation.

[0044] (3.1) Constructing a model loss function.

[0045] The reconstruction error loss is as follows: the model loss function is as follows:

[0046]

[0047] In the formula, μ and are hyperparameters, and In the model test stage, the reconstructed representations D1 and D2 of the input w are obtained by the model.

[0048] (3.2) Update the local model parameters.

[0049] The loss function of the kth participant can be obtained from formula (8), and then the model parameters θ of each local model are updated according to the Adam optimizer, as shown in formula (9); formula (10) is the parameter update formula.

[0050]

[0051]

[0052] In the formula, m is the first moment estimation of the gradient, that is, the mean of the gradient, v is the second moment estimation of the gradient, that is, the biased variance of the gradient, g is the gradient, t represents the current learning iteration number, ⊙ is a kind of multiplication of the same element; γ1, γ2∈[0,1) is a set of hyperparameters and is the mean and biased variance of the corrected gradient; η is the learning rate; after updating the local model parameters through the above process, upload to the terminal server to aggregate to generate the global model.

[0053] (3.3) After updating the local model parameters through the above process, upload to the terminal server to aggregate to generate the global model.

[0054] Aggregate the local model parameters generated in the above steps on the trusted server side, and update the parameters of the local model related layers As a feature, a Resnet residual neural network is used to extract features, and then the corresponding layer parameters of the global model are adaptively generated; the number of input layer neurons of Resnet is consistent with the number of neurons of the corresponding layer of the local model, and the number of output layer neurons is U, which corresponds to the number of neurons of the global model related layer, and the local training iteration number of the local model is local ep , and the communication number is R; after the local model is fully trained using local data, the client uploads the model parameters to the terminal server, and when the local model parameters of each layer reach a certain number, the Resnet is trained to extract features and generate the global model, that is, as shown in formula (11):

[0055]

[0056] Where ResNet(·) represents the ResNet network. is the global neural network parameter, and l represents the number of layers.

[0057] A computer storage medium, which stores a computer program, the computer program is executed by a processor to implement the power data anomaly detection method based on federated learning.

[0058] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the power data anomaly detection method based on federated learning when executing the computer program.

[0059] Beneficial effects: compared with the prior art, the present application has the following advantages: the present application can simultaneously consider data local dependence and data global dependence to improve the effectiveness of multi-dimensional time series anomaly detection, and the introduction of the federated learning algorithm improves the prediction effect of the model using big data and protects data privacy. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a time window schematic diagram;

[0061] Figure 2 is a local model schematic diagram;

[0062] Figure 3 is a single-layer fusion encoder schematic diagram;

[0063] Figure 4 is a federated learning framework based on ResNet schematic diagram;

[0064] Figure 5 is a step flowchart of the method of the present application. DETAILED DESCRIPTION

[0065] The technical solutions of the present application will be further described below in combination with the drawings.

[0066] The application is based on existing federal learning, time domain convolutional neural network, self-attention mechanism and other artificial intelligence technologies, and proposes a power data anomaly detection method based on federal learning. The method fully considers the local data dependence and global data correlation of the time series by using the time domain convolution network and the self-attention mechanism, and then fuses the information of the time series model parameters to the features through the cross-stitch unit and obtains the anomaly scores of each timestamp. Whether the abnormal data appears is judged by the size of the anomaly score. In addition, federal learning has a prerequisite: the data of each participant has a certain correlation, which includes target tasks, user IDs, feature variables and the like. According to the correlation requirement of the data, the parameters used in the embodiment also have a certain correlation, so a model with good generalization performance can be trained. Under the federal learning framework, the local client participating in joint training can well protect the privacy of each participant. The existing federal learning framework uses the federal average algorithm on the server side, and in the application, the specific one is the ResNet residual network algorithm. The ResNet residual network can solve the network degradation capability and some gradient problems caused by the deepening of the network.

[0067] As shown in Figure 5 , a power data anomaly detection method based on federal learning, the steps are as follows:

[0068] S1, fault prediction data preprocessing

[0069] S1.1, establish a multi-dimensional time series client. A multi-dimensional time series can be regarded as a time stamp sequence with a length of T formed by sampling multi-dimensional time variables at equal time intervals, X=(x1,x2,…,x t ). A time observation point x t is the multi-dimensional data obtained at the time stamp t, x t ∈R m ,t∈{1,2,…,T}, m is the total dimension number. In this paper, m is set to 21, and the multi-dimensional data all reflect the device state information from different aspects, such as x1 for high-voltage voltage monitoring, x2 for high-voltage current monitoring, x3 for high-voltage temperature monitoring, x4 for high-voltage humidity monitoring, x5 for high-voltage smoke monitoring, x6 for high-voltage communication monitoring, x7 for high-voltage circuit breaker monitoring, x8 for high-voltage joint monitoring, x9 for high-voltage screw position monitoring, x 10 for high-voltage fuse monitoring; x 11 for low-voltage voltage detection, x 12 for low-voltage current monitoring, x 13 for low-voltage temperature monitoring, x 14 for low-voltage humidity monitoring, x 15 for low-voltage smoke monitoring, x 16 for low-voltage communication monitoring, x 17 for low-voltage communication monitoring circuit breaker monitoring, x18 For low-voltage capacitor monitoring, x 19 For low-voltage screw position monitoring, x 20 For low-voltage knife switch position monitoring, etc. Multidimensional time series anomaly detection is used to determine whether a time observation point x t is abnormal, given a time series X, the corresponding abnormal label sequence Y = (y1, y2, …, y T ), y t ∈{0, 1}, y t = 0 indicates that the time point at time t is normal, and y t = 1 indicates that the time point at time t is abnormal.

[0070] S1.2, data preprocessing. The data collected in the present application is divided into data sets in step S1.1. Generally speaking, when the input data is close to the "0" average value, the learning efficiency of the deep learning algorithm is best. For the collected data, the maximum and minimum standardization is used to map the data to [0, 1], and the standardization function is as follows:

[0071]

[0072] Where x t is the relevant feature parameter of the power distribution station collected at time t, x max is the maximum value of the collected sample parameters, x min is the minimum value of the collected sample parameters, is the standardization result of the relevant feature parameter of the power distribution station collected at time t, and α is a very small constant vector set to avoid the denominator being 0.

[0073] S1.3, construct a time window. Considering the dependence of the time observation point x t and the historical time point, a time window with a length of N is constructed as shown in Figure 1

[0074] w t = (x t-N+1 , x t-N+2 ,..., x t ) (2)

[0075] In order to avoid the influence of short time sequence window on the reconstruction of model sequence normal mode, for the time observation point x t ​Since there is not enough information to judge the degree of abnormality, it is regarded as a normal time point, and the multidimensional time sequence X is converted into a sliding window sequence W as the model input. When the time sequence window is too small, the input time sequence cannot well represent the local context information, but when the time sequence window is too large, subtle local anomalies are more likely to be hidden in a large number of normal time points, which makes the anomaly detection effect decline. Multiple experiments show that when the time sequence window size is about 100, the model performs best in various performance indicators,

[0076] S2, based on a time domain convolutional neural network and a self-attention mechanism, a local model is established.

[0077] The local model is as shown in Figure 2 Based on the data preprocessing in step S1, in step S2, the application needs to extract features from the data. For real-time operation data of the power system, feature extraction needs to be performed convolution operation. The receptive field of the ordinary convolutional neural network is relatively small, which is not suitable for the parameters required by the application, and cannot capture the information dependence of the multidimensional time sequence in locality and globality at the same time. In this embodiment, a self-attention mechanism is added to the time domain convolutional network. The time domain convolution module and the self-attention module of the fusion encoder are used to capture the data dependence of the multidimensional time sequence in locality and globality respectively, and a cross-stitch unit is used to extract and fuse the features. The stacking of the multi-layer fusion encoder is conducive to learning deeper potential time sequence correlation.

[0078] S2.1, model input initialization.

[0079] The stacking of the multi-layer fusion encoder is conducive to learning deeper potential time sequence correlation. Assuming that there are L layers of fusion encoders, a single-layer model is as shown in Figure 3 The input length of the time sequence window w is N, and w e R m The calculation process of the lth layer and the initial input of the model can be formalized as follows:

[0080]

[0081] are respectively the two inputs of the fusion encoder, and represent the input of the (l+1)th fusion encoder, where l e {1,2,…,L}, d model represents the dimension of the feature space vector, is the value after the fusion encoder processing, FusionEncoder represents the fusion encoder, and LayerNorm function represents normalization, is the value of the fusion encoder input decoder feedforward network, Convld represents convolution operation, and FeedForward represents feedforward network.

[0082] S2.2, Single-layer fusion encoder operation. The deep learning attention mechanism is inspired by the information processing mechanism of human beings in cognitive science. Due to the limitation of information processing capacity, it is necessary to selectively focus on a part of the complete information while ignoring other parts of the information. This mechanism greatly improves the efficiency of human information processing. In order to obtain the information dependence of multi-dimensional time series in locality and globality, the time sliding window W is taken as the 2 same inputs Z1, Z2 of the model. At this time, there are 2 inputs and 2 outputs for the fusion encoder. The attention weight of each time point compared with other time points is obtained by time domain convolutional neural network and self-attention mechanism. The input of the lth layer encoder is obtained by formula (3), and then the information is extracted by time domain convolutional network and self-attention, and the information fusion of extracted features is performed by cross-stitch unit. The specific formula is shown in formula (4).

[0083]

[0084] Where Q, K, V represent query, key value and value in self-attention calculation, respectively, and are obtained by linear transformation of the input Z2 and 3 linear transformation matrices W Q , W K , W V . M represents the self-attention matrix. Softmax function represents normalization processing, and TemporalBlock function represents time domain convolution operation. I1 is the hidden layer representation after adding the self-attention matrix information of the time domain convolution, I2 is the hidden layer representation after adding the self-attention module processing of the time domain convolution, O1 and O2 are the results of Z1 and Z2 through the single-layer fusion encoder. γ 11 , γ 12 , γ 21 , γ 22 are weight parameters.

[0085] S2.3, decoding operation. The decoder decodes the two hidden layer features output by the Lth layer fusion encoder by two decoding parts. The first decoding part is an inverse time domain convolution layer, which is realized by replacing the dilated causal convolution in the time domain convolution residual block with a transposed convolution. The second decoding part is composed of a single-layer feedforward neural network layer and a sigmoid function. The cross-stitch unit can fuse the two kinds of feature representations which focus on extracting local data dependence and global data correlation.

[0086]

[0087] S2.4, constructing model loss function. Reconstruction error loss, the model loss function is

[0088] ​a x ||w-C1||+b x ||w-C2|| (5)

[0089] a and b are hyperparameters that can be set, and a+b=1, here a and b are set to 0.5 and 0.5 respectively, and C1 and C2 are the final output results of the decoder

[0090] S2.5, Abnormal score. Calculate the abnormal score by (5), the higher the score, the greater the possibility of abnormality at this time point.

[0091] S2.6, Update local model parameters. The loss function of the kth participant can be obtained from formula (5), and then the model parameters theta of each local model are updated according to the Adam optimizer, as shown in formula (6). Formula (7) is the parameter update formula.

[0092]

[0093]

[0094] Where m is the first moment estimate of the gradient, i.e. the mean of the gradient, v is the second moment estimate of the gradient, i.e. the biased variance of the gradient, g is the gradient, t represents the current learning iteration number, ⊙ is a kind of multiplication of the same element. Gamma1, gamma2 [0, 1) is a set of hyperparameters And The mean and biased variance of the corrected gradient. The learning rate is eta. After updating the local model parameters through the above process, upload to the terminal server to generate a global model. As shown in step S2.2.

[0095] S3, the server side uses a federated learning model to realize joint training.

[0096] Aggregated global model. The terminal server aggregates the local model parameters uploaded in step S2.1. Unlike the traditional federated average aggregation algorithm, only the local model parameters are averaged. The application uses a ResNet residual network-based aggregation module, as shown in Figure 4 , to aggregate the client model parameters to generate a global model. This aggregation module can automatically aggregate client parameters without the need to establish additional aggregation rules.

[0097] S3.1, Construct model loss function. Reconstruction error loss, model loss function is

[0098]

[0099] Mu and are hyperparameters, and In the model test stage, the reconstruction representation D1 and D2 of the input w are obtained through the model.

[0100] S3.2, update the local model parameters. The loss function of the kth participant can be derived from equation (8), and then the model parameters θ of each local model are updated according to the Adam optimizer, as shown in equation (9). Equation (10) is the parameter update formula.

[0101]

[0102]

[0103] where m is the first moment estimate of the gradient, i.e., the mean of the gradient, v is the second moment estimate of the gradient, i.e., the biased variance of the gradient, g is the gradient, t represents the current iteration number of learning, is a kind of multiplication of the same element. γ1, γ2∈ [0, 1) is a set of hyperparameters and is the mean and biased variance of the corrected gradient. η is the learning rate. After updating the local model parameters through the above process, upload to the terminal server to generate a global model by aggregation. As shown in step S2.2.

[0104] S3.3, after updating the local model parameters through the above process, upload to the terminal server to generate a global model by aggregation. Aggregate the local model parameters generated in the above steps on the trusted server side, and update the parameters of the local model related layers As a feature, a Resnet residual neural network is used to extract features, and then the corresponding layer parameters of the global model are adaptively generated, as shown in Figure 4 The number of input layer neurons of Resnet is consistent with the number of neurons of the corresponding layer of the local model, and the number of output layer neurons is U, corresponding to the number of neurons of the related layer of the global model, and the local training iteration number of the local model is local ep , and the number of communications is R. After the local model is fully trained using local data, the client uploads the model parameters to the terminal server, and when the local model parameters of each layer reach a certain number, the Resnet is trained to extract features and generate a global model, as shown in equation (8):

[0105]

[0106] where ResNet(·) represents the ResNet network in the present application. is the global neural network parameter, and l represents the number of layers. By implementing the present framework, both data local dependence and data global dependence can be considered to improve the effectiveness of multi-dimensional time series anomaly detection.

[0107] The effect finally achieved in the example is that the experiment obtains a training set and a test set according to original division of a data set, and selects 25% of the training set as a validation set. According to comparative experimental results of the seven multi-dimensional time series anomaly detection methods on the four multi-dimensional time series data sets, the remaining performance indicators of the algorithm on the four data sets are better than those of the most popular multi-dimensional time series anomaly detection method, proving that the algorithm helps the model to learn better time dependence, metric dependence, data local and data global features.

Claims

1. A power data anomaly detection method based on federated learning, characterized in that, Comprise the following steps: (1) fault detection data preprocessing: establish a multi-dimensional time series client; Data preprocessing; build a time window; (2) based on time domain convolutional neural network, self-attention mechanism, establish local model: model input initialization; single layer fusion encoder operation; decoding operation; construct model loss function; perform abnormal scoring; Update local model parameters; specifically including: (2.1) model input initialization; The stacking of multi-layer fusion encoders facilitates learning deeper latent temporal sequence dependencies; assuming there are L layers of fusion encoders, the input length of the time series window is N The calculation process for the l-th layer and the initial input of the model is formalized as follows in equation form: (3) wherein, , are two inputs of the fusion encoder, respectively, represent the input of the (l+1)-th layer fusion encoder, where l ∈ {1, 2, …, L}, represents the dimension of the feature space vector, is the value after the fusion encoder processing, represents the fusion encoder, and the LayerNorm function represents normalization, is the value of the fusion encoder input decoder feedforward network, Convld represents the convolution operation, and FeedForward represents the feedforward network; (2.2) single layer fusion encoder operation; a time sliding window W as 2 same inputs of the model , , at this time, there are 2 inputs and 2 outputs for the fusion encoder respectively; the attention weight of each time point compared with other time points is obtained by matrix multiplication through the time domain convolutional neural network and the self-attention mechanism; the input of the lth layer encoder is obtained by formula (3), and then information extraction is carried out through the time domain convolutional network and the self-attention, and information fusion of the extracted features is carried out through the cross-stitch unit; the specific formula is shown in (4): (4) In the formula, Q, K, V respectively represent query, key value and value in self-attention calculation, which are input of self-attention module And three linear transformation matrices W Q , W K , W V Linear transformation is carried out; M represents a self-attention matrix; the Softmax function represents normalization processing, and the TemporalBlock function represents a time domain convolution operation; Is a hidden layer representation after adding the self-attention matrix information to the time domain convolution, Is a hidden layer representation after adding the self-attention module processing to the time domain convolution, And Is And The result after single-layer fusion encoder; , , , Weight parameter; (2.3) decoding operation; The decoder is composed of two decoding parts which respectively decode the two hidden layer features output by the Lth layer fusion encoder; the first decoding part is an inverse time domain convolutional layer, which is realized by replacing the dilated causal convolution in the time domain convolutional residual block with a transposed convolution, and the second decoding part is composed of a single layer of feedforward neural network layer and a function sigmoid; the cross-stitch unit fuses the two kinds of feature representations which focus on extracting local data dependence and global data correlation respectively; (3) the server side uses federated learning model to realize joint training: construct model loss function; update local model parameters; after updating the local model parameters through the above process, upload to the terminal server, aggregate to generate global model.

2. The power data anomaly detection method based on federated learning according to claim 1, characterized in that, Said step (1) is specifically: (1.1) establish a multi-dimensional time series client; A multi-dimensional time series is a time stamp sequence with length T formed by equally time-interval sampled multi-dimensional time variables, ; time observation points are multi-dimensional data acquired at time stamp t, m is the dimension of data, and the multi-dimensional data all reflect device state information from different aspects; in the client, given a time series X, the corresponding abnormal label sequence is given, and thus the client of the multi-dimensional time series anomaly detection is established. (1.2) data preprocessing; For the client's multi-dimensional time series data, the maximum and minimum standardization is adopted, and the data is mapped to between, the standardization function is as follows: (1) wherein, is the collected relevant feature parameter of the power distribution station at time t, is the maximum value in the collected sample parameters, is the minimum value in the collected sample parameters, is the standardization result of the collected relevant feature parameter of the power distribution station at time t, is a very small constant vector set to avoid the denominator being 0; (1.3) build a time window; Consider the time observation point And the dependence of historical time points, build a length of N time window: (2) In the formula, is the relevant feature parameter of the power distribution station collected at time t, is the starting position of the time window; To avoid the influence of short time window on the reconstruction of normal pattern of model sequence, for the time observation point t<N Since there is not enough information to judge its abnormal degree, it is regarded as a normal time point, and the multi-dimensional time sequence X is converted into a sliding window sequence W as the input of the model.

3. The power data anomaly detection method based on federated learning according to claim 1, characterized in that, Said step (2) further comprises: (2.4) construct model loss function; Reconstruction error loss, model loss function is: (5) wherein and are hyperparameters, and , and is the final output result of the decoder; (2.5) perform abnormal scoring; Calculate the abnormal score by formula (5), the higher the score, the greater the possibility of abnormality at that time point; (2.6) update local model parameters; From formula (5), we can obtain the first... The loss function of each participant is used to update the model parameters of each local model based on the Adam optimizer. Specifically, as shown in formula (6), formula (7) is the parameter update formula; (6) (7) In the formula, m is the first moment estimation of the gradient, that is, the mean of the gradient, v is the second moment estimation of the gradient, that is, the biased variance of the gradient, g is the gradient, t represents the current learning iteration number, , is a multiplication of the same element; is a set of hyperparameters and is the mean and biased variance of the modified gradient; is a learning rate; after updating the local model parameters through the above process, the local model parameters are uploaded to the terminal server to generate a global model through aggregation.

4. The power data anomaly detection method based on federated learning according to claim 1, characterized in that, Said step (3) is specifically: (3.1) construct model loss function; Reconstruction error loss, model loss function is: (8) wherein and are hyperparameters, and At the model testing phase, the reconstruction representation of the input w is obtained by the model and ; (3.2) update local model parameters; From formula (8), we can obtain the first... The loss function of each participant is used to update the model parameters of each local model based on the Adam optimizer. Specifically, as shown in formula (9); formula (10) is the parameter update formula; (9) (10) In the formula, m is the first moment estimation of the gradient, that is, the mean of the gradient, v is the second moment estimation of the gradient, that is, the biased variance of the gradient, g is the gradient, t represents the current learning iteration number, , is a multiplication of the same element; is a set of hyperparameters and is the mean and biased variance of the modified gradient; is a learning rate; after updating the local model parameters through the above process, the local model parameters are uploaded to the terminal server to generate a global model through aggregation. (3.3) after updating the local model parameters through the above process, upload to the terminal server, aggregate to generate global model; In the trusted server side aggregation of the local model parameters generated in the above steps, the parameters of the local model related layer are As a feature, the features are extracted by a Resnet residual neural network, and then the corresponding layer parameters of the global model are adaptively generated; the number of input layer neurons of Resnet is consistent with the number of neurons of the corresponding layer of the local model, the number of output layer neurons is U, which corresponds to the number of neurons of the global model related layer, and the number of local model local training iterations is , and the number of communications is R; after the local model is fully trained using local data, the client uploads the model parameters to the terminal server, and when the number of local model parameters of each layer reaches a certain number, the Resnet is trained to extract features and generate a global model, that is, as shown in formula (11): (11) wherein denotes a ResNet network; are global neural network parameters, and l denotes the number of layers.

5. A computer storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize the power data anomaly detection method based on federated learning in any one of claims 1-4.

6. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the power data anomaly detection method based on federated learning in any one of claims 1-4.

Images