A low-communication-cost smart meter fault diagnosis method based on single-round federated learning

Abstract

The present invention discloses a low-communication-cost smart meter fault diagnosis method based on single-round federated learning. Local model training is performed at each smart meter node, and each node uploads the local model parameters and the mean and variance recorded in the local model of each batch normalization layer after training to a central server. The central server generates a data set corresponding to each node by aligning the given fault classification and the statistics of the batch normalization layer. In addition, the present invention further improves the accuracy and robustness of the global model by clipping, mixing and generating soft labels for the generated data set. Finally, the generated data set and soft labels are input into the global model for training to obtain the final federated learning global fault diagnosis model. This method improves the accuracy and robustness of the model while protecting data privacy and reducing communication costs. Compared with other federated learning methods, the fault diagnosis performance is significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fault detection of smart meters in power systems, and in particular to a low-communication-cost smart meter fault diagnosis method based on single-round federated learning. Background Art

[0002] In view of the fault diagnosis problem of smart meters in power systems, traditional fault diagnosis models are usually deployed in the metering system operation center, and fault diagnosis is performed through data collected by smart meters, metering point monitoring data, meter reading data and other traditional operation methods. In the era of big data and the Internet of Things, smart devices are gradually developing towards high-performance computing and high intelligence. Using big data to build machine learning models, build smart meter data portraits and mine data information has become a hot topic in the current data research field. However, information security and data privacy issues have attracted great attention from researchers. This is mainly due to the contradiction between the data owner's demand for data protection and the expectation to obtain high-performance models and combine data. In order to protect the privacy of smart meter end nodes, existing technologies have set up microprocessors on the edge side of smart meters in power grid smart meters, trained local models, and exchanged model parameters through multiple rounds of communication with the central server. The central server performs multiple rounds of aggregation, thus realizing a smart meter diagnosis method based on federated learning that protects the privacy of smart meters. However, this method requires multiple rounds of federated learning communication, and the communication cost is high. In order to further solve the problem of high communication cost of traditional smart meter fault diagnosis methods based on federated learning, the present invention proposes a low communication cost smart meter fault diagnosis method based on single-round federated learning. DETAILED DESCRIPTION

[0003] The purpose of the present invention is to address the deficiencies of the prior art and to provide a low-communication cost smart meter fault diagnosis method based on single-round federated learning.

[0004] According to a first aspect of the present invention, a low-communication-cost smart meter fault diagnosis method based on single-round federated learning is provided, the method comprising the following steps:

[0005] S1: The central server determines the global fault diagnosis model structure, initializes the parameters of the global model, and sends it to each smart meter node;

[0006] S2: Each smart meter node locally trains the received initial global model using local data until all nodes complete the training of the local fault diagnosis model and obtain local model parameters;

[0007] S3: Each smart meter node uploads the local model parameters, as well as the mean and variance statistics recorded in the local model of each batch normalization layer after training to the central server;

[0008] S4: In the central server, the corresponding data set of each node is generated by aligning the given fault classification and the statistics of the batch normalization layer, and using the local model parameters uploaded by each smart meter node;

[0009] S5: Crop, mix and generate soft labels for the generated dataset;

[0010] S6: The central server inputs the generated data set and soft labels into the global model for training, and sends the trained global model parameters to each smart meter node to update the local model parameters;

[0011] S7: Each smart meter node inputs the real-time collected data into the latest local model to obtain the fault diagnosis result.

[0012] Furthermore, the local model deployed by the smart meter node adopts a ResNet network structure, including an initial convolutional layer and several residual modules. The input data x is input to the residual module, and the output results after the first 3×3 convolutional layer, the first batch normalization layer, the first activation function layer, the second 3×3 convolutional layer, and the second batch normalization layer are added to the input data x and then passed through the second activation function layer to obtain the output data of the residual module.

[0013] Furthermore, the total number of smart meter nodes is N, and the data sets on each smart meter node are respectively:

[0014]

[0015] Among them, T k is the data set on the k node, |T k | is the total number of samples on k nodes; is the sample data vector on the k node, For the corresponding x k The sample label vector of ;

[0016] The data sets corresponding to each node generated by the central server Denoted as:

[0017]

[0018] in, is the dataset generated by the model parameters uploaded by k nodes to the central server; is the sample data vector corresponding to the generated k nodes, For the corresponding Soft label vector of ;

[0019] The original sample data of each node is recorded as x, and the corresponding sample label is recorded as y; the generated sample data corresponding to each node is recorded as The corresponding soft label is recorded as

[0020] Furthermore, on the central server, the global model training goal using the generated data corresponding to all nodes is:

[0021]

[0022] in, The input sample data is The global model parameters are The global model prediction result is yes With soft label The information cross entropy loss function between them, E is the expectation;

[0023] The goal of data generation is stated as:

[0024]

[0025] Among them, sup(·) is the supremum function, ∈ is the performance gap between the global model trained on the generated dataset and the original dataset; The input sample data is x, and the global model parameter is θ T The global model prediction result is yes The information cross entropy loss function between the sample label y; generate a data set by optimizing To achieve the optimization goal:

[0026]

[0027] Furthermore, the central server generates a data set corresponding to each node by aligning the given fault classification and the statistics of the batch normalization layer and using the local model parameters uploaded by each smart meter node, specifically:

[0028] Used to generate sample data corresponding to k nodes The target formula is as follows:

[0029]

[0030] in, is the trained local model uploaded by each node, τ is the information cross entropy loss function, and R is the regularization term. The formula is as follows:

[0031]

[0032] Among them, α tv ,α l ,αBN They are the weights of the three regular terms respectively;

[0033] Used to make the generated sample data not diverge;

[0034] Used to reduce the noise of generated data, for The i-th element of , β is the smoothing factor;

[0035] It is used to achieve the alignment of the statistics of the batch normalization layer. The formula is as follows:

[0036]

[0037] Where m represents the index of the batch normalization layer, and Respectively represent the mean and variance of the generated data on the mth batch normalization layer; and They represent the mean and variance recorded by the mth batch normalization layer in the local model after training; T represents the local original dataset.

[0038] Furthermore, when the central server trains the global model, a block optimization technique is used to update only part of the data blocks in each iteration to prevent overfitting.

[0039] Furthermore, S5 is specifically: using the pre-generated soft label method, that is, randomly exchanging the generated data vector Part of the data vector in the , to obtain a new generated data vector with the same length Soft Label The formula is as follows:

[0040]

[0041] in, yes The i-th sample data, is the corresponding soft tag, Indicates that Input the trained local model to obtain the local model prediction result.

[0042] Further, the following objective function is used to generate the data vector Train the global model on Until convergence:

[0043]

[0044] in, is the target loss function, is the global model to be trained, Refers to the mapping function of the global model to the input.

[0045] Furthermore, after the local model training of each smart meter node is completed, the local model parameters are compressed and then uploaded to the central server, specifically:

[0046] The local model parameter vector θ of smart meter node k k The i-th element of The compression value The calculation formula is:

[0047]

[0048] Where s is the compression factor, Q s (· is the compression function, sgn(· is the sign function, ||·|| is the modulus of the orientation quantity, ξ i (θ k ,) are independent random variables, defined as follows:

[0049] Let ρ∈[0,s) be an integer, so That is, [ρ / ,(ρ+1 / ] corresponds to The compression interval; at this time, ξ i (θ k ,) is defined as:

[0050]

[0051] Among them, p is the probability of taking the corresponding value,

[0052] According to a second aspect of the present invention, a low-communication-cost smart meter fault diagnosis device based on single-round federated learning is provided, comprising a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement the above-mentioned low-communication-cost smart meter fault diagnosis method based on single-round federated learning.

[0053] The beneficial effects of the present invention are as follows: the present invention proposes a low-communication-cost smart meter fault diagnosis method based on single-round federated learning, performs local model training at each smart meter node, and uploads local model parameters and the mean and variance recorded in the local model of each batch normalization layer after training to the central server. The central server generates data sets corresponding to each node by aligning the given fault classification and statistics of the batch normalization layer, and determines the amount of data generated by the model according to the amount of original data of each node. In addition, the present invention further improves the accuracy and robustness of the global model by trimming, mixing and generating soft labels for the generated data sets. Finally, the generated data sets and soft labels are input into the global model for training to obtain the final federated learning global fault diagnosis model. This method improves the accuracy and robustness of the model while protecting data privacy and reducing communication costs. Compared with other federated learning methods, the fault diagnosis performance is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A flow chart of a low communication cost smart meter fault diagnosis method based on single-round federated learning provided by an exemplary embodiment;

[0056] Figure 2 A fault diagnosis model structure diagram provided for an exemplary embodiment;

[0057] Figure 3 A structural diagram of a low-communication-cost smart meter fault diagnosis device based on single-round federated learning provided as an exemplary embodiment. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0061] like Figure 1As shown, an embodiment of the present invention provides a low communication cost smart meter fault diagnosis method based on single-round federated learning. The method is based on a federated learning framework, with smart meter nodes as participants. Each participant locally deploys a fault diagnosis model with the same structure (hereinafter referred to as the local model, which is a classification neural network for fault type identification), and uses local data to train the local model. Specifically, the central server determines the structure of the global fault diagnosis model (hereinafter referred to as the global model), initializes the parameters of the global model, and sends it to each smart meter node.

[0062] Specifically, in this embodiment, the local model deployed in each smart meter node adopts a ResNet network structure, such as Figure 2 As shown in the figure, it is mainly composed of an initial convolution layer and multiple residual modules, wherein the residual module is the core component of ResNet. Each residual module includes the first 3×3 convolution layer, the first batch normalization layer (BN), the first activation function layer, the second 3×3 convolution layer, and the second batch normalization layer (BN) connected in sequence. The result of the input data x after the first 3×3 convolution layer, the first batch normalization layer, the first activation function layer, the second 3×3 convolution layer, and the second batch normalization layer is added to the input data x and then input into the second activation function layer. The output of the second activation function layer is used as the output data of the residual module. The first activation function layer and the second activation function layer can use the ReLU activation function.

[0063] Specifically, in this embodiment, the local data of each smart meter node is the voltage, current and other data collected by the smart meter within a certain period of time. The total number of smart meter nodes is N, and the data sets on each smart meter node are respectively:

[0064]

[0065] Among them, T k is the data set on smart meter node k, |T k | is the total number of samples on k nodes; x={x1,..., |x|} is the sample data vector, y=[y1,..., |y|} is the sample label vector; for k nodes, the sample data vector Corresponding to x k The sample label vector

[0066] The present invention hopes to generate a data set corresponding to each node on the central server when each smart meter node does not upload data but only uploads model parameters. The formula is as follows:

[0067]

[0068] in, It is the data set generated by the model parameters uploaded by k nodes to the central server; is the generated sample data vector, It corresponds to For k nodes, the generated sample data vector correspond Soft label vector

[0069] On the central server, the global model training goal using the generated data corresponding to all nodes is:

[0070]

[0071] in, The input sample is The global model parameters are When , the prediction result of the global model is is the global model prediction result With soft label The information cross entropy loss function between them, E is the expectation.

[0072] When generating data, the present invention hopes that the performance obtained by training the global model on the generated data set can have the smallest performance gap with that obtained by training on the original data set. Therefore, the task goal of data generation can be expressed as achieving:

[0073]

[0074] Among them, sup(·) is the supremum function, that is, taking the minimum upper bound, ∈ is the performance gap between the global model trained on the generated dataset and the original dataset; The input sample is x and the global model parameter is θ T When , the prediction result of the global model is is the global model prediction result The information cross entropy loss function between y and the sample label y. Therefore, the optimization goal of the present invention is to generate a data set by optimizing To achieve:

[0075]

[0076] Based on the above analysis, an embodiment of the present invention provides a low communication cost smart meter fault diagnosis method based on single-round federated learning, and the specific implementation process is as follows:

[0077] S1: The central server determines the global model structure, initializes the parameters of the global model, and sends it to each smart meter node.

[0078] S2: Each smart meter node trains the received initial global model locally using local data until all nodes complete local model training and obtain local model parameters θ 1～N , where N is the total number of smart meter nodes.

[0079] S3: Each smart meter node uploads the local model parameters, as well as the mean and variance recorded in the local model of each batch normalization layer after training to the central server.

[0080] S4: The central server uses the fault category labels and regularization items and the local model parameters uploaded by each smart meter node to generate a data set. Specifically:

[0081] This stage is used to generate data by aligning the statistics (mean and variance) of the given fault classification and batch normalization layers. The target formula is as follows:

[0082]

[0083] Among them, k is the index of each smart meter node, It is the trained local model uploaded by each smart meter node, which will be frozen at this stage. As a training variable. R is the regularization term, as follows:

[0084]

[0085] Among them, α tv ,l,BN are the weights of three regularization terms respectively. In this embodiment, α tv =0.1,α l =0.4,α BN =0.5, but not limited to this.

[0086] This regularization term makes the generated sample data not diverge.

[0087] in, for The i-th element of , β is a smoothing factor used to reduce the noise of generated data. In this embodiment, β=0.2, but is not limited thereto.

[0088] It is used to achieve the alignment of the statistics of the batch normalization layer. The formula is as follows:

[0089]

[0090] Where m represents the index of the batch normalization layer (BN), and They represent the mean and variance of the generated data at the mth batch normalization layer respectively. and They represent the mean and variance recorded by the mth batch normalization layer in the local model after training. T represents the local original dataset.

[0091] When the central server trains the global model, it uses block optimization technology to update only part of the data block in each iteration to prevent overfitting. This method helps improve the generated data from the perspective of cropping area.

[0092] S5: Reset soft labels. Use the pre-generated soft label method, that is, randomly exchange the generated data vector Part of the data vector in the , to obtain a new generated data vector with the same length Soft Label The formula is as follows:

[0093]

[0094] in, yes The i-th sample data, is the corresponding soft tag, Indicates that Input the trained local model to obtain the local model prediction result.

[0095] S6: Use the following objective function to generate data vectors Train the global model on Until convergence:

[0096]

[0097] in, is the target loss function, is the global model to be trained, Refers to the mapping function of the global model to the input.

[0098] S7: The central server sends the trained global model parameters to each smart meter node to update the local model parameters. The smart meter node inputs the real-time collected data into the latest local model to obtain the fault diagnosis result, that is, the specific fault category.

[0099] In this embodiment, preferably, after each smart meter node in S2 completes training of the local model, the local model parameter vector θ k ,=1,…,compress and obtain the compressed local model parameter vector Q s (θ k), and then the compressed local model parameter vector Q s (θ k ), and the mean and variance recorded in the local model of each batch normalization layer after training are uploaded to the central server. That is, the compression step of the local model parameters is added to further save the communication volume, and other processes remain unchanged. Specifically, the compression process is as follows:

[0100] The local model parameter vector θ of smart meter node k k The i-th element of The compression value The calculation formula is:

[0101]

[0102] Wherein, s is a compression factor. The larger s is, the lower the compression degree is, and the more the data precision loss can be reduced. In this embodiment, s=128, but it is not limited thereto. s (· is the compression function, sgn(· is the sign function, ||·|| is the modulus of the orientation quantity, ξ i (θ k ,) are independent random variables, defined as follows:

[0103] Let ρ∈[0,s) be an integer, so That is, [ρ / ,(ρ+1 / ] corresponds to The compression interval. At this time, ξ i (θ k ,) is defined as:

[0104]

[0105] Among them, p is the probability of taking the corresponding value,

[0106] Corresponding to the aforementioned embodiment of the low communication cost smart meter fault diagnosis method based on single-round federated learning, the present invention also provides an embodiment of the low communication cost smart meter fault diagnosis device based on single-round federated learning.

[0107] See also Figure 3 The low communication cost smart meter fault diagnosis device based on single-round federated learning provided in an embodiment of the present invention includes a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement the low communication cost smart meter fault diagnosis method based on single-round federated learning in the above embodiment.

[0108] The embodiment of the low communication cost smart meter fault diagnosis device based on single-round federated learning of the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capability in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 3 As shown, it is a hardware structure diagram of any device with data processing capability where the low communication cost intelligent meter fault diagnosis device based on single-round federated learning of the present invention is located, except Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0109] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0110] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0111] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the low-communication-cost smart meter fault diagnosis method based on single-round federated learning in the above embodiment is implemented.

[0112] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capability. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0113] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A low communication cost smart meter fault diagnosis method based on single-round federated learning, characterized in that: include: S1: The central server determines the global fault diagnosis model structure, initializes the parameters of the global model, and sends it to each smart meter node; S2: Each smart meter node locally trains the received initial global model using local data until all nodes complete the training of the local fault diagnosis model and obtain local model parameters; S3: Each smart meter node uploads the local model parameters, as well as the mean and variance statistics recorded in the local model of each batch normalization layer after training to the central server; S4: In the central server, the corresponding data set of each node is generated by aligning the given fault classification and the statistics of the batch normalization layer, and using the local model parameters uploaded by each smart meter node; S5: Crop, mix and generate soft labels for the generated dataset; S6: The central server inputs the generated data set and soft labels into the global model for training, and sends the trained global model parameters to each smart meter node to update the local model parameters; S7: Each smart meter node inputs the real-time collected data into the latest local model to obtain the fault diagnosis result.

2. The low communication cost smart meter fault diagnosis method based on single-round federated learning according to claim 1 is characterized in that: The local model deployed in the smart meter node adopts a ResNet network structure, including an initial convolutional layer and several residual modules. The input data x is input to the residual module, and the output results after the first 3×3 convolutional layer, the first batch normalization layer, the first activation function layer, the second 3×3 convolutional layer, and the second batch normalization layer are added to the input data x and then passed through the second activation function layer to obtain the output data of the residual module.

3. The low communication cost smart meter fault diagnosis method based on single-round federated learning according to claim 1 is characterized in that: The total number of smart meter nodes is N, and the data sets on each smart meter node are: Among them, T k is the data set on the k node, |T k | is the total number of samples on k nodes; is the sample data vector on the k node, For the corresponding x k The sample label vector of ; The data sets corresponding to each node generated by the central server Denoted as: in, is the dataset generated by the model parameters uploaded by k nodes to the central server; is the sample data vector corresponding to the generated k nodes, For the corresponding Soft label vector of ; The original sample data of each node is recorded as x, and the corresponding sample label is recorded as y; the generated sample data corresponding to each node is recorded as The corresponding soft label is recorded as 4. The low communication cost smart meter fault diagnosis method based on single-round federated learning according to claim 3 is characterized in that: On the central server, the global model training goal using the generated data corresponding to all nodes is: in, The input sample data is The global model parameters are The global model prediction result is yes With soft label The information cross entropy loss function between them, E is the expectation; The goal of data generation is stated as: Among them, sup(·) is the supremum function, ∈ is the performance gap between the global model trained on the generated dataset and the original dataset; The input sample data is x, and the global model parameter is θ T The global model prediction result is yes The information cross entropy loss function between the sample label y; generate a data set by optimizing To achieve the optimization goal:

5. The low communication cost smart meter fault diagnosis method based on single-round federated learning according to claim 4 is characterized in that: The central server generates a data set corresponding to each node by aligning the given fault classification and the statistics of the batch normalization layer and using the local model parameters uploaded by each smart meter node, specifically: Used to generate sample data corresponding to k nodes The target formula is as follows: in, is the trained local model uploaded by each node, τ is the information cross entropy loss function, and R is the regularization term. The formula is as follows: Among them, α tv ,α l ,α BN They are the weights of the three regular terms respectively; Used to make the generated sample data not diverge; Used to reduce the noise of generated data, for The i-th element of , β is the smoothing factor; It is used to achieve the alignment of the statistics of the batch normalization layer. The formula is as follows: Where m represents the index of the batch normalization layer, and Respectively represent the mean and variance of the generated data on the mth batch normalization layer; and They represent the mean and variance recorded by the mth batch normalization layer in the local model after training; T represents the local original dataset.

6. The low communication cost smart meter fault diagnosis method based on single-round federated learning according to claim 1 is characterized in that: When the central server trains the global model, block optimization technology is used to update only part of the data blocks in each iteration to prevent overfitting.

7. The low communication cost smart meter fault diagnosis method based on single-round federated learning according to claim 1 is characterized in that: S5 is specifically: using the pre-generated soft label method, that is, randomly exchanging the generated data vector Part of the data vector in the , to obtain a new generated data vector with the same length Soft Label The formula is as follows: in, yes The i-th sample data, is the corresponding soft tag, Indicates that Input the trained local model to obtain the local model prediction result.

8. The low communication cost smart meter fault diagnosis method based on single-round federated learning according to claim 7 is characterized in that: Use the following objective function to generate the data vector Train the global model on Until convergence: in, is the target loss function, is the global model to be trained, Refers to the mapping function of the global model to the input.

9. The low communication cost smart meter fault diagnosis method based on single-round federated learning according to claim 1 is characterized in that: After the local model training of each smart meter node is completed, the local model parameters are compressed and then uploaded to the central server. Specifically: The local model parameter vector θ of smart meter node k k The i-th element of The compression value The calculation formula is: Where s is the compression factor, Q s (·) is the compression function, sgn(·) is the sign function, ||·|| is the modulus of the orientation quantity, ξ i (θ k ,s) are independent random variables, defined as follows: Let ρ∈[0,s) be an integer, so That is, [ρ / s,(ρ+1) / s] corresponds to The compression interval; at this time, ξ i (θ k ,s) is defined as: Among them, p is the probability of taking the corresponding value, 10. A low-communication-cost smart meter fault diagnosis device based on single-round federated learning, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, it is used to implement the low communication cost smart meter fault diagnosis method based on single-round federated learning as described in any one of claims 1 to 9.

Images