A smart grid fault detection model training optimization method based on federated learning

Abstract

The present invention provides a method for training and optimizing a smart grid fault detection model based on federated learning, which relates to the field of fault detection technology. By using federated learning to perform model assignment, real-time data training, data similarity judgment, model adjustment, and model parameter interaction between a client and a central server, the method realizes efficient training and optimization of the smart grid fault detection model, protects data privacy, and improves the model's adaptability to power grids in different regions and the accuracy of fault detection.

Description

Technical Field

[0001] The present invention relates to the field of fault detection technology, and in particular to a method for training and optimizing a smart grid fault detection model based on federated learning. Background Art

[0002] Smart grids are a core component of modern power systems. With the increasing complexity and intelligence of power systems, fault detection has become a key link in ensuring the safe operation of smart grids. Traditional fault detection methods rely primarily on centralized data processing and single model training. In smart grids, data contains a large amount of sensitive information, such as user electricity usage habits and equipment operating status. Traditional centralized data processing methods are prone to data leakage, raising data privacy protection issues. Grid equipment types and failure modes vary significantly across regions, making single fault detection methods difficult to adapt to diverse fault characteristics, resulting in insufficient detection accuracy and timeliness. Centralized training requires large amounts of data to be transmitted to a central server, resulting in high data transmission and processing overhead, low training efficiency, and difficulty meeting the needs of implementing fault detection. As grid equipment is updated and failure modes change, models need to be continuously updated to maintain high performance. Traditional model update methods are complex and often lead to performance degradation.

[0003] Federated learning, as a new distributed machine learning technology, provides a new approach to solving the above problems. Federated learning allows each client to train the model locally and only upload the model parameters or feature maps to the central party for aggregation, thereby protecting data privacy and improving training efficiency. However, existing federated learning methods still have some shortcomings when applied to smart grid fault detection, such as inaccurate model tailoring, complex data processing procedures, and low interaction efficiency with the central party. Summary of the Invention

[0004] The present invention provides a federated learning-based smart grid fault detection model training optimization method, the purpose of which is to improve the adaptability and detection accuracy of the model.

[0005] To achieve the above objectives, the present invention provides a method for training and optimizing a smart grid fault detection model based on federated learning, comprising:

[0006] Step 1: The central server dispatches the pre-trained main fault detection model as the initial model and training tasks to the clients in each smart grid area;

[0007] Step 2: Each client performs coarse-grained labeling on the collected local data according to the training task and then extracts features to obtain a global prompt pool and upload it to the central server;

[0008] Step 3: The central server selects the global prompt pool uploaded by any client as the student pool, uses the student pool to perform knowledge distillation on all global prompt pools, obtains the aggregated global prompt pool and distributes it to each client;

[0009] Step 4: Each client performs similarity determination on the local data collected in real time based on the aggregated global prompt pool to obtain a determination result. The local data collected in real time is then input into the initial model. The initial model is trained based on the characteristics of the local smart grid area and the determination result to obtain a local model.

[0010] Step 5: Each client inputs the model parameters of the local model into the central server for aggregation to obtain the global model parameters, and uses the global model parameters to update the model parameters of the initial model to obtain the global fault detection model for fault detection of the target smart grid;

[0011] In step 6, the central server distributes the model parameters of the global fault detection model to the clients in each smart grid area, optimizes the local model of each client and uses it as the initial model, and returns to step 2 until the training stops when each client stops collecting new data.

[0012] More specifically, step 2 includes:

[0013] Each client tags the local data to obtain local data with coarse-grained prompts;

[0014] Pre-install the pre-trained Vision Transformer model in each client as the base model, and build an initialized global prompt pool for each client;

[0015] Input the local data with coarse-grained prompts into the basic model for feature extraction to obtain embedded features, and calculate the input key of each data in the local data with coarse-grained prompts through the basic model;

[0016] Calculate the similarity between the key of each prompt in the initialized global prompt pool and each input key;

[0017] Select the indexes of multiple prompts whose similarity is greater than a preset threshold, and concatenate the prompts and embedded features corresponding to the indexes to obtain the concatenated embedded features;

[0018] The basic model is trained for classification using the spliced ​​embedded features to obtain a global prompt pool and upload it to the central server.

[0019] Specifically, the expression for selecting multiple prompts with similarity greater than a preset threshold is:

[0020] ;

[0021] in, Represents the selected prompt set, Represents the distance function, which is used to calculate the similarity. The smaller the distance, the greater the similarity. Representation data Enter key, Indicates the A reminder key, Indicates the number of selections.

[0022] Furthermore, each client performs similarity determination on the local data collected in real time based on the aggregated global prompt pool, and obtains the determination results, including:

[0023] Each client calculates the similarity between the features of the local data currently input and the features of the local data previously input. The calculation expression of the similarity value is:

[0024] ;

[0025] in, Represents the similarity value, Indicates the characteristics of the local data currently input. Represents the characteristics of the local data input last time;

[0026] A similarity threshold is set. When the similarity value is greater than or equal to the similarity threshold, the result is determined to be that the local data currently input is repeated with the local data previously input;

[0027] When the similarity value is less than the similarity threshold, the determination result is that the local data currently input is non-duplicate data.

[0028] Furthermore, before each client inputs the model parameters of the local model into the central server for aggregation, it also includes:

[0029] Gaussian noise is added to the model parameters of the local model on each client to obtain the noisy model parameters, and the noisy model parameters are input into the central server.

[0030] Furthermore, the central server selects multiple similar model parameters from the uploaded model parameters for knowledge aggregation.

[0031] Furthermore, the loss function of the central server for knowledge aggregation is:

[0032] ;

[0033] in, represents the cross entropy loss between the predicted value and the true label, represents the divergence function, represents the predicted value of the central server, Indicates the The predicted value of each client, 、 represents the hyperparameters of the central server, represents the KL divergence, 、 Respectively represent The predicted probabilities of the clients and the central server.

[0034] Specifically, the loss function for optimizing the local model of each client is:

[0035] ;

[0036] in, 、 Represents the client's hyperparameters.

[0037] More specifically, the model parameters include:

[0038] Model weights, biases, and key metrics during training.

[0039] The above solution of the present invention has the following beneficial effects:

[0040] The present invention uses federated learning to perform model assignment, real-time data training, data similarity judgment, model adjustment, and model parameter interaction between the client and the central server to achieve efficient training and optimization of the smart grid fault detection model, while protecting data privacy and improving the model's adaptability to power grids in different regions and the accuracy of fault detection.

[0041] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the framework of federated learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] To make the technical problems, technical solutions, and advantages to be solved by the present invention more clear, the following is a detailed description with reference to the accompanying drawings and specific embodiments. It is obvious that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0045] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0046] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to a locking connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0047] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0048] In response to existing problems, the present invention provides a smart grid fault detection model training optimization method based on federated learning.

[0049] like Figure 1 、 Figure 2 As shown, an embodiment of the present invention provides a method for training and optimizing a smart grid fault detection model based on federated learning, comprising:

[0050] Step 1: The central server dispatches the pre-trained main fault detection model as the initial model and training tasks to the clients in each smart grid area;

[0051] Step 2: Each client performs coarse-grained labeling on the collected local data according to the training task and then extracts features to obtain a global prompt pool and upload it to the central server;

[0052] Step 3: The central server selects the global prompt pool uploaded by any client as the student pool, uses the student pool to perform knowledge distillation on all global prompt pools, obtains the aggregated global prompt pool and distributes it to each client;

[0053] Step 4: Each client performs similarity determination on the local data collected in real time based on the aggregated global prompt pool to obtain a determination result. The local data collected in real time is then input into the initial model. The initial model is trained based on the characteristics of the local smart grid area and the determination result to obtain a local model.

[0054] Step 5: Each client inputs the model parameters of the local model into the central server for aggregation to obtain the global model parameters, and uses the global model parameters to update the model parameters of the initial model to obtain the global fault detection model for fault detection of the target smart grid;

[0055] In step 6, the central server distributes the model parameters of the global fault detection model to the clients in each smart grid area, optimizes the local model of each client and uses it as the initial model, and returns to step 2 until the training stops when each client stops collecting new data.

[0056] Specifically, the main fault detection model is a neural network model based on deep learning, which has the ability to detect a variety of common power grid fault types.

[0057] Specifically, step 2 includes:

[0058] Each client tags the local data to obtain local data with coarse-grained prompts;

[0059] Pre-install the pre-trained Vision Transformer model in each client as the base model, and build an initialized global prompt pool for each client;

[0060] Input the local data with coarse-grained prompts into the basic model for feature extraction to obtain embedded features, and calculate the input key of each data in the local data with coarse-grained prompts through the basic model;

[0061] Calculate the similarity between the key of each prompt in the initialized global prompt pool and each input key;

[0062] Select the indexes of multiple prompts whose similarity is greater than a preset threshold, and concatenate the prompts and embedded features corresponding to the indexes to obtain the concatenated embedded features;

[0063] The basic model is trained for classification using the spliced ​​embedded features to obtain a global prompt pool and upload it to the central server.

[0064] In an embodiment of the present invention, the operating data in the smart grid is used as an example of local data. Each client uses a coarse-grained prompt pool to label the electricity consumption data as local data with coarse-grained prompts, such as "high-voltage transmission line data", "substation equipment data", "distribution line data" and other general categories; then, a preset pre-trained VisionTransformer model is used to extract vital signs from these data with coarse-grained prompts to obtain embedded features, such as the degree of voltage anomaly, current fluctuation trend, equipment temperature change and other features.

[0065] It should be noted that the parameters of the basic model remain frozen during the training process. The structure of the basic model includes an embedding layer and a computation layer. The embedding layer is used to obtain its embedding features, and the expression is:

[0066] ;

[0067] The calculation layer is used to calculate the input key of local data. The expression is:

[0068] ;

[0069] in, represents the embedded features, represents the embedding function, Represents local data, Representation data Enter key.

[0070] Specifically, the expression for selecting multiple prompts whose similarity is greater than a preset threshold is:

[0071] ;

[0072] in, Represents the selected prompt set, Represents the distance function, which is used to calculate the similarity. The smaller the distance, the greater the similarity. Representation data Enter key, Indicates the A reminder key, Indicates the number of selections.

[0073] Specifically, the hint and embedding features corresponding to the index Perform splicing to obtain the spliced ​​embedded features , the expression is:

[0074] ;

[0075] in, Indicates the first index The corresponding prompts, Indicates the The hint corresponding to the index.

[0076] Specifically, the basic model is trained by classification using the spliced ​​embedded features, and the global prompt pool is obtained as , the loss function for classification training for:

[0077] ;

[0078] in, represents the softmax cross entropy loss function, Represents the classification head, represents the embedded features after splicing, represents the hyperparameter, represents the proxy loss used to make the selected keys closer to their corresponding embedding features.

[0079] Specifically, step 3 includes:

[0080] The central server collects the global prompt pool from all clients and selects the global prompt pool of any client as the student pool;

[0081] The student pool will be used to perform knowledge distillation on all global prompt pools to obtain the aggregated global prompt pool. Taking the small proxy dataset of the central server as an example, the distillation loss is calculated and the student pool is updated to minimize the distillation loss. The calculation expression of the distillation loss is:

[0082] ;

[0083] in, represents the distillation loss value, represents the mean square error, represents the embedded features after the input small agent data and the prompts in the student pool are concatenated, Represents the embedded features after the input small agent data is concatenated with the prompts in the teacher pool;

[0084] The central server distributes the aggregated global prompt pool to each client to determine whether the newly collected local data is repeated with the previously collected local data.

[0085] In an embodiment of the present invention, the pre-trained Vision Transformer model is also used to perform fine-grained labeling through a fine-grained prompt pool. Taking high-voltage transmission line data as an example, the fine-grained prompt pool performs fine-grained labeling under the classification of "high-voltage transmission line data" into more detailed types such as "icing data of high-voltage transmission lines in mountainous areas" and "bird damage data of high-voltage transmission lines in mountainous areas". The pre-trained Vision Transformer model is then used to more clearly identify the fine-grained features of the data, such as ice thickness, icicle shape, bird activity area, bird nest construction location and other detailed features closely related to faults, providing strong support for accurate fault detection.

[0086] Specifically, each client performs similarity determination on the local data collected in real time based on the aggregated global prompt pool, and obtains the determination results, including:

[0087] Each client calculates the similarity between the features of the local data currently input and the features of the local data previously input. The calculation expression of the similarity value is:

[0088] ;

[0089] in, Represents the similarity value, Indicates the characteristics of the local data currently input. Indicates the characteristics of the local data input last time, ;

[0090] Setting similarity threshold , when the similarity value Greater than or equal to the similarity threshold , the result of the determination is that the local data currently input is the same as the local data previously input;

[0091] When the similarity value Less than the similarity threshold , the result is that the local data currently input is non-duplicate data.

[0092] In this embodiment of the present invention, when the local data inputted this time is repeated with the local data inputted previously, such as two similar ice growth pattern data, backpropagation is performed on the model's masks and weights, the corresponding weights are appropriately reduced, and the mask importance flags are adjusted, allowing the model to better learn and utilize the important information in the repeated data, thereby improving the model's personalization capabilities.

[0093] Local Model: Local Model Already learned data set ;

[0094] Saved weights: previous data The weight is expressed as ;

[0095] New missions are coming: New missions Arrived, and For all ;

[0096] Real-valued mask: data The real-valued mask of is represented as ;

[0097] Save a copy of the weights: Save to prevent the impact of other data;

[0098] Retrain masks and weights: Retrain real-valued masks and unoccupied weights In the new task On the top, the loss function is used to update through back propagation, and finally generate and ;

[0099] Apply new weights: In the next repetition of the task Before arriving, use Processing Data Otherwise, use Maintain accuracy on other tasks.

[0100] In an embodiment of the present invention, when the local data input last time is non-repetitive data, for example, when a new type of smart grid energy storage device fault data appears, which is different from the previous traditional grid equipment fault data, the model readjusts the relevant weights and masks so that it can learn and understand the characteristics of this new device fault, such as abnormal charging and discharging of energy storage equipment, changes in internal resistance caused by battery aging, and other new features, thereby improving the model's detection ability for new faults.

[0101] Local Model: Local Model Already learned data set ;

[0102] Saved weights: previous data The weight is expressed as ;

[0103] New missions are coming: New missions Arrived, and For all ;

[0104] Learnable Mask: Apply a learnable binary mask To select the old weight ,in yes Dimensions;

[0105] Element-wise product: The weights of the choices are expressed as ;

[0106] Real-valued mask: learn a real-valued mask and pass Threshold binarization generation ;

[0107] Optimizing Masks and Weights: Masks and unoccupied weights Together in a new mission Optimize on the training data and update using the loss function through back propagation;

[0108] Prune again: After the optimization is completed, prune the model again until the new task The weights meet the task requirements and then freeze these weights to avoid forgetting old tasks.

[0109] In the embodiment of the present invention, each collected data is regarded as a task, a new task is to process new local data, and an old task is to process local data completed before the new task.

[0110] Model pruning optimization

[0111] Budget presetting identifies low-importance parameters that may be adjusted or trimmed during new task learning. By constructing a proxy loss function, the key information of these parameters is transferred to other parameters within the budget, gradually approaching zero for the parameters to be adjusted or trimmed, achieving both information transfer and preservation.

[0112] Before pruning, identify and mark parameters with smaller weights , these parameters are considered candidates for pruning, with the smallest 10% of weights. During training, a proxy loss function is used:

[0113] ;

[0114] in, Is the original loss function, used to measure the model's performance on each device, is a hyperparameter that controls the strength of information transfer. is a low-weight parameter Norm, used to measure the size of these parameters.

[0115] The budget threshold in the embodiment of the present invention It is a preset training budget that controls the learning rate adjustment during information transfer. It ensures that information transfer is completed within limited training resources, thereby maintaining the performance of the model after pruning.

[0116] In order to complete the information transfer within a limited training budget, a budget-aware schedule is introduced, which accelerates the information transfer process by dynamically adjusting the learning rate. The expression is:

[0117] ;

[0118] in, is the REX scheduling factor, defined as ,in is the preset training budget, is the original enrollment rate, is a scaling term that dynamically adjusts the learning rate based on the magnitude of the parameter to be pruned.

[0119] Specifically, before each client inputs the model parameters of the local model into the central server for aggregation, it also includes:

[0120] Gaussian noise is added to the model parameters of the local model on each client to obtain noisy model parameters, and the noisy model parameters are input into the central server. In this way, the central party can use these noisy model parameters to optimize and update the model without obtaining the original data of the client, thereby protecting the client's data privacy.

[0121] It should be noted that the parameters of Gaussian noise need to be adjusted according to the privacy budget. and failure probability And the sensitivity of the function The specific calculation formula is:

[0122] ;

[0123] in, Is the privacy parameter, indicating the strength of privacy protection, the smaller the A value of indicates stronger privacy protection. is the failure probability, indicating that the algorithm may not satisfy differential privacy in rare cases. Values ​​indicate higher reliability, It is the sensitivity of the function, that is, the maximum change value of the function on adjacent data sets.

[0124] Specifically, the central server selects multiple similar model parameters from the uploaded model parameters for knowledge aggregation.

[0125] Specifically, the loss function of the central server for knowledge aggregation is:

[0126] ;

[0127] in, represents the cross entropy loss between the predicted value and the true label, represents the divergence function, used for knowledge distillation, represents the predicted value of the central server, Indicates the The predicted value of each client, 、 Represents the hyperparameters of the central server, used to control the cross entropy loss and KL divergence The proportion of 、 Respectively represent The predicted probabilities of the clients and the central server.

[0128] Specifically, the loss function for optimizing the local model of each client is:

[0129] ;

[0130] in, 、 Represents the client's hyperparameters, which are used to control the weights of cross entropy loss and KL divergence in the loss function.

[0131] Specifically, the model parameters include:

[0132] Model weights, biases, and key metrics during training, including loss function value and accuracy.

[0133] It should be noted that when the central server aggregates model parameters, it also needs to consider the stability and convergence of the model. By introducing regularization terms or constraints, it can prevent the model from overfitting or parameter drift during the aggregation process.

[0134] Specifically, the model parameters of the initial model are updated using the global model parameters to obtain a global fault detection model for fault detection in the target smart grid. This global fault detection model is deployed in the clients of each smart grid area. In actual operation, the model inputs the grid operation data in real time for fault detection and early warning. The model can quickly and accurately identify various fault types in the smart grid. For example, in the high-voltage transmission line area in mountainous areas, the fault types identified are line icing, line breakage and other faults, and timely alarms are issued so that operation and maintenance personnel can take corresponding measures.

[0135] The embodiment of the present invention implements the steps of model assignment, real-time data training, data similarity judgment, model adjustment, and model parameter interaction between the client and the central server through federated learning, thereby achieving efficient training and optimization of the smart grid fault detection model, while protecting data privacy, and improving the model's adaptability to power grids in different regions and the accuracy of fault detection.

[0136] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A smart grid fault detection model training optimization method based on federated learning, characterized in that: include: Step 1: The central server dispatches the pre-trained main fault detection model as the initial model and training tasks to the clients in each smart grid area; Step 2: Each client performs coarse-grained labeling on the collected local data according to the training task and then performs feature extraction to obtain a global prompt pool and upload it to the central server, including: Each client tags the local data to obtain local data with coarse-grained prompts; Pre-install the pre-trained Vision Transformer model in each client as the base model, and build an initialized global prompt pool for each client; Inputting the local data with coarse-grained prompts into the basic model for feature extraction to obtain embedded features, and calculating the input key of each data in the local data with coarse-grained prompts through the basic model; Calculating the similarity between the key of each prompt in the initialized global prompt pool and each input key; Selecting indexes of multiple prompts whose similarity is greater than a preset threshold, and concatenating the prompts corresponding to the indexes with the embedded features to obtain concatenated embedded features; Using the spliced ​​embedded features to perform classification training on the basic model, a global prompt pool is obtained and uploaded to the central server; Step 3: The central server selects the global prompt pool uploaded by any client as the student pool, uses the student pool to perform knowledge distillation on all global prompt pools, obtains the aggregated global prompt pool and distributes it to each client; Step 4: Each client performs similarity determination on the local data collected in real time based on the aggregated global prompt pool to obtain a determination result, and inputs the local data collected in real time into the initial model. The initial model is trained based on the characteristics of the local smart grid area and the determination result to obtain a local model; Step 5: Each client inputs the model parameters of the local model into the central server for aggregation to obtain global model parameters, and uses the global model parameters to update the model parameters of the initial model to obtain a global fault detection model for fault detection of the target smart grid; In step 6, the central server distributes the model parameters of the global fault detection model to the clients in each smart grid area, optimizes the local model of each client and uses it as the initial model, and returns to step 2 until the training stops when each client stops collecting new data.

2. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 1, characterized in that: The expression for selecting multiple prompts whose similarity is greater than a preset threshold is: in, Represents the selected prompt set, Represents the distance function, which is used to calculate the similarity. The smaller the distance, the greater the similarity. Representation data Enter key, Indicates the A reminder key, Indicates the number of selections.

3. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 2, wherein: Each client performs similarity determination on the local data collected in real time based on the aggregated global prompt pool, and obtains the determination results, including: Each client calculates the similarity between the features of the local data currently input and the features of the local data previously input. The calculation expression of the similarity value is: ； in, Represents the similarity value, Indicates the characteristics of the local data currently input. Represents the characteristics of the local data input last time; A similarity threshold is set. When the similarity value is greater than or equal to the similarity threshold, the result is determined to be that the local data currently input is repeated with the local data previously input; When the similarity value is less than the similarity threshold, the determination result is that the local data currently input is non-duplicate data.

4. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 3, wherein: Before each client inputs the model parameters of the local model into the central server for aggregation, the method further includes: Gaussian noise is added to the model parameters of the local model on each client to obtain noisy model parameters, and the noisy model parameters are input into the central server.

5. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 4, wherein: The central server selects multiple similar model parameters from the uploaded model parameters for knowledge aggregation.

6. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 5, characterized in that: The loss function of the central server for knowledge aggregation is: in, represents the cross entropy loss between the predicted value and the true label, represents the divergence function, represents the predicted value of the central server, Indicates the The predicted value of each client, 、 represents the hyperparameters of the central server, represents the KL divergence, 、 Respectively represent The predicted probabilities of the clients and the central server.

7. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 6, wherein: The loss function for optimizing the local model of each client is: in, 、 Represents the client's hyperparameters.

8. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 7, characterized in that: The model parameters include: Model weights, biases, and key metrics during training.

Images