Intelligent power grid fault detection model training optimization method based on federated learning

Abstract

The invention provides an intelligent power grid fault detection model training optimization method based on federated learning, which relates to the technical field of fault detection and comprises the steps of model assignment, real-time data training, data similarity judgment, model adjustment, model parameter interaction between a client and a central server and the like through federated learning. Efficient training and optimization of the intelligent power grid fault detection model are realized, data privacy is protected, and adaptability of the model to different regional power grids and accuracy of fault detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fault detection, and in particular to a method for optimizing training of a smart grid fault detection model based on federated learning. Background Art

[0002] As a core component of modern power systems, smart grids have become increasingly complex and intelligent, and fault detection has become a key link in ensuring the safe operation of smart grids. Traditional fault detection methods mainly rely on centralized data processing and single model training. For example, in smart grids, data contains a lot of sensitive information, such as users' electricity usage habits, equipment operating status, etc. Traditional centralized data processing methods are prone to data leakage, causing data privacy protection issues; the types of power grid equipment and failure modes in different regions vary greatly, and single fault detection is difficult to adapt to diverse fault characteristics, resulting in insufficient detection accuracy and timeliness; centralized training requires a large amount of data to be transmitted to the central server, which has high data transmission and processing overhead, low training efficiency, and is difficult to meet the needs of implementing fault detection; with the update of power grid equipment and changes in failure modes, the model needs to be continuously updated to maintain high performance, and the traditional model update method is complex and often leads to a decline in model performance.

[0003] As a new type of distributed machine learning technology, federated learning provides a new idea for solving the above problems. Federated learning allows each client to perform model training locally and only upload model parameters or feature maps to the center for aggregation, thereby protecting data privacy and improving training efficiency. However, the existing federated learning methods still have some shortcomings when applied to smart grid fault detection, such as inaccurate model tailoring, complex data processing procedures, and low interaction efficiency with the center. Summary of the invention

[0004] The present invention provides a method for optimizing the training of a smart grid fault detection model based on federated learning, the purpose of which is to improve the adaptability and detection accuracy of the model.

[0005] In order to achieve the above object, the present invention provides a smart grid fault detection model training optimization method based on federated learning, comprising: Step 1: The central server dispatches the pre-trained main fault detection model as the initial model and training tasks to the clients in each smart grid area; Step 2: Each client performs coarse-grained labeling on the collected local data according to the training task and then extracts features to obtain a global prompt pool and upload it to the central server; Step 3: The central server selects the global prompt pool uploaded by any client as the student pool, uses the student pool to perform knowledge distillation on all global prompt pools, obtains the aggregated global prompt pool and distributes it to each client; Step 4: Each client performs similarity determination on the local data collected in real time according to the aggregated global prompt pool to obtain the determination result, and inputs the local data collected in real time into the initial model, and trains the initial model according to the characteristics of the local smart grid area and the determination result to obtain the local model; Step 5: Each client inputs the model parameters of the local model into the central server for aggregation to obtain the global model parameters, and uses the global model parameters to update the model parameters of the initial model to obtain a global fault detection model for fault detection of the target smart grid; In step 6, the central server distributes the model parameters of the global fault detection model to the clients in each smart grid area, optimizes the local model of each client as the initial model, and returns to execute step 2 until the training stops when each client stops collecting new data.

[0006] More specifically, step 2 includes: Each client tags the local data to obtain local data with coarse-grained prompts; Preset the pre-trained Vision Transformer model as the base model in each client, and build an initialized global prompt pool for each client; Input the local data with coarse-grained prompts into the basic model for feature extraction to obtain embedded features, and calculate the input key of each data in the local data with coarse-grained prompts through the basic model; Calculate the similarity between the key of each prompt in the initialized global prompt pool and each input key; Selecting indexes of multiple prompts whose similarity is greater than a preset threshold, and concatenating the prompts and embedded features corresponding to the indexes to obtain concatenated embedded features; The concatenated embedded features are used to perform classification training on the basic model, and a global prompt pool is obtained and uploaded to the central server.

[0007] Specifically, the expression for selecting multiple prompts whose similarity is greater than a preset threshold is: ; in, represents the selected prompt set, Represents the distance function, which is used to calculate the similarity. The smaller the distance, the greater the similarity. Representation data The input key, Indicates A reminder key, Indicates the number of selections.

[0008] Furthermore, each client performs similarity determination on the local data collected in real time according to the aggregated global prompt pool, and obtains the determination results, including: Each client calculates the similarity value between the features of the local data currently input and the features of the local data previously input. The calculation expression of the similarity value is: ; in, Represents the similarity value, Indicates the characteristics of the local data currently input. Represents the characteristics of the local data input last time; A similarity threshold is set. When the similarity value is greater than or equal to the similarity threshold, the result of the determination is that the local data currently input is repeated with the local data previously input; When the similarity value is less than the similarity threshold, the determination result is that the local data currently input is non-duplicate data.

[0009] Specifically, before each client inputs the model parameters of the local model into the central server for aggregation, it also includes: Gaussian noise is added to the model parameters of the local model on each client to obtain the noisy model parameters, and the noisy model parameters are input into the central server.

[0010] Furthermore, the central server selects multiple similar model parameters from the uploaded model parameters for knowledge aggregation.

[0011] Specifically, the loss function of the central server for knowledge aggregation is: ; in, represents the cross entropy loss between the predicted value and the true label, represents the divergence function, represents the predicted value of the central server, Indicates The predicted value of each client, , represents the hyperparameters of the central server, represents the KL divergence, , Respectively represent The predicted probabilities of the clients and the central server.

[0012] Specifically, the loss function for optimizing the local model of each client is: ; in, , Represents the hyperparameters of the client.

[0013] More specifically, the model parameters include: Model weights, biases, and key metrics during training.

[0014] The above scheme of the present invention has the following beneficial effects: The present invention uses federated learning to perform model assignment, real-time data training, data similarity judgment, model adjustment, and model parameter interaction between the client and the central server to achieve efficient training and optimization of the smart grid fault detection model, while protecting data privacy and improving the model's adaptability to power grids in different regions and the accuracy of fault detection.

[0015] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention; Figure 2 Schematic diagram of the framework of federated learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0019] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0020] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] In view of the existing problems, the present invention provides a smart grid fault detection model training optimization method based on federated learning.

[0022] like Figure 1 , Figure 2 As shown, an embodiment of the present invention provides a method for optimizing training of a smart grid fault detection model based on federated learning, comprising: Step 1: The central server dispatches the pre-trained main fault detection model as the initial model and training tasks to the clients in each smart grid area; Step 2: Each client performs coarse-grained labeling on the collected local data according to the training task and then extracts features to obtain a global prompt pool and upload it to the central server; Step 3: The central server selects the global prompt pool uploaded by any client as the student pool, uses the student pool to perform knowledge distillation on all global prompt pools, obtains the aggregated global prompt pool and distributes it to each client; Step 4: Each client performs similarity determination on the local data collected in real time according to the aggregated global prompt pool to obtain the determination result, and inputs the local data collected in real time into the initial model, and trains the initial model according to the characteristics of the local smart grid area and the determination result to obtain the local model; Step 5: Each client inputs the model parameters of the local model into the central server for aggregation to obtain the global model parameters, and uses the global model parameters to update the model parameters of the initial model to obtain a global fault detection model for fault detection of the target smart grid; In step 6, the central server distributes the model parameters of the global fault detection model to the clients in each smart grid area, optimizes the local model of each client as the initial model, and returns to execute step 2 until the training stops when each client stops collecting new data.

[0023] Specifically, the main fault detection model is a neural network model based on deep learning, which has the ability to detect many common power grid fault types.

[0024] Specifically, step 2 includes: Each client tags the local data to obtain local data with coarse-grained prompts; Preset the pre-trained Vision Transformer model as the base model in each client, and build an initialized global prompt pool for each client; Input the local data with coarse-grained prompts into the basic model for feature extraction to obtain embedded features, and calculate the input key of each data in the local data with coarse-grained prompts through the basic model; Calculate the similarity between the key of each prompt in the initialized global prompt pool and each input key; Selecting indexes of multiple prompts whose similarity is greater than a preset threshold, and concatenating the prompts and embedded features corresponding to the indexes to obtain concatenated embedded features; The concatenated embedded features are used to perform classification training on the basic model, and a global prompt pool is obtained and uploaded to the central server.

[0025] The embodiment of the present invention takes the operation data in the smart grid as the local data as an example. Each client uses the coarse-grained prompt pool to mark the electricity consumption data as local data with coarse-grained prompts, for example, roughly classified as "high-voltage transmission line data", "substation equipment data", "distribution line data", etc.; then the preset pre-trained VisionTransformer model is used to extract vital signs from these data with coarse-grained prompts to obtain embedded features, such as voltage anomaly degree, current fluctuation trend, equipment temperature change and other features.

[0026] It should be noted that the parameters of the basic model remain frozen during the training process. The structure of the basic model includes an embedding layer and a computation layer. The embedding layer is used to obtain its embedding features, and the expression is: ; The calculation layer is used to calculate the input key of local data. The expression is: ; in, represents the embedded features, represents the embedding function, Represents local data, Representation data Enter key.

[0027] Specifically, the expression for selecting multiple prompts whose similarity is greater than a preset threshold is: ; in, represents the selected prompt set, Represents the distance function, which is used to calculate the similarity. The smaller the distance, the greater the similarity. Representation data The input key, Indicates A reminder key, Indicates the number of selections.

[0028] Specifically, the hint and embedding features corresponding to the index Perform splicing to obtain the spliced ​​embedded features , the expression is: ; in, Indicates the first index The corresponding tips, Indicates The hint corresponding to the index.

[0029] Specifically, the concatenated embedded features are used to classify and train the basic model, and the global prompt pool is obtained as , the loss function for classification training for: ; in, represents the softmax cross entropy loss function, Represents the classification head, represents the concatenated embedded features, represents the hyperparameter, represents the proxy loss used to make the selected keys closer to their corresponding embedded features.

[0030] Specifically, step 3 includes: The central server collects the global prompt pool from all clients and selects the global prompt pool of any client as the student pool; The student pool will be used to perform knowledge distillation on all global prompt pools to obtain the aggregated global prompt pool. Taking the small proxy dataset of the central server as an example, the distillation loss is calculated. The student pool is updated to minimize the distillation loss. The calculation expression of the distillation loss is: ; in, represents the distillation loss value, represents the mean square error, represents the embedded features after the input small agent data is concatenated with the prompts in the student pool, Represents the embedded features after the input small agent data is concatenated with the prompts in the teacher pool; The central server distributes the aggregated global prompt pool to each client to determine whether the newly collected local data is repeated with the local data collected last time.

[0031] In an embodiment of the present invention, the pre-trained Vision Transformer model is also used to perform fine-grained labeling through a fine-grained prompt pool. Taking high-voltage transmission line data as an example, the fine-grained prompt pool performs fine-grained labeling under the classification of "high-voltage transmission line data" into more subdivided types such as "ice coverage data of high-voltage transmission lines in mountainous areas" and "bird damage data of high-voltage transmission lines in mountainous areas". The pre-trained Vision Transformer model is then used to more clearly identify the fine-grained features of the data, such as ice thickness, icicle shape, bird activity area, bird nest construction location and other detailed features closely related to faults, thereby providing strong support for accurate fault detection.

[0032] Specifically, each client performs similarity determination on the local data collected in real time based on the aggregated global prompt pool, and obtains the determination results, including: Each client calculates the similarity value between the features of the local data currently input and the features of the local data previously input. The calculation expression of the similarity value is: ; in, Represents the similarity value, Indicates the characteristics of the local data currently input. Represents the characteristics of the local data input last time, ; Setting similarity threshold , when the similarity value Greater than or equal to the similarity threshold , the result of the determination is that the local data currently input is repeated with the local data previously input; When the similarity value Less than the similarity threshold , the result is that the local data currently input is non-duplicate data.

[0033] In the embodiment of the present invention, when the local data inputted at the current time is repeated with the local data inputted at the previous time, such as two similar ice growth pattern data, the mask and weight of the model are back-propagated, the corresponding weight is appropriately reduced, and the mask importance mark is adjusted, so that the model can better learn and utilize the important information in the repeated data, thereby improving the personalization ability of the model.

[0034] Local Model: Local Model Already learned data set ; Saved weights: previous data The weight is expressed as ; New Mission Arrives: New Mission Arrival, and For all ; Real-valued mask: data The real-valued mask of is represented as ; Save a copy of the weights: Save to prevent the impact of other data; Retrain masks and weights: Retrain real-valued masks and the unoccupied weights In the new task We use the loss function to update through back propagation and finally generate and ; Apply new weights: In the next repetition of the task Before arriving, use Processing Data Otherwise, use Maintain accuracy on other tasks.

[0035] In an embodiment of the present invention, when the local data input last time is non-repetitive data, for example, when a new type of smart grid energy storage equipment fault data appears, which is different from the previous traditional grid equipment fault data, the model readjusts the relevant weights and masks so that it can learn and understand the characteristics of this new equipment fault, such as abnormal charging and discharging of energy storage equipment, changes in internal resistance caused by battery aging, and other new features, thereby improving the model's detection capability for new faults.

[0036] Local Model: Local Model Already learned data set ; Saved weights: previous data The weight is expressed as ; New Mission Arrives: New Mission Arrival, and For all ; Learnable Mask: Apply a learnable binary mask To select the old weight ,in yes Dimensions; Element-wise product: The weights of the choices are expressed as ; Real-valued mask: Learn a real-valued mask and pass Generate by threshold binarization ; Optimizing Masks and Weights: Masks and the unoccupied weights Together on a new mission Optimize on the training data and update using the loss function through back propagation; Re-pruning: After the optimization is completed, the model is pruned again until the new task The weights meet the task requirements and then freeze these weights to avoid forgetting old tasks.

[0037] In the embodiment of the present invention, each collected data is regarded as a task, the new task is to process the new local data, and the old task is to process the local data completed before the new task.

[0038] Model pruning optimization Budget presetting is used to identify low-importance parameters that may be adjusted or trimmed during the learning of new tasks. By constructing a proxy loss function, the key information of these parameters is transferred to other parameters within the budget range, so that the values ​​of the parameters to be adjusted or trimmed gradually approach zero, thus achieving information transfer and retention.

[0039] Before pruning, identify and mark parameters with smaller weights , these parameters are considered candidates for pruning, with the smallest 10% of weights. During training, a proxy loss function is used: ; in, is the original loss function, used to measure the model's performance on each device, is a hyperparameter that controls the strength of information transfer. is a low weight parameter norm, used to measure the magnitude of these parameters.

[0040] The budget threshold in the embodiment of the present invention It is a preset training budget used to control the learning rate adjustment during information transfer. It ensures that information transfer is completed within limited training resources, thereby maintaining the performance of the model after pruning.

[0041] In order to complete the information transfer within the limited training budget, a budget-aware schedule is introduced, which accelerates the information transfer process by dynamically adjusting the learning rate. The expression is: ; in, is the REX scheduling factor, defined as ,in is the preset training budget. is the original enrollment rate, is a scaling term that dynamically adjusts the learning rate based on the magnitude of the parameter to be pruned.

[0042] Specifically, before each client inputs the model parameters of the local model into the central server for aggregation, it also includes: Gaussian noise is added to the model parameters of the local model on each client to obtain noisy model parameters, and the noisy model parameters are input into the central server. In this way, the central party can use these noisy model parameters to optimize and update the model without obtaining the original data of the client, thereby protecting the client's data privacy.

[0043] It should be noted that the parameters of Gaussian noise need to be based on the privacy budget and failure probability And the sensitivity of the function The specific calculation formula is: ; in, is a privacy parameter, indicating the strength of privacy protection. A value of 0 indicates stronger privacy protection. is the failure probability, indicating that the algorithm may not satisfy differential privacy in rare cases. Values ​​indicate higher reliability, It is the sensitivity of the function, that is, the maximum change value of the function on adjacent data sets.

[0044] Specifically, the central server selects multiple similar model parameters from the uploaded model parameters for knowledge aggregation.

[0045] Specifically, the loss function of the central server for knowledge aggregation is: ; in, represents the cross entropy loss between the predicted value and the true label, represents the divergence function, which is used for knowledge distillation. represents the predicted value of the central server, Indicates The predicted value of each client, , Represents the hyperparameters of the central server, used to control the cross entropy loss and KL divergence The proportion of , Respectively represent The predicted probabilities of the clients and the central server.

[0046] Specifically, the loss function for optimizing the local model of each client is: ; in, , Represents the client's hyperparameters, which are used to control the weights of cross entropy loss and KL divergence in the loss function.

[0047] Specifically, the model parameters include: Model weights, biases, and key indicators during training. Key indicators include loss function value and accuracy.

[0048] It should be noted that when the central server aggregates model parameters, it also needs to consider the stability and convergence of the model, and introduce regularization terms or constraints to prevent overfitting or parameter drift during the aggregation process.

[0049] Specifically, the model parameters of the initial model are updated using the global model parameters to obtain a global fault detection model for fault detection of the target smart grid. The global fault detection model is deployed in the clients of each smart grid area. In actual operation, the model inputs the grid operation data in real time for fault detection and early warning. The model can quickly and accurately identify various fault types in the smart grid. For example, in the area of ​​high-voltage transmission lines in mountainous areas, the fault types identified are line icing, line break and other faults, and timely alarms are issued so that operation and maintenance personnel can take corresponding measures.

[0050] The embodiment of the present invention implements the steps of model assignment, real-time data training, data similarity judgment, model adjustment, model parameter interaction between the client and the central server through federated learning, so as to realize efficient training and optimization of the smart grid fault detection model, protect data privacy at the same time, and improve the adaptability of the model to power grids in different regions and the accuracy of fault detection.

[0051] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for optimizing the training of a smart grid fault detection model based on federated learning, characterized in that: include: Step 1: The central server dispatches the pre-trained main fault detection model as the initial model and training tasks to the clients in each smart grid area; Step 2: Each client performs coarse-grained labeling on the collected local data according to the training task and then performs feature extraction to obtain a global prompt pool and upload it to the central server; Step 3, the central server selects a global prompt pool uploaded by any client as a student pool, uses the student pool to perform knowledge distillation on all global prompt pools, obtains an aggregated global prompt pool and distributes it to each client; Step 4, each client performs similarity determination on the local data collected in real time according to the aggregated global prompt pool to obtain a determination result, and inputs the local data collected in real time into the initial model, and trains the initial model according to the characteristics of the local smart grid area and the determination result to obtain a local model; Step 5, each client inputs the model parameters of the local model into the central server for aggregation to obtain global model parameters, and uses the global model parameters to update the model parameters of the initial model to obtain a global fault detection model for fault detection of the target smart grid; In step 6, the central server distributes the model parameters of the global fault detection model to the clients in each smart grid area, optimizes the local model of each client and uses it as the initial model, and returns to execute step 2 until the training is stopped when each client stops collecting new data.

2. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 1, characterized in that: The step 2 comprises: Each client marks the local data to obtain local data with coarse-grained prompts; Preset the pre-trained Vision Transformer model as the base model in each client, and build an initialized global prompt pool for each client; Inputting the local data with coarse-grained prompts into the basic model for feature extraction to obtain embedded features, and calculating the input key of each data in the local data with coarse-grained prompts through the basic model; Calculating the similarity between the key of each prompt in the initialized global prompt pool and each input key; Selecting indexes of multiple prompts whose similarity is greater than a preset threshold, and concatenating the prompts corresponding to the indexes with the embedded features to obtain concatenated embedded features; The basic model is trained for classification using the concatenated embedded features to obtain a global prompt pool and upload it to the central server.

3. According to claim 2, the method for optimizing the training of a smart grid fault detection model based on federated learning is characterized in that: The expression for selecting multiple prompts whose similarity is greater than a preset threshold is: in, represents the selected prompt set, Represents the distance function, which is used to calculate the similarity. The smaller the distance, the greater the similarity. Representation data The input key, Indicates A reminder key, Indicates the number of selections.

4. The method for optimizing the training of a smart grid fault detection model based on federated learning according to claim 3 is characterized in that: Each client performs similarity determination on the local data collected in real time based on the aggregated global prompt pool, and obtains the determination results, including: Each client calculates the similarity value between the features of the local data currently input and the features of the local data previously input. The calculation expression of the similarity value is: in, Represents the similarity value, Indicates the characteristics of the local data currently input. Represents the characteristics of the local data input last time; A similarity threshold is set. When the similarity value is greater than or equal to the similarity threshold, the result of the determination is that the local data currently input is repeated with the local data previously input; When the similarity value is less than the similarity threshold, the determination result is that the local data currently input is non-duplicate data.

5. The method for optimizing the training of a smart grid fault detection model based on federated learning according to claim 4, characterized in that: Before each client inputs the model parameters of the local model into the central server for aggregation, the method further includes: Gaussian noise is added to the model parameters of the local model on each client to obtain the noisy model parameters, and the noisy model parameters are input into the central server.

6. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 5, characterized in that: The central server selects multiple similar model parameters from the uploaded model parameters for knowledge aggregation.

7. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 6, characterized in that: The loss function of the central server for knowledge aggregation is: in, represents the cross entropy loss between the predicted value and the true label, represents the divergence function, represents the predicted value of the central server, Indicates The predicted value of each client, , represents the hyperparameters of the central server, represents the KL divergence, , Respectively represent The predicted probabilities of the clients and the central server.

8. The method for training and optimizing a smart grid fault detection model based on federated learning according to claim 7, characterized in that: The loss function for optimizing the local model of each client is: in, , Represents the hyperparameters of the client.

9. The method for optimizing the training of a smart grid fault detection model based on federated learning according to claim 8, characterized in that: The model parameters include: Model weights, biases, and key metrics during training.

Images