Transformer fault diagnosis method, terminal and medium based on personalized federated learning

Abstract

The present invention relates to the technical field of transformer fault diagnosis, and discloses a transformer fault diagnosis method, terminal and medium based on personalized federated learning. The method obtains heterogeneous transformer data of N transformer sites, and respectively serves as local private data sets of N clients; uses a central server to initialize the training environment of federated learning, generates an initialized global model and configures a fusion network; the central server connects all clients, and the clients preprocess their respective heterogeneous transformer data; the clients initialize local personalized models and federated models; the central server selects different clients to participate in personalized federated learning training after multiple rounds, and obtains an optimal transformer fault diagnosis model; the real-time heterogeneous transformer data of the target transformer site is input into the corresponding optimal transformer fault diagnosis model, and the fault diagnosis result of the target transformer site is output; the present invention improves the accuracy of transformer fault prediction and diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of transformer fault diagnosis, and in particular to a transformer fault diagnosis method, a terminal and a medium based on personalized federated learning. Background Art

[0002] As an important equipment in the substation, the transformer is responsible for voltage conversion and power transmission between different electrical equipment. Its operating status has a direct impact on the safe and stable operation of the entire power grid system. If there is no timely diagnosis and warning of transformer failure, it will not only increase the maintenance cost, but even cause major accidents, causing serious harm to the entire power grid system. Therefore, it is of great significance to monitor the operating status of the transformer, which can prevent problems before they happen, discover potential hidden dangers and deal with them as soon as possible.

[0003] At present, compared with the traditional transformer fault diagnosis method, the combination of artificial intelligence and traditional preventive tests to diagnose the abnormal operation status of transformers has gradually become a research hotspot for scholars at home and abroad. However, due to the dependence of AI algorithms on data, they face two major challenges: first, in the transformer fault diagnosis test, there is a small amount of sample data and poor data quality; second, regulatory agencies strengthen data protection and prohibit information sharing, and most data exist in the form of isolated islands. Although transformer fault diagnosis can be performed based on a small amount of effective information, the prediction accuracy is not high due to the small amount of sample data. Therefore, in order to improve the power transformer fault diagnosis effect, there is an urgent need for a method that can improve the accuracy of transformer fault prediction and diagnosis on the basis of ensuring the legitimacy and security of data. Summary of the invention

[0004] In order to solve the technical problems existing in the prior art, the present invention provides a transformer fault diagnosis method, terminal and medium based on personalized federated learning. The present invention applies the concept of local client personalization to traditional federated learning, which can ensure the good personalized performance of each client in the case of data heterogeneity, improve the accuracy of transformer fault prediction and diagnosis on the basis of ensuring the legitimacy and security of data, and complete the customization of private clients.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention discloses a transformer fault diagnosis method based on personalized federated learning, comprising steps S1 to S6.

[0007] S1. Obtain heterogeneous transformer data of N transformer sites as local private data sets of N clients respectively; the heterogeneous transformer data includes at least two types of waveform data generated when the transformer is in a working state.

[0008] S2. Use the central server to initialize the training environment of federated learning, generate an initialized global model and configure the fusion network, and the clients preprocess their respective heterogeneous transformer data.

[0009] S3. The client initializes the local personalized model and the federated model.

[0010] S4. The central server selects n clients in each round to participate in personalized federated learning training, sends the latest global model to the clients participating in the training and updates the client's federated model; the clients participating in the training start local training after receiving the latest global model, and use the dynamic regularization algorithm and local private data set to update their own local personalized model during the training process; n∈(1,N).

[0011] S5. The central server receives the updated federated models from all the clients participating in the training, updates the global model using the model aggregation algorithm, and then returns to step S4 for the next round of training until the global model converges or reaches the specified number of iterations, so that each client obtains a trained local personalized model, i.e., the optimal transformer fault diagnosis model;

[0012] S6. Input the real-time heterogeneous transformer data of the target transformer site into the corresponding optimal transformer fault diagnosis model, thereby outputting the fault diagnosis result of the target transformer site.

[0013] As a further improvement of the above solution, the types of waveform data include: transformer sound waveform data and winding vibration waveform data.

[0014] As a further improvement of the above solution, in step S2, the process of the client preprocessing the respective heterogeneous transformer data includes:

[0015] Each waveform data is serialized in the image preprocessing layer and converted into an embedding matrix. The expression formula for serialization is:

[0016] ;

[0017] In the formula, x j,i’ Indicates j The first i 'Serialized feature vector of positions; b j,i’ Indicates j The first i 'positions; F (·) represents a specific nonlinear transformation function; vec (·) means flattening the matrix into vector processing; W 0 and p j,i’ is a learnable parameter.

[0018] As a further improvement of the above solution, step S3 includes the following specific steps, namely S31~S32.

[0019] S31. In the initial round, each client receives the global model sent from the central server g , the global model g Set as client-side federation model f , and replicate the federated model f The structure to set the client's local personalization model l The shared layer in the fusion network is then used to initialize the local personalized model using the weight parameters sent by the fusion network. l The shared layers in .

[0020] S32. Initialize the federated model using different random seeds f and local personalization models l .

[0021] As a further improvement of the above solution, in step S4, when the central server sends the latest global model to the clients participating in the training, it also requests local updates; then the clients participating in the training sample several waveform data from their respective local private data sets for this local update.

[0022] During the local update, there is a low-loss subspace between the client’s local personalized model and the federated model, expressed as:

[0023] ;

[0024] In the formula, Represents the client i The corresponding low loss subspace is λ is the mixing constant; i ∈[1,n]; For Clients i The federation model parameters of For Clients i Local personalized model parameters.

[0025] As a further improvement of the above solution, in step S4, during the local training process, the client uses the following loss function to update and optimize the parameters of the federated model and the shared layer parameters of the local personalized model:

[0026] ;

[0027] ;

[0028] In the formula, represents partial derivative; LIt is the optimization target of the local personalized model; l ' is the cross entropy loss; μ is a hyperparameter used to regularize the federated model parameters; is another hyperparameter used to induce connectivity in the subspace between the local model and the federated model; is the global model parameter; cos(·) is the cosine function.

[0029] As a further improvement of the above scheme, in the first k In the communication round, the client i Download the first k -1 communication round to update the aggregation parameters , using a converged network located at a central server Generate Client i Projection matrix of the personalization layer in the local personalization model W i ;in k ∈(1,K], K is the specified number of training times; the parameter update of the fusion network is expressed as:

[0030] ;

[0031] In the formula, for t The parameters of the network are integrated at all times; for t -1 The parameters of the fusion network at the moment; is the learning rate; for t Second moment estimation of the parameters of the moment fusion network; for t -1 The first moment estimate of the parameters of the fusion network; for t Always participate in training n A collection of clients; For Clients i The number of samples in the local dataset; Indicates the parameters The gradient of T represents transpose; is the change in projection parameters after K rounds; , , is a constant; for t Moment Client i The corresponding learnable embedding vector; for t -1 moment client iThe corresponding learnable embedding vector; for t Moment Client i Embedded second moment estimation; for t -1 moment client i Embedded first moment estimation; Represents the client embedding vector gradient.

[0032] As a further improvement of the above solution, in step S5, the central server uses the model aggregation algorithm to update the expression formula of the global model as follows:

[0033] ;

[0034] In the formula, d i Represents the client i The amount of training data consumed.

[0035] The present invention also discloses a computer terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned transformer fault diagnosis method based on personalized federated learning are implemented.

[0036] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the above-mentioned transformer fault diagnosis method based on personalized federated learning are implemented.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The transformer fault diagnosis method based on personalized federated learning disclosed in the present invention allows personalized parameters to be effectively shared among clients through a fusion network mechanism, and generates a personalized self-attention layer by learning a unique embedding vector for each client. Therefore, as the number of clients increases, the method can be better generalized to new clients that have not seen data during the training process, providing the scalability of the model.

[0039] 2. The present invention selects transformer sound waveform data and winding vibration waveform data as heterogeneous transformer data. The transformer sound waveform data can reflect changes in the internal operating state of the transformer, and the winding vibration waveform data reflects the stability of the internal structure of the transformer, thereby realizing multi-angle and all-round diagnosis of transformer faults and improving the accuracy and reliability of fault diagnosis.

[0040] 3. This diagnostic method proposes a personalized federated learning framework for transformer fault diagnosis, completes model training without sharing transformer fault samples, and protects the data privacy of the power company.

[0041] 4. The personalized federated learning framework of this diagnostic method adopts a hierarchical parameter aggregation strategy to identify the fault types of power transformers. It can learn both inter-client features and intra-client features, and fully considers the heterogeneity of power transformer data between clients.

[0042] 5. This diagnostic method learns a personalized network for each client, which to some extent alleviates the impact on the accuracy of transformer fault diagnosis in the presence of data heterogeneity and realizes customized model training.

[0043] 6. The diagnostic method establishes a low-loss subspace between each client’s local model and the federated model, improving personalization performance while providing robustness to label noise, so that each client has a well-performing and calibrated personalization model.

[0044] 7. The computer terminal and computer-readable storage medium disclosed in the present invention can produce the same beneficial effects as the above-mentioned method by applying the above-mentioned diagnostic method, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The figure is a flow chart of a transformer fault diagnosis method based on personalized federated learning in an embodiment of the present invention.

[0046] Figure 2 Schematic diagram of the structure of the personalized federated learning framework in an embodiment of the present invention.

[0047] Figure 3 It is a performance diagram of transformer fault diagnosis accuracy of various methods in the embodiments of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1An embodiment of the present invention provides a transformer fault diagnosis method based on personalized federated learning. The personalized federated learning framework consists of a central server and N clients, and the N clients correspond one-to-one to N transformer sites. The fault diagnosis method includes steps, namely S1~S6.

[0050] S1. Obtain heterogeneous transformer data of N transformer sites as local private data sets of N clients respectively; the heterogeneous transformer data includes at least two types of waveform data generated when the transformer is in a working state.

[0051] In this embodiment, the types of waveform data include: transformer sound waveform data and winding vibration waveform data. In some embodiments, there may also be other types of waveform data that can reflect transformer faults.

[0052] S2. Use the central server to initialize the training environment of federated learning, generate an initialized global model and configure the fusion network to generate the weight coefficients of the personalized layer; connect all clients through the central server, and the clients preprocess their respective heterogeneous transformer data.

[0053] The personalization layer is understood as a neural network module that updates parameters through a fusion network and local training of the client, and can extract features from data.

[0054] In step S2, the process of the client preprocessing the respective heterogeneous transformer data includes:

[0055] Each waveform data is serialized in the image preprocessing layer and converted into an embedding matrix. The expression formula for serialization is:

[0056] ;

[0057] In the formula, x j,i’ Indicates j The first i 'Serialized feature vector of positions; b j,i’ Indicates j The first i 'positions; F (·) represents a specific nonlinear transformation function; vec (·) means flattening the matrix into vector processing; W 0 and p j,i’ is a learnable parameter.

[0058] The said j The first i ' position, which means jThe corresponding feature vector can be obtained by serializing the pixel points at specific positions in the divided small image area and converting them into an embedding matrix.

[0059] The image preprocessing layer is understood as follows: because the model cannot process the image directly, it needs to be serialized first.

[0060] S3. The client initializes the local personalized model and the federated model; both the local personalized model and the federated model are deep neural networks. Step S3 may include the following specific steps, namely S31~S32.

[0061] S31. In the initial round, each client receives the global model sent from the central server g , the global model g Set as client-side federation model f , and replicate the federated model f The structure to set the client's local personalization model l The shared layer in the fusion network is then used to initialize the local personalized model using the weight parameters sent by the fusion network. l The shared layers in .

[0062] The shared layer is understood as: updating parameters through a federated model, and is mainly responsible for operations such as pooling feature vectors.

[0063] S32. Initialize the federated model using different random seeds f and local personalization models l .

[0064] S4. The central server selects n clients in each round to participate in personalized federated learning training, sends the latest global model to the clients participating in the training, and uses the parameters of the sent global model to replace and update the parameters of the federated model in the selected clients; the clients participating in the training start local training after receiving the latest global model, and use the dynamic regularization algorithm and local private data set to update their own local personalized model during the training process; n∈(1,N).

[0065] In step S4, when the central server sends the latest global model to the clients participating in the training, it also requests local updates; then the clients participating in the training sample several waveform data from their respective local private data sets for this local update.

[0066] During the local update, there is a low-loss subspace between the client’s local personalized model and the federated model, expressed as:

[0067] ;

[0068] In the formula, Represents the client i The corresponding low loss subspace is λ is the mixing constant; i ∈[1,n]; For Clients i The federation model parameters of For Clients i Local personalized model parameters.

[0069] During the local training process, the client uses the following loss function to update and optimize the parameters of the federated model and the shared layer parameters of the local personalized model:

[0070] ;

[0071] ;

[0072] In the formula, represents partial derivative; L It is the optimization target of the local personalized model; l ' is the cross entropy loss; μ is a hyperparameter used to regularize the federated model parameters; is another hyperparameter used to induce connectivity in the subspace between the local model and the federated model; is the global model parameter; cos(·) is the cosine function.

[0073] In the personalized federated learning training k In the communication round, the client i Download the first k -1 communication round to update the aggregation parameters , using a converged network located at a central server Generate Client i Projection matrix of the personalization layer in the local personalization model W i ;in k ∈(1,K], K is the specified number of training times; the parameter update of the fusion network is expressed as:

[0074] ;

[0075] In the formula, for t The parameters of the network are integrated at all times; for t -1 The parameters of the fusion network at the moment; is the learning rate; for t Second moment estimation of the parameters of the moment fusion network; fort -1 The first moment estimate of the parameters of the fusion network; for t Always participate in training n A collection of clients; For Clients i The number of samples in the local dataset; Indicates the parameters The gradient of T represents transpose; is the change in projection parameters after K rounds; , , is a constant; for t Moment Client i The corresponding learnable embedding vector; for t -1 moment client i The corresponding learnable embedding vector; for t Moment Client i Embedded second moment estimation; for t -1 moment client i Embedded first moment estimation; Represents the client embedding vector gradient.

[0076] S5. The central server receives the updated federated models from all the clients participating in the training, uses the model aggregation algorithm to update the global model, and then returns to step S4 for the next round of training until the global model converges or reaches the specified number of iterations, so that each client obtains a trained local personalized model, that is, the optimal transformer fault diagnosis model.

[0077] In step S5, the central server uses the model aggregation algorithm to update the global model using the following expression formula:

[0078] ;

[0079] In the formula, d i Represents the client i The amount of training data consumed.

[0080] S6. Input the real-time heterogeneous transformer data of the target transformer site into the corresponding optimal transformer fault diagnosis model, thereby outputting the fault diagnosis result of the target transformer site.

[0081] In the present invention, the fault diagnosis result is mainly used to diagnose whether a fault occurs (normal or abnormal). Generation principle: Through training, the client already has a local personalized model that can perform fault diagnosis. The client connects the local associated transformer data to the diagnosis model in real time. The diagnosis model extracts features from the data through the local personalized model and the federated model, and then processes the data features to obtain the probability of the state of the current transformer site. When the probability of one state reaches a certain threshold, it is determined to be the corresponding state, such as 90% normal and 10% abnormal. The transformer site is determined to be normal and no fault has occurred.

[0082] In order to verify the transformer fault diagnosis method based on personalized federated learning in this embodiment, this embodiment also sets a control test, and the test conditions are: the transformer fault diagnosis model provided by the present invention and the two comparison models are trained to convergence using the training set, and then the accuracy of the converged model is tested on the test set. In addition, the consistency of the number of communication rounds, fault scenarios, computing resources, etc. must be ensured.

[0083] See also Figure 3 In the figure, method 1 is the existing federated average algorithm, method 2 is the transformer fault diagnosis method of the present invention, and method 3 is the existing federated proximal algorithm. It can be seen that the method of the present invention has a higher accuracy rate for transformer fault diagnosis, and the convergence curve is smoother.

[0084] This embodiment also provides a computer terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0085] The computer device may be a smart phone, tablet computer, laptop computer, etc. capable of executing a program. The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data. When the processor executes the program, the steps of the transformer fault diagnosis method based on personalized federated learning are implemented.

[0086] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the above-mentioned transformer fault diagnosis method based on personalized federated learning are implemented.

[0087] The computer-readable storage medium may include flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the storage medium may also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the memory is generally used to store an operating system and various application software installed on the computer device, etc. In addition, the memory may also be used to temporarily store various types of data that have been output or are to be output.

[0088] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A transformer fault diagnosis method based on personalized federated learning, characterized in that: The personalized federated learning framework consists of a central server and N clients, which correspond one-to-one to N transformer sites; each client includes a local personalized model, a federated model, and an image preprocessing layer; The local personalized model includes a personalized layer and a shared layer; the personalized layer is a neural network module that updates parameters through the fusion network and local training of the client to extract data features; The shared layer updates parameters through a federated model to pool feature vectors; the method comprises the steps of: S1. Obtain heterogeneous transformer data of N transformer sites as local private datasets of N clients respectively; The heterogeneous transformer data includes at least two types of waveform data generated when the transformer is in a working state; The types of waveform data include: transformer sound waveform data and winding vibration waveform data; S2. Initialize the training environment of federated learning using the central server, generate an initialized global model and configure the fusion network, and the client preprocesses the respective heterogeneous transformer data; In step S2, the process of the client preprocessing the respective heterogeneous transformer data includes: Each waveform data is serialized in the image preprocessing layer and converted into an embedding matrix. The expression formula for serialization is: xj,i'=W0·vec(F(bj,i'))+pj,i'; In the formula, x j,i’ b represents the serialized feature vector of the i'th position in the jth block; j,i’ represents the i'th position in the jth block; F(·) represents a specific nonlinear transformation function; vec(·) represents flattening the matrix into a vector; W0 and p j,i’ is a learnable parameter; S3. The client initializes the local personalized model and the federated model; step S3 includes the following specific steps: S31. In the initial round, each client receives the global model g sent from the central server, sets the global model g as the client's federated model f, and copies the structure of the federated model f to set the shared layer in the client's local personalized model l, and then initializes the shared layer in the local personalized model l using the weight parameters sent by the fusion network; S32. Initialize the federated model f and the local personalized model l using different random seeds; S4. The central server selects n clients in each round to participate in personalized federated learning training, sends the latest global model to the clients participating in the training, and updates the client's federated model; the clients participating in the training start local training after receiving the latest global model, and use the dynamic regularization algorithm and the local private data set to update their own local personalized model during the training process; n∈(1,N); In step S4, when the central server sends the latest global model to the clients participating in the training, it also requests local updates; then the clients participating in the training sample several waveform data from their respective local private data sets for this local update; During the local update, there is a low-loss subspace between the client’s local personalized model and the federated model, expressed as: Where W(λ) i represents the low-loss subspace corresponding to client i, λ is a mixing constant; i∈[1,n]; is the federation model parameter of client i; is the local personalized model parameter of client i; S5. The central server receives the updated federated models of all the clients participating in the training, updates the global model using the model aggregation algorithm, and then returns to step S4 for the next round of training until the global model converges or reaches the specified number of iterations, so that each client obtains a trained local personalized model, i.e., the optimal transformer fault diagnosis model; in the kth communication round of personalized federated learning training, client i downloads the aggregated parameter ξ updated in the k-1th communication round from the central server k-1 , using a converged network located at a central server Generate the projection matrix W of the personalization layer in the local personalization model of client i i ; where k∈(1,K], K is the specified number of training times; the parameter update of the fusion network is expressed as: In the formula, is the parameter of the fusion network at time t; is the parameter of the fusion network at time t-1; α is the learning rate; is the second moment estimate of the parameters of the fusion network at time t; is the first moment estimate of the parameters of the fusion network at time t-1; C t is the set of n clients participating in the training at time t; m i is the number of samples in the local dataset of client i; Indicates the parameters The gradient of ; the superscript T indicates transposition; ΔW i is the change of projection parameters after K rounds; β1, β2, ∈ are constants; is the learnable embedding vector corresponding to client i at time t; is the learnable embedding vector corresponding to client i at time t-1; is the second moment estimate of the embedding of client i at time t; is the first moment estimate of the embedding of client i at time t-1; Represents the client embedding vector z i The gradient of S6. Input the real-time heterogeneous transformer data of the target transformer site into the corresponding optimal transformer fault diagnosis model, thereby outputting the fault diagnosis result of the target transformer site.

2. The transformer fault diagnosis method based on personalized federated learning according to claim 1 is characterized in that: In step S4, during the local training process, the client uses the following loss function to update and optimize the parameters of the federated model and the shared layer parameters of the local personalized model: In the formula, represents partial derivative; L is the optimization target of the local personalized model; l' is the cross entropy loss; μ is a hyperparameter used to regularize the federated model parameters; η is another hyperparameter used to induce connectivity in the subspace between the local model and the federated model; w g is the global model parameter; cos(·) is the cosine function.

3. The transformer fault diagnosis method based on personalized federated learning according to claim 2 is characterized in that: In step S5, the central server uses the model aggregation algorithm to update the global model using the following expression formula: Where, d i Represents the amount of training data consumed by client i.

4. A computer terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the transformer fault diagnosis method based on personalized federated learning as described in any one of claims 1 to 3 are implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the transformer fault diagnosis method based on personalized federated learning as described in any one of claims 1 to 3 are implemented.

Images