Fault diagnosis method and system based on hierarchical federated learning

Abstract

The application provides a hierarchical federated learning-based fault diagnosis method and system, and relates to the field of fault detection; the method divides a client local model into a feature extractor model used for extracting common features between client data, a feature classifier model used for extracting private features of the client data, and a feature reconstructor model used for restoring the extracted common features to original data, a training process only jointly trains feature extractor model parameters at the client and the server, and the feature classifier model and the feature reconstructor model are locally trained at the client; a trained feature extractor model and a client private feature classifier model form a local prediction model of the client to perform prediction; and the application can solve the technical problem of low model prediction accuracy caused by heterogeneous client data.

Description

Technical Field

[0001] This invention relates to the field of fault detection technology, and specifically to a fault diagnosis method and system based on hierarchical federated learning. Background Technology

[0002] In the field of equipment fault diagnosis, traditional data-driven machine learning methods can distinguish different types of equipment faults and exhibit excellent discrimination performance. However, most machine learning methods rely on large amounts of high-quality training data, i.e., a large number of equipment fault sample data. In real-world industrial scenarios, it is difficult for a single enterprise to obtain sufficient fault type data. The intuitive idea is for multiple enterprises to collaborate on centralized learning. However, since the operational data of enterprise equipment reflects, to some extent, the enterprise's production capacity and other private data, and uploading all the equipment operational data to a central server would lead to significant communication overhead, privacy protection and communication costs are increasingly attracting attention and becoming a driving factor for enterprises to participate in collaborative learning. Federated learning is a special distributed machine learning method that allows the private data of each enterprise to remain in the local storage center. It is a machine learning method in which enterprises (clients) and the central server only communicate model parameters with each other. Moreover, the model parameters can be protected using compression mechanisms, secure multi-party computation, differential privacy, and other technologies, which greatly protects user privacy.

[0003] Federated learning frameworks are increasingly becoming the framework for joint learning among different enterprises. In theory, the traditional workflow of federated learning is as follows: (1) The central server shares the initial model with each client; (2) Each client trains the model on its local data and sends the trained weights back to the central server; (3) The central server updates its global model using the weights from each client; (4) This process is repeated until the global model of federated learning converges; (5) The trained global model is sent to each client; (6) Each client uses the global model as the starting model, which can be used for fault diagnosis of the incoming data.

[0004] While federated learning frameworks offer a solution for collaborative equipment fault diagnosis among different enterprises, they also present a significant challenge: data heterogeneity. In centralized learning, each client uploads its local dataset to the central server, creating a shared training set. However, in federated learning, clients use their local datasets for model training, naturally leading to heterogeneity due to the different environments of each client. This data heterogeneity can negatively impact client-side model performance, and currently, most methods do not adequately address this issue within federated learning frameworks.

[0005] Both the federated learning-based fault diagnosis method disclosed in patent CN114662618B and the federated learning-based smart meter fault diagnosis method disclosed in patent CN111537945B fail to consider the issue of non-independent and identically distributed data during the federated learning process. Since data from different enterprises exhibits non-independent and identically distributed characteristics, this affects the accuracy of model classification. Therefore, there is a current need for a federated learning-based fault diagnosis method under heterogeneous data conditions to improve fault prediction accuracy in situations with heterogeneous client-side data. Summary of the Invention

[0006] The purpose of this invention is to provide a fault diagnosis method and system based on hierarchical federated learning. This method can effectively solve the technical problem of low accuracy in fault diagnosis and prediction of federated equipment under heterogeneous client data conditions, and provide new ideas for subsequent personalized federated learning research and related engineering applications, so as to apply federated learning to more practical scenarios.

[0007] To achieve the above objectives, the present invention proposes the following technical solution: a fault diagnosis method based on hierarchical federated learning, applied to a client, comprising:

[0008] For any client, a local model for the client is built in layers; the local model includes three parts: a feature extractor model, a feature classifier model, and a feature reconstructor model, and the feature extractor model and the feature classifier model are combined to form a client classification model, and the feature extractor model and the feature reconstructor model are combined to form a client reconstruction model;

[0009] For any client, receive the global feature extractor model parameters and the client update round broadcast by the server;

[0010] Based on the client's local data and the parameters of the global feature extractor model it receives, as well as the client's update rounds, train the corresponding client classification model and client reconstruction model;

[0011] Upload the updated feature extractor model parameters from the local model training on each client to the server so that the server can perform weight aggregation, obtain the updated global feature extractor model parameters, and broadcast them.

[0012] Repeat the local training and update process on the client side until all feature extractor models in the client converge or reach the global feature extractor model accuracy preset on the server side;

[0013] The client classification model trained for each client is used as its local prediction model to diagnose faults in the input data.

[0014] Furthermore, during the local update training of the client classification model and the client reconstruction model, the client classification model is optimized using classification loss and the client reconstruction model is optimized using reconstruction loss, so that the client's feature extractor is optimized twice.

[0015] Furthermore, we define a federated learning system with K clients in its network nodes, each client having its corresponding dataset. Where, N k This indicates the amount of data in the dataset; the K clients jointly train a deep learning model, and the datasets of different clients are heterogeneous; that is, for any i ≠ j, there exists a relation

[0016] The client-side fault classification task is defined to include M classes. Cross-entropy loss is used for fault classification tasks, and mean squared error loss is used for reconstruction tasks.

[0017] For client k, the classification loss is... and reconstruction loss In order:

[0018]

[0019] in, Indicates the category y to determine i Is it the same as category m? If the same, the value is 1; otherwise, it is 0. m This represents the probability that the softmax function predicts the class as m-th.

[0020]

[0021] in, This represents the original data after reconstruction.

[0022] Furthermore, we define the model parameters of the feature extractor model for client k in the t-th round of update as follows: The model parameters of the feature classifier model are: The model parameters of the feature reconstructor model are: The corresponding update formulas for each model are as follows:

[0023]

[0024]

[0025]

[0026] Where, η F η C and η RThese represent the learning rates of the feature extractor model, feature classifier model, and feature reconstructor model during optimization, respectively.

[0027] This invention also provides a fault diagnosis method based on hierarchical federated learning, applied to the server side, including:

[0028] Initialize the global feature extractor model parameters, and broadcast the initialized global feature extractor model parameters and the client update rounds;

[0029] The system receives the feature extractor model parameters uploaded by each client, which are trained and updated based on local data. Each client's local model consists of three parts: a feature extractor model, a feature classifier model, and a feature reconstructor model. The feature extractor model and the feature classifier model are combined to form the client classification model, and the feature extractor model and the feature reconstructor model are combined to form the client reconstruction model.

[0030] The federated averaging algorithm is used to aggregate the weights of the feature extractor model parameters uploaded by each client, and then the global feature extractor model parameters are updated and broadcast. This allows each client to repeatedly execute the local model training and update process based on the updated global feature extractor model parameters until the feature extractor models in all clients converge or reach the global feature extractor model accuracy preset by the server. In turn, each client uses its corresponding trained client classification model to diagnose faults in the input data.

[0031] Another technical solution of the present invention is to provide a fault diagnosis system based on hierarchical federated learning, the system comprising:

[0032] The module is used to build a local model for any client in layers. The local model includes three parts: a feature extractor model, a feature classifier model, and a feature reconstructor model. The feature extractor model and the feature classifier model are combined to form a client classification model, and the feature extractor model and the feature reconstructor model are combined to form a client reconstruction model.

[0033] The receiving module is used to receive the global feature extractor model parameters and the client update round broadcast by the server for any client.

[0034] The training and update module is used to train the corresponding client classification model and client reconstruction model based on the client's local data, the parameters of the global feature extractor model it receives, and the client update rounds.

[0035] The upload module is used to upload the updated feature extractor model parameters from the local model training of each client to the server so that the server can perform weight aggregation, obtain the updated global feature extractor model parameters, and broadcast them.

[0036] The loop module is used to repeatedly execute the local training and update process on the client until all feature extractor models in the client converge or reach the global feature extractor model accuracy preset on the server.

[0037] The fault diagnosis module is used to diagnose faults in the input data by using the client classification model trained by each client as its local prediction model.

[0038] Furthermore, when the training and update module trains the client classification model and the client reconstruction model, it uses classification loss to optimize the client classification model and reconstruction loss to optimize the client reconstruction model, so that the client's feature extractor is optimized twice.

[0039] Furthermore, the system also includes:

[0040] The configuration module is used to configure a federated learning system, in which there are K clients in the network nodes, and each client has its corresponding dataset. Where, N k This indicates the amount of data in the dataset; the K clients jointly train a deep learning model, and the datasets of different clients are heterogeneous; that is, for any i ≠ j, there exists a relation

[0041] Define a computation module to define the client's fault classification task, which includes M classes. Cross-entropy loss is used for fault classification, while mean squared error loss is used for reconstruction.

[0042] For client k, the classification loss is... and reconstruction loss The calculations are as follows:

[0043]

[0044] in, Indicates the category y to determine i Is it the same as category m? If the same, the value is 1; otherwise, it is 0. m This represents the probability that the softmax function predicts the class as m-th.

[0045]

[0046] in, This represents the original data after reconstruction.

[0047] Furthermore, the update process of the training update module, which uses classification loss to optimize the client classification model and reconstruction loss to optimize the client reconstruction model, is as follows:

[0048] Define the model parameters of the feature extractor model for client k in the t-th round of update as follows: The model parameters of the feature classifier model are: The model parameters of the feature reconstructor model are: The corresponding update formulas for each model are as follows:

[0049]

[0050]

[0051]

[0052] Where, η F η C and η R These represent the learning rates of the feature extractor model, feature classifier model, and feature reconstructor model during optimization, respectively.

[0053] The present invention also proposes an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor runs the computer program, it implements the fault diagnosis method based on hierarchical federated learning as described above.

[0054] As can be seen from the above technical solutions, the technical solutions of the present invention have achieved the following beneficial effects:

[0055] This invention discloses a fault diagnosis method and system based on hierarchical federated learning. The method implements a training process for both the server and client sides. Specifically, it hierarchically divides each client's local model into three layers: a feature extractor, a classifier, and a reconstructor. The server and each client share the initial parameters of the feature extractor model. Each client trains its model on its local data and sends the trained feature extractor model weights back to the server. The server updates its global feature extractor model parameters using the weights from each client. This process is repeated until the accuracy of each client's feature extractor model on the test set converges or reaches the server's preset global feature extractor model accuracy. The trained global feature extractor model parameters are then sent to each client. Each client uses the global feature extractor combined with its own trained classifier as a local prediction model to diagnose faults in the input data.

[0056] The method and system of this invention avoid using the traditional whole client model for federated learning. Instead, they use a feature extractor as the structure for target optimization in federated learning, which effectively solves the problem of data heterogeneity in federated learning systems, significantly improves model testing accuracy, and enables federated learning to be applied to more real-world scenarios.

[0057] It should be understood that all combinations of the foregoing concepts and the additional concepts described in more detail below can be considered part of the inventive subject matter of this disclosure, provided that such concepts do not contradict each other.

[0058] The foregoing and other aspects, embodiments, and features of the teachings of the present invention will be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the invention, such as features and / or beneficial effects of exemplary embodiments, will become apparent from the following description or may be learned through practice of specific embodiments according to the teachings of the present invention. Attached Figure Description

[0059] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in the various figures may be denoted by the same reference numeral. For clarity, not every component is labeled in each figure. Embodiments of various aspects of the invention will now be described by way of example and with reference to the accompanying drawings, wherein:

[0060] Figure 1 This is the network structure of the federated learning system in this embodiment of the invention;

[0061] Figure 2 This is a flowchart illustrating the overall process of the device fault diagnosis method based on hierarchical federated learning, as described in an embodiment of the present invention.

[0062] Figure 3 This is a flowchart illustrating the device fault diagnosis method based on hierarchical federated learning, as described in an embodiment of the present invention.

[0063] Figure 4 This describes the specific structure of the client-side local model disclosed in an embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art to which this invention pertains.

[0065] The terms "first," "second," and similar words used in the specification and claims of this patent application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular forms of "an," "a," or "the," etc., do not indicate a quantity limitation, but rather indicate the presence of at least one. Terms such as "comprising" or "including" mean that the element or object preceding "comprising" encompasses the features, integrals, steps, operations, elements, and / or components listed following "comprising" or "including," and do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0066] To address the problem of data heterogeneity in existing federated learning technologies, which leads to low prediction accuracy of client-side models within federated learning systems and fails to meet the needs of industrial applications, this invention provides a fault diagnosis method and system based on hierarchical federated learning. This effectively solves the data heterogeneity problem in federated learning systems and improves model prediction accuracy.

[0067] The following detailed description of the fault diagnosis method and system based on hierarchical federated learning of the present invention, with reference to the embodiments shown in the accompanying drawings, will further illustrate this invention.

[0068] like Figure 1 The federated learning system shown includes a central server and several clients connected to the central server. The fault diagnosis method based on hierarchical federated learning proposed in this invention is applied to this system.

[0069] To achieve fault diagnosis under heterogeneous data conditions in the aforementioned system, the above method faces the following challenges: (1) how to extract similar features between client data under heterogeneous client data conditions; (2) how to ensure the effectiveness of similar features extracted from each client; and (3) how to determine the client model structure optimized by federated learning. Therefore, this invention first constructs a local model for the client in a hierarchical manner, and then uses only a part of the local model to obtain the common features of all clients for training and updating, ultimately achieving fault diagnosis under heterogeneous data conditions.

[0070] like Figure 2As shown in the embodiment, the overall process of the fault diagnosis method based on hierarchical federated learning disclosed includes the following steps: hierarchically building a local model on the client side, initializing the global feature extractor model parameters, local training on the client side, aggregation on the server side, and iterative training until the client model converges or the global model reaches a preset accuracy. Finally, the classification model trained on the client side is used for fault diagnosis. The above process establishes a hierarchical local model on the client side at the beginning of the method, allowing the parts of the local model involving common features of all clients to participate in the iterative training and updates on the server side. This solves the data heterogeneity problem in federated learning systems and improves the fault diagnosis accuracy of models within each client.

[0071] Combination Figure 3 As shown, when the fault diagnosis method based on hierarchical federated learning disclosed in the embodiments is applied to a client, it specifically includes the following steps:

[0072] Step S102: For any client, build a local model for the client in layers; the local model includes three parts: feature extractor model, feature classifier model, and feature reconstructor model, and the feature extractor model and feature classifier model are combined to form a client classification model, and the feature extractor model and feature reconstructor model are combined to form a client reconstruction model;

[0073] The main task of the feature extractor model is to extract the common features of the data from each client, while the main task of the feature classifier model is to extract the personalized features of the data from each client. To decouple the feature extractor model and the feature classifier model, when the feature extractor model extracts the data features from the client, a feature reconstructor model is introduced to reconstruct the data features from the client back to the original data, thereby strengthening the correlation between the data features extracted by the feature extractor model and the original data. Furthermore, in the federated learning framework of this scheme, the feature extractor model is used as the structure for target optimization.

[0074] In specific implementation, the structures of the feature extractor model, feature reconstructor model, and feature classifier model are as follows: Figure 4As shown. First, before model training, the fault data is divided into 1024×1 one-dimensional data. The feature extractor consists of two one-dimensional convolutional layers, two one-dimensional pooling layers, and two ReLU activation functions. Specifically, the 1024×1 fault data first passes through a one-dimensional convolutional layer and a ReLU activation function (the ReLU activation function does not change the feature dimension) to output 256×32 data features, then passes through a one-dimensional pooling layer to output 128×32 data features, then passes through another one-dimensional convolutional layer and a ReLU activation function to output 128×64 data features, and finally passes through a one-dimensional pooling layer to output 64×64 data features. The feature reconstruction model consists of two one-dimensional convolutional layers, two one-dimensional pooling layers, two LeakyReLU activation functions, and a reshape layer. Specifically, the 64×64 data features output by the feature extractor model first pass through a one-dimensional convolutional layer and a LeakyReLU activation function (the LeakyReLU activation function does not change the feature dimension) to output 356×32 data features. The data features are first processed as follows: 128 × 16 × 128 data features are then passed through a one-dimensional pooling layer to output 16 × 128 data features; then through a one-dimensional convolutional layer and a Leaky ReLU activation function to output 8 × 256 data features; then through another one-dimensional pooling layer to output 4 × 256 data features; finally, a reshape layer transforms the data features into the dimensions of the initial fault data. The feature classifier model consists of a one-dimensional convolutional layer, a one-dimensional pooling layer, two linear layers, two ReLU activation functions, and a Flatten layer. Specifically, the 64 × 64 data features output by the feature extractor model are first processed through a one-dimensional convolutional layer and a ReLU activation function to output 32 × 64 data features; then through a one-dimensional pooling layer to output 16 × 64 data features; then through a Flatten layer to flatten the data features into 1024 data features; then through a linear layer and a ReLU activation function to output 128 data features; finally, through a linear layer to output 10 data features, which are then classified.

[0075] To address the challenges (1), this solution builds a feature extractor model on each client to extract similar features between client data and a feature classifier model to extract private features of client data and complete the classification task. To address the challenges (2), this solution builds a feature reconstructor model on each client, which not only alleviates the coupling effect between the client's feature extractor model and the feature classifier model, but also strengthens the connection between the data features extracted by the feature extractor model and the original data, thus improving the effectiveness of the features. To address the challenges (3), it is clear that the traditional optimization of the client's full model structure is no longer suitable for this solution. Therefore, this application determines that the client's feature extractor model is used as the client model structure for federated learning optimization. In summary, the local model built on the client in this invention can fully solve the data heterogeneity problem of the federated learning system and improve the testing accuracy of the local prediction model.

[0076] Step S104: For any client, receive the global feature extractor model parameters and the client update round broadcast by the server; initially, the global feature extractor model parameters are also initialized by the server before being broadcast.

[0077] Step S106: Based on the client's local data and the parameters of the global feature extractor model it receives, as well as the client's update rounds, train the corresponding client classification model and client reconstruction model; that is, not only train the feature extractor model, but also train the feature extractor model, feature classifier model, and feature reconstructor model locally while ensuring the integrity of the network.

[0078] Step S108: Upload the updated feature extractor model parameters from the local model training of each client to the server so that the server can perform weight aggregation, obtain the updated global feature extractor model parameters, and broadcast them.

[0079] Step S110: Repeat the local training and update process on the client until all feature extractor models in the client converge or reach the global feature extractor model accuracy preset on the server.

[0080] The two steps described above enable the client to send the trained feature extractor model parameters to the central server and then receive the global feature extractor model parameters back from the central server.

[0081] Step S112: Use the client classification model trained by each client as its local prediction model to perform fault diagnosis on the input data.

[0082] Another embodiment of the present invention provides a fault diagnosis method based on hierarchical federated learning applied on the server side, the method comprising:

[0083] Initialize the global feature extractor model parameters and broadcast the initialized global feature extractor model parameters and the number of client update rounds; optionally, in this embodiment, the number of client update rounds is fixed and set to 5 times.

[0084] The system receives the feature extractor model parameters uploaded by each client, which are trained and updated based on local data. Each client's local model consists of three parts: a feature extractor model, a feature classifier model, and a feature reconstructor model. The feature extractor model and the feature classifier model are combined to form the client classification model, and the feature extractor model and the feature reconstructor model are combined to form the client reconstruction model.

[0085] The federated averaging algorithm is used to aggregate the weights of the feature extractor model parameters uploaded by each client, and then the global feature extractor model parameters are updated and broadcast. This allows each client to repeatedly execute the local model training and update process based on the updated global feature extractor model parameters until the feature extractor models in all clients converge or reach the global feature extractor model accuracy preset by the server. In turn, each client uses its corresponding trained client classification model to diagnose faults in the input data.

[0086] Optional, such as Figure 1 The client sends its locally trained feature extractor model parameters to the central server, and the central server simultaneously records the information and upload time of each client.

[0087] In the aforementioned fault diagnosis methods applied to the client or server sides, to mitigate the impact of data heterogeneity on the model during federated learning, the traditional entire client model is no longer used for federated learning. Instead, only the feature extractor model is used as the structure for objective optimization in federated learning, effectively solving the data heterogeneity problem in federated learning systems and focusing on the common features of data from various clients. Furthermore, this invention first trains the feature extractor model on the training set according to the federated learning method, adjusts the model's hyperparameters on the validation set until the model's accuracy converges on the test set; finally, the trained feature extractor model and the local feature classifier models of each client are combined to form the client's local prediction model for application, significantly improving prediction accuracy.

[0088] As an optional implementation, in the above method, when the client classification model and the client reconstruction model are trained locally, the client classification model is optimized using classification loss and the client reconstruction model is optimized using reconstruction loss, thereby enabling the client's feature extractor to be optimized twice.

[0089] For example, for Figure 1 The federated learning system shown has K clients in its network nodes, each client having its corresponding dataset. Where, N k This indicates the amount of data in the dataset; the K clients jointly train a deep learning model, and the datasets of different clients are heterogeneous; that is, for any i ≠ j, there exists a relation

[0090] The client-side fault classification task is defined to include M classes. Cross-entropy loss is used for fault classification tasks, and mean squared error loss is used for reconstruction tasks.

[0091] For client k, the classification loss is... and reconstruction loss In order:

[0092]

[0093] in, Indicates the category y to determine i Is it the same as category m? If the same, the value is 1; otherwise, it is 0. m This represents the probability that the softmax function predicts the class as m-th.

[0094]

[0095] in, This represents the original data after reconstruction.

[0096] Define the model parameters of the feature extractor model for client k in the t-th round of update as follows: The model parameters of the feature classifier model are: The model parameters of the feature reconstructor model are: The update formulas for each model are as follows:

[0097]

[0098]

[0099]

[0100] Where, η F η C and η R These represent the learning rates of the feature extractor model, feature classifier model, and feature reconstructor model during optimization, respectively.

[0101] In addition, whether the feature extractor model converges in accuracy on the test set is calculated using the following formula:

[0102]

[0103] Where, ntesting n represents the amount of sample data used in model testing. correct This represents the amount of sample data in the model test where the test results are consistent with the actual results.

[0104] In this embodiment, an electronic device is also provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor runs the computer program, it implements the fault diagnosis method based on hierarchical federated learning as described above.

[0105] The aforementioned computer program may run on a processor or be stored in a computer-readable storage medium, including permanent and non-permanent, removable and non-removable media. Information storage can be achieved by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, storage media does not include transient media, such as modulated data signals and carrier waves.

[0106] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented using different modules, and different steps can be implemented using different modules.

[0107] For example, this embodiment provides such a system, namely a fault diagnosis system based on hierarchical federated learning. This system includes: a building module, used to build a local model for any client in a hierarchical manner; the local model includes three parts: a feature extractor model, a feature classifier model, and a feature reconstructor model, wherein the feature extractor model and the feature classifier model are combined to form a client classification model, and the feature extractor model and the feature reconstructor model are combined to form a client reconstruction model; a receiving module, used to receive the global feature extractor model parameters and the client update round broadcast by the server for any client; and a training update module, used to update the system based on the client's local data. The system includes a global feature extractor model parameter and client update rounds received by the client, and a client classification model and client reconstruction model trained accordingly. An upload module uploads the updated feature extractor model parameters from each client's local model training to the server, allowing the server to perform weight aggregation, obtain the updated global feature extractor model parameters, and broadcast them. A loop module repeatedly executes the client's local training and update process until all client-side feature extractor models converge or reach the server's preset global feature extractor model accuracy. A fault diagnosis module uses the client classification model trained by each client as its local prediction model to diagnose faults in the incoming data.

[0108] The system is used to implement the steps of the fault diagnosis method based on hierarchical federated learning disclosed in the above embodiments, and therefore, the steps already described will not be repeated here.

[0109] For example, when the training update module trains the client classification model and the client reconstruction model, it uses classification loss to optimize the client classification model and reconstruction loss to optimize the client reconstruction model, so that the feature extractor of the client is optimized twice.

[0110] For example, the system also includes:

[0111] The configuration module is used to configure a federated learning system, in which there are K clients in the network nodes, and each client has its corresponding dataset. Where, N k This indicates the amount of data in the dataset; the K clients jointly train a deep learning model, and the datasets of different clients are heterogeneous; that is, for any i ≠ j, there exists a relation

[0112] Define a computation module to define the client's fault classification task, which includes M classes. Cross-entropy loss is used for fault classification, while mean squared error loss is used for reconstruction.

[0113] For client k, the classification loss is... and reconstruction loss The calculation formula is:

[0114]

[0115] in, Indicates the category y to determine i Is it the same as category m? If the same, the value is 1; otherwise, it is 0. m This represents the probability that the softmax function predicts the class as m-th.

[0116]

[0117] in, This represents the original data after reconstruction.

[0118] For example, based on the definitions and calculation formulas of the setting module and the definition calculation module, the update process of the training and update module, which uses classification loss to optimize the client classification model and reconstruction loss to optimize the client reconstruction model, is as follows:

[0119] Define the model parameters of the feature extractor model for client k in the t-th round of update as follows: The model parameters of the feature classifier model are: The model parameters of the feature reconstructor model are: The corresponding model update formula is as follows:

[0120]

[0121]

[0122]

[0123] Where, η F η C and η R These represent the learning rates of the feature extractor model, feature classifier model, and feature reconstructor model during optimization, respectively.

[0124] This invention discloses a fault diagnosis method and system based on hierarchical federated learning. On the one hand, it designs a hierarchical local model on the client side, defining the local model as a feature extractor model for extracting common features among client data, a feature classifier model for extracting private features from client data, and a feature reconstructor model for restoring the extracted common features to the original data. On the other hand, it addresses the problem of extracting similar features among client data under heterogeneous client data conditions and ensuring the effectiveness of similar features extracted from each client by uploading the training parameters of the feature extractor model on the client side to the server side, while the feature classifier model and the feature reconstructor model are trained locally. This fully addresses the technical problem of low accuracy in federated equipment fault diagnosis prediction under heterogeneous client data conditions, providing new ideas for subsequent personalized federated learning research and related engineering applications, and applying federated learning to more practical scenarios.

[0125] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A fault diagnosis method based on hierarchical federated learning, characterized in that, The application is applied to a client, comprising: For any client, a local model of the client is built in layers; the local model comprises three parts of a feature extractor model, a feature classifier model and a feature reconstructor model, and the feature extractor model and the feature classifier model combine to form a client classification model, and the feature extractor model and the feature reconstructor model combine to form a client reconstruction model; For any client, global feature extractor model parameters broadcast by a server end and a client update round are received; According to the local data of the client and the global feature extractor model parameters and the client update round received by the client, a corresponding client classification model and a client reconstruction model are trained; Feature extractor model parameters of local model training and update of each client are uploaded to the server end, so that the server end performs weight aggregation, obtains updated global feature extractor model parameters and broadcasts; The client local training and update process is repeatedly executed until the feature extractor model in all clients converges or the preset global feature extractor model accuracy of the server end is reached; The corresponding trained client classification model of each client is used as a local prediction model of the client to perform fault diagnosis on incoming data.

2. The hierarchical federated learning based fault diagnosis method according to claim 1, characterized in that, When the client classification model and the client reconstruction model are locally updated and trained, the client classification model is optimized by using a classification loss, and the client reconstruction model is optimized by using a reconstruction loss, so that the feature extractor of the client is optimized twice.

3. The fault diagnosis method based on layered federated learning according to claim 2, characterized in that: A federated learning system is set, and K clients exist in the network nodes of the federated learning system, and each client has its corresponding data set Wherein, N k represents the amount of data owned by the data set; the K clients jointly train a deep learning model, and the data sets between different clients are heterogeneous; that is, for any i≠j, there is a relationship the fault classification task of the client is defined to contain M classes, a cross-entropy loss is used for the fault classification task, and a mean square error loss is used for the reconstruction task; Then for client k, its classification loss and reconstruction loss are in turn given by wherein, represents a judgment class y i whether the same as class m, same is 1, different is 0; p m represents the probability of the softmax function prediction as the mth class; wherein, represents the original data after reconstruction.

4. The hierarchical federated learning based fault diagnosis method according to claim 3, characterized in that, The model parameters of the feature extractor model when the client k updates in the tth round are defined as The model parameters of the feature classifier model are The model parameters of the feature reconstructor model are The corresponding model update formulas are as follows: where η F , η C and η R denote the learning rate of the feature extractor model, the feature classifier model and the feature reconstructor model, respectively, at the time of optimization.

5. A fault diagnosis method based on hierarchical federated learning, characterized in that, The application is applied to a server end, comprising: Global feature extractor model parameters are initialized, and the initialized global feature extractor model parameters and a client update round are broadcast; Feature extractor model parameters of local model training and update according to local data of each client are received, wherein the local model of each client comprises three parts of a feature extractor model, a feature classifier model and a feature reconstructor model built in layers, and the feature extractor model and the feature classifier model combine to form a client classification model, and the feature extractor model and the feature reconstructor model combine to form a client reconstruction model; The feature extractor model parameters updated and uploaded by each client are aggregated in weight by using a federated average algorithm, the global feature extractor model parameters are updated and broadcast, so that each client repeatedly executes the local model training and update process according to the updated global feature extractor model parameters until the feature extractor model in all clients converges or the preset global feature extractor model accuracy of the server end is reached; and then, each client uses the corresponding trained client classification model to perform fault diagnosis on incoming data.

6. A fault diagnosis system based on layered federated learning, characterized in that, It comprises: The building module is configured to build a local model of the client in layers for any client; the local model comprises three parts of a feature extractor model, a feature classifier model and a feature reconstructor model, and the feature extractor model and the feature classifier model combine to form a client classification model, and the feature extractor model and the feature reconstructor model combine to form a client reconstruction model; The receiving module is configured to receive, for any client, global feature extractor model parameters broadcast by the server and a client update round; The training update module is configured to train the corresponding client classification model and the client reconstruction model according to the local data of the client, the global feature extractor model parameters received by the client and the client update round; The uploading module is configured to upload feature extractor model parameters of the local model of each client to the server for weight aggregation to obtain updated global feature extractor model parameters and broadcast the updated global feature extractor model parameters; The cycle module is configured to repeat the client local training and updating process until the feature extractor model in all clients converges or reaches a preset global feature extractor model accuracy of the server. The fault diagnosis module is configured to use the trained client classification model of each client as a local prediction model to perform fault diagnosis on incoming data.

7. The layered federated learning based fault diagnosis system of claim 6, wherein, When the training update module trains the client classification model and the client reconstruction model, the client classification model is optimized by using a classification loss, and the client reconstruction model is optimized by using a reconstruction loss, so that the feature extractor of the client is optimized twice.

8. The layered federated learning based fault diagnosis system of claim 7, wherein, Further comprising: The application discloses a federated learning system and a method for training a deep learning model Wherein, N k represents the amount of data owned by the data set; the K clients collectively train a deep learning model, and the data sets between different clients are heterogeneous; that is, for any i≠j, there is a relationship The definition calculation module is configured to define that the fault classification task of the client comprises M classes, and use a cross-entropy loss for the fault classification task and use a mean square error loss for the reconstruction task. Then for client k, its classification loss and reconstruction loss are computed as follows: wherein, represents a judgment class y i whether the same as class m, same is 1, different is 0; p m represents the probability of the softmax function predicting the mth class; wherein represents the original data after reconstruction.

9. The layered federated learning based fault diagnosis system of claim 8, wherein, The updating process of the training update module for optimizing the client classification model by using the classification loss and optimizing the client reconstruction model by using the reconstruction loss is as follows: The model parameters of the feature extractor model when the client k updates in the tth round are defined as The model parameters of the feature classifier model are The model parameters of the feature reconstructor model are The corresponding model update formulas are as follows: where η F , η C , and η R denote the learning rates of the feature extractor model, the feature classifier model, and the feature reconstructor model, respectively, at the time of optimization.

10. An electronic device, comprising: The computer program stored in the memory and executable on the processor, when the processor executes the computer program, implements the fault diagnosis method based on the hierarchical federated learning according to any one of claims 1-4.

Images