A method for diagnosing imbalanced data faults based on self-balancing federated learning

Abstract

This invention discloses a fault diagnosis method for imbalanced data based on self-balancing federated learning, belonging to the technical field of artificial intelligence and edge computing. The method includes the following steps: S1, acquiring client data and monitoring for imbalance; S2, data augmentation; S3, sharing the weights of the generator and discriminator; S4, mediator training scheduling; S5, the mediator server scheduling the client to train the model; S6, the federated learning server aggregating the model parameters; S7, the aggregated model parameters are returned to the mediator server to start the next round of synchronous training; S8, the client obtains the accuracy result. This invention employs the above-mentioned fault diagnosis method for imbalanced data based on self-balancing federated learning. This method, through gradient monitoring, data augmentation, and mediator training scheduling techniques, can effectively solve the data imbalance problem and improve the accuracy and stability of the fault diagnosis model.

Description

Technical Field

[0001] This invention belongs to the technical field of combining artificial intelligence and edge computing, specifically relating to a method for diagnosing imbalanced data faults based on self-balancing federated learning. Background Technology

[0002] Utilizing condition monitoring data for fault diagnosis is of great significance in the modern Industrial Internet, as it can improve equipment reliability, enhance operational safety, and reduce maintenance costs. In recent years, with the rapid development of artificial intelligence, data-driven fault diagnosis methods have achieved great success and have been widely applied in many applications. Deep learning-based fault diagnosis technology, with its powerful data representation learning and analysis capabilities, has attracted widespread attention from industry and academia, enabling more automated and effective intelligent process control.

[0003] In the field of artificial intelligence, traditional data processing is often centralized, with data collected from various parties being centralized for preprocessing, cleaning, and modeling. However, with the development and application of big data technologies and artificial intelligence, data security and privacy are receiving increasing attention from relevant departments and enterprises. Restrictions on the flow and sharing of private data have created the problem of "data silos." In practical applications, training data is scattered across institutions or enterprises and is privacy-sensitive, making it impossible to centralize it on a server for training.

[0004] Imbalanced data is another problem in fault diagnosis. Certain types of faults, such as catastrophic failures and unexpected mechanical failures, are rare. Deep learning requires a sufficient number of data samples for representation learning, and numerous studies have shown that the more high-quality data a model trains on, the better its classification performance. Because models are highly dependent on the quality and quantity of data, when the amount of data is small, deep learning algorithms cannot exhibit high performance and are prone to overfitting and crashing. This is especially true when the number of classes in some categories is significantly lower than in others, leading to a large skew in the classification results. This skew increases with the magnitude of the difference in data size, significantly impacting model performance. In actual training, the amount of data for certain data types is insufficient, and this situation may occur both globally and locally. Furthermore, data collection can be difficult, requiring significant time and manpower. Therefore, effectively utilizing imbalanced datasets for industrial equipment fault diagnosis is crucial. Summary of the Invention

[0005] The purpose of this invention is to provide a fault diagnosis method for imbalanced data based on self-balancing federated learning. This method can effectively solve the problem of data imbalance and improve the accuracy and stability of the fault diagnosis model through gradient monitoring, data augmentation and mediation training scheduling techniques.

[0006] To achieve the above objectives, this invention provides a method for diagnosing imbalanced data faults based on self-balancing federated learning, comprising the following steps:

[0007] S1. Obtain client data and perform data preprocessing; the monitor performs gradient monitoring on the client data.

[0008] S2. If a data imbalance is found in the client data, data augmentation is performed.

[0009] S3. Improve the imbalance in model training by sharing generator weights and discriminator weights;

[0010] S4. Employ intermediate training scheduling to reduce the impact of data imbalance;

[0011] S5. The intermediary server sends the model parameters to be updated to the client, performs local training, and returns the updated model parameters to the intermediary server.

[0012] S6. After receiving the updated model parameters, the intermediary server sends the updated model parameters to the waiting training clients. After all clients have completed one round of training, the intermediary server sends the updated model parameters to the federated learning server for model parameter aggregation.

[0013] S7. The federated learning server uses the federated averaging algorithm to aggregate the model parameters obtained in S6, and sends the aggregated model parameters to the intermediary server to start the next round of synchronous training.

[0014] S8. After training is complete, the client obtains the classification accuracy results.

[0015] Preferably, in S1, the data preprocessing includes normalization.

[0016] Preferably, the gradient monitoring infers the composition of each round of training data based on the gradient of the client data and detects whether there is a global imbalance. The specific steps are as follows:

[0017] S101. In round t+1, the monitor downloads the global model G for round t. t And provide it with samples of client data;

[0018] S102. For each class, the monitor obtains the corresponding gradient and updates the global model G. t These updates are then compared with the global model G. t By comparing with +1, the monitor obtains the composition of the training data in round t+1;

[0019] S103. If imbalanced components are continuously detected in the data, data augmentation and mediator training scheduling are used for further processing.

[0020] Preferably, in S102, each type of sample in round t+1 is individually fed into the global model in round t to obtain weight updates, and the weights with larger update magnitudes are selected by calculating the ratio of each weight composition.

[0021] Preferably, in S2, the data augmentation is based on a conditional generative adversarial network model that is aware of data label distribution deviation. The conditional generative adversarial network model consists of a generator and a discriminator, and the generator is a conditional generator with random labels.

[0022] Preferably, in S4, the mediation training scheduling is based on the KL divergence-based mediation training scheduling algorithm, and the KL divergence formula is as follows:

[0023]

[0024] Where p(k) represents the true distribution of the client data, and q(k) represents a uniform distribution.

[0025] Preferably, the specific steps of the KL divergence-based mediation training scheduling algorithm are as follows:

[0026] S401, Initial set of intermediary servers S mediator =φ, its number of elements |S mediator | = 0, Client set S client = {1, 2, ..., n}, where the set S of clients to be assigned to the intermediary server is... clients The number of elements | S clients If |≠0, proceed to the next step;

[0027] S402. Create intermediary server m;

[0028] S403, if |S clients |≠0 and|S mediator If |<γ, then the value of k is calculated using the following formula:

[0029]

[0030] Among them, P m P represents the distribution of data on the intermediary server. u D represents a uniform distribution. KL P represents m With P u KL divergence between them;

[0031] S404, Client k joins intermediary server m;

[0032] S405, Remove client k from client set S client Remove from;

[0033] S406, Output the set of intermediary servers S mediator .

[0034] Preferably, in S5, the local training involves training the model for several local rounds using the mini-batch gradient descent method.

[0035] Preferably, the client initiates and participates in the federated learning process, stores local data and uses it to train a local model, and transmits the model parameters to a cloud server via a wireless network.

[0036] Preferably, the client is an edge server, which is equipped with sensors and control data acquisition devices for industrial production. The client aggregates the acquired data to obtain client data and processes the client data.

[0037] Therefore, the present invention employs the above-mentioned method for diagnosing imbalanced data based on self-balancing federated learning. Compared with the prior art, the present invention has the following significant advantages:

[0038] (1) Solving global and local data imbalance: This invention considers both global and local data imbalance and provides a complete technical solution to effectively improve the accuracy and stability of the fault diagnosis model;

[0039] (2) Data augmentation enhances generalization ability: This invention adopts an LCGAN model based on data label distribution bias perception to generate more minority class data samples, improve the imbalance in model training, and enhance the generalization ability of the model.

[0040] (3) Mediation scheduling optimizes performance and efficiency: This invention uses KL divergence to redistribute the client training order, balance local data imbalance, further improve the accuracy and stability of the model, and effectively control communication overhead;

[0041] (4) Protecting privacy to achieve self-balancing federated learning: Under the premise of protecting data privacy, this invention realizes the self-balancing federated learning of the fault diagnosis model, providing an effective solution for complex unbalanced fault diagnosis.

[0042] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0043] Figure 1 This is an architecture diagram of an imbalanced data fault diagnosis method based on self-balancing federated learning, according to the present invention.

[0044] Figure 2 This is a diagram of the conditional generative adversarial network model architecture for an imbalanced data fault diagnosis method based on self-balancing federated learning, as described in this invention.

[0045] Figure 3 This is a comparison chart of real data and generated data for an imbalanced data fault diagnosis method based on self-balancing federated learning according to the present invention. a-1 is the distribution chart of real samples for outer circle faults; a-2 is the distribution chart of generated samples for outer circle faults; b-1 is the distribution chart of real samples for inner circle faults; b-2 is the distribution chart of generated samples for inner circle faults; c-1 is the distribution chart of real samples for ball bearing faults; c-2 is the distribution chart of generated samples for ball bearing faults.

[0046] Figure 4 This diagram illustrates the improved classification accuracy of the mediation training scheduling algorithm in the imbalanced data fault diagnosis method based on self-balancing federated learning, as presented in this invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used in the present invention should have the ordinary meaning understood by those skilled in the art.

[0048] Example 1

[0049] like Figures 1-4 As shown, this invention provides a method for diagnosing imbalanced data faults based on self-balancing federated learning. This method operates on multiple local user nodes and one server-side aggregation node, and includes the following steps:

[0050] S1. Clients collect fault data from their respective local devices; this data constitutes client data. Client data undergoes preprocessing. Preprocessing includes noise reduction, normalization, and feature extraction. Normalization maps the sample data to the range of 0 to 1, using the formula:

[0051]

[0052] Where x represents the training data, x min x is the minimum value of the training data. max This represents the maximum value of the training data.

[0053] At the start of each training round, the client initializes the model weights and optimizer, and downloads the global model from the previous round. Then, the client feeds each class sample from its client-side data into the global model, calculating gradient updates for each class. A monitor tracks the gradients in the client-side data, analyzing these updates to infer the composition of the current training data and determine if global data imbalance exists. If imbalance is detected, the monitor triggers data augmentation.

[0054] The gradient monitoring method involves the monitor, in round t+1, sending each class of samples from the client data separately into the same global model G in round t. t In this process, the corresponding weight updates are obtained. Each of them The gradient obtained for sample of class p.

[0055] Typically, not all weights in the model receive significant updates; some weights increase only slightly and are easily offset by negative updates from other classes. Therefore, a filter is designed to select weights with larger update magnitudes. Specifically, for class p, from... Obtain the weight update of the p-th output node. Then, the ratios for each weight are calculated. The specific steps are as follows:

[0056] S101. In round t+1, the monitor downloads the global model G for round t. t And provide it with samples of client data;

[0057] S102. For each class, the monitor receives the corresponding gradient update g. L These updates and G t By comparing with +1, the monitor can obtain the composition of the training data in round t+1;

[0058] S103. If similar imbalanced components are detected continuously, it proves that the global model has learned the imbalanced data, and data augmentation and mediator training scheduling will be adopted to mitigate its impact.

[0059] S2. Data augmentation is a conditional generative adversarial network (GAN) model based on the awareness of data label distribution bias. The GAN model consists of a generator and a discriminator. By designing a conditional generator with random labels and training it adversarially against the classifier, the number of minority class data samples can be effectively increased.

[0060] S3. A weight-sharing scheme is adopted between the generator and the discriminator to achieve collaborative adversarial training among the generator, discriminator and classifier, thereby improving the imbalance in model training, reducing the number of training parameters and improving training efficiency.

[0061] The classifier in S3 is a CNN network, whose structure consists of an input layer, a 3x3 convolutional layer with 32 filters, a 2x2 pooling layer, another 3x3 convolutional layer with 64 filters, another 2x2 pooling layer, a 3x3 convolutional layer with 128 filters, followed by another 2x2 pooling layer, a fully connected layer with 2048 to 1024 neurons, and finally an output layer with the sigmoid activation function.

[0062] The conditional generator in S3 employs a Generative Adversarial Network (GAN) model, which consists of a generator and a discriminator. The generator receives conditional labels and generates samples with those labels; the discriminator determines whether the input sample is a real sample from the real dataset or a sample generated by the generator. In this model, both the generator and discriminator are built using a three-layer Long Short-Term Memory (LSTM) network.

[0063] An adversarial training mechanism based on shared feature weights is set up between the generator and the classifier, which can gradually correct the distribution bias of each generated data during the data augmentation process, thereby effectively solving the problem of insufficient minority class data.

[0064] S4. During the model aggregation phase, the federated learning server dynamically adjusts the weights of each client based on the degree of data imbalance, ensuring that important data sources, even with limited data volume, are not overlooked. An intermediary server is created using KL divergence-based intermediary training scheduling to rearrange client training, averaging local imbalances and further improving model accuracy and stability. The specific formula for calculating the KL divergence of each client distribution is as follows:

[0065]

[0066] Where p(k) represents the true distribution of the local dataset of client k, and q(k) represents the probability distribution of the balanced dataset, which is a uniform distribution.

[0067] This invention designs a mediation training scheduling algorithm based on KL divergence to assign clients to mediation servers. The algorithm iterates through all unassigned client data distributions and selects clients whose data distribution most closely approximates a uniform distribution within the mediation server group. Assume the initial set of mediation servers is S. mediator =φ, its number of elements |S mediator | = 0, Client set S client ={1,2,…,n}, the specific steps are as follows:

[0068] S401, When the set of clients S to be assigned to the intermediary server clients The number of elements | Sclients If |≠0, proceed to the next step;

[0069] S402. Create intermediary server m;

[0070] S403, if |S clients |≠0 and|S mediator If |<γ, then the value of k is calculated using the following formula:

[0071]

[0072] Where D KL P represents the data distribution of the intermediary server. m With uniform distribution P u KL divergence between them;

[0073] S404, Client k joins intermediary server m;

[0074] S405, Remove client k from client set S client Remove from;

[0075] S406, Output the set of intermediary servers S mediator .

[0076] When the number of intermediary servers reaches the preset maximum number of clients γ, the federated learning server will create a new intermediary server and repeat the above steps until all clients have been assigned.

[0077] S5. Each intermediary server sends the model parameters to be updated to the client for local training. The model is trained for several local rounds using the mini-batch gradient descent method, and the updated model parameters are returned to the corresponding intermediary server.

[0078] S6. The intermediary server receives the updated model parameters and sends them to the next waiting training client. This continues until all clients have completed one round of training. After all clients have completed one round of training, the intermediary servers send the updated model parameters to the federated learning server for model parameter aggregation.

[0079] S7. The federated learning server uses the federated averaging algorithm to aggregate the model parameters obtained in S6. Then, the federated server sends the aggregated model parameters to the intermediary server to start the next round of synchronous training.

[0080] S8. After training is completed, each client obtains the data generator and fault classifier from S3, obtains a local fault diagnosis classifier that solves class imbalance problems including local imbalance problems, and obtains the classification accuracy result.

[0081] Example 2

[0082] like Figure 1 As shown in this embodiment, the workflow of the present invention is illustrated, and the steps include:

[0083] 1. Gradient monitoring phase:

[0084] After the federated learning server initializes the weights and optimizer of the classification network model, training begins. In round t+1, the monitor downloads the global model G from round t. t It then provides samples of client data to the monitor. For each type of sample, the monitor obtains the corresponding gradient update g. L And will update these updates with G t +1 for comparison. Through this monitoring, the monitor can identify the composition of the training data in round t+1. If similar imbalanced components are detected continuously, it indicates that the global model has learned the imbalanced data, and therefore subsequent data augmentation and mediation scheduling measures will be taken to mitigate the impact of imbalance;

[0085] 2. Data rebalancing phase:

[0086] When implementing federated data augmentation, a Label Conditioned GAN (LCGAN) based on label distribution bias awareness was used, along with a federated averaging algorithm for aggregation. A complementary classifier was incorporated into the GAN model, and a conditional generator with random labels was designed. Through adversarial training and interaction with the classifier, the number of minority class samples was effectively increased. Simultaneously, a method of sharing generator and discriminator weights was adopted to achieve collaborative adversarial training among the generator, discriminator, and classifier, thereby mitigating the global imbalance problem in the training data.

[0087] After obtaining the global LCGAN model, each client uses the model for data augmentation to train a more accurate classification model. However, this process may exacerbate local imbalances, affecting the stability of model weights and classification accuracy. Therefore, a mediator training scheduling algorithm based on KL divergence is adopted. The federated learning server reschedules clients by creating mediator servers to achieve local balance of the model. Each mediator server is only responsible for coordinating γ clients to control communication overhead.

[0088] 3. Model Training and Updates:

[0089] During the training and updating of the federated learning model, at the beginning of each round of communication, the intermediary server sends the model to be updated to its lower-level client. The lower-level client then uses the mini-batch gradient descent algorithm to train the model to be updated locally for E rounds and returns the updated model to the intermediary server. This process does not increase additional communication overhead because the number of local training rounds only affects the client's training time.

[0090] After receiving the updated model, the intermediary server forwards the updated model to the next client waiting to be trained, and so on until all clients have completed one round of training.

[0091] After several such cycles, the intermediary will send the collected updated model to the federated learning server for aggregation. The federated learning server will then use the federated averaging algorithm to aggregate the model data and send the aggregated model back to the intermediary server, thus starting the next round of synchronous training.

[0092] The self-balancing federated learning method aims to diagnose faults in unbalanced data, particularly suitable for motor bearing data with a sampling frequency of 12kHz, which covers three states: normal, external fault, and internal fault. Specific steps include:

[0093] 1. Gradient monitoring phase:

[0094] In the initial stage of federated learning training, the federated learning server is responsible for initializing the model weights and continuously monitoring the model gradient in each subsequent training round. In round t+1, the monitor uses samples from the client data to adjust the global model G. t Perform tests and calculate the gradient update g for each category. L If the monitor detects that a certain type of data exhibits gradient imbalance in multiple consecutive training rounds, the monitor will determine that there is data bias in the global model and initiate a data rebalancing strategy accordingly.

[0095] 2. Data rebalancing phase:

[0096] To address the data imbalance problem, a local LCGAN was used for data augmentation. The conditional generator used a 128-dimensional random noise vector as input and was trained using an Adam optimizer with a learning rate of 0.0002. The classifier network consisted of three convolutional layers, each with a 3*3 kernel size and a stride of 1, followed by a pooling layer with a 2*2 kernel. A random label conditional generation strategy was applied to perform adversarial training on minority class data. A weight-sharing mechanism was used to coordinate the parameters of the generator, discriminator, and classifier to increase the proportion of minority class samples in the training set. The amount of augmented data for each client was twice the amount of the original data. A mediator training scheduling algorithm based on KL divergence was used to ensure the consistency of the generated data on local nodes.

[0097] 3. Model Training and Updates:

[0098] In the model training and update process of federated learning, at the beginning of each round of communication, the intermediary server sends the model to be updated to up to 5 lower-level clients (γ=5). These lower-level clients train the model to be updated locally by performing mini-batch gradient descent algorithm 10 times (E=10), with a batch size of 32 each time. After local training is completed, the lower-level clients return the updated model to the intermediary server.

[0099] The intermediary server sends the collected updated model data to the federated learning server for aggregation. The federated learning server then updates the global model using a federated averaging algorithm. Finally, the federated learning server sends the updated model back to the intermediary server to begin the next round of training.

[0100] After 100 rounds of training, the updated model converged, and the classification accuracy steadily improved. Specific data generation and scheduling algorithm performance are shown below. Figure 3 and Figure 4 middle.

[0101] It is worth noting that all contents not described in detail in this invention are existing technologies and are well known to those skilled in the art.

[0102] Therefore, this invention employs the aforementioned method for diagnosing imbalanced data based on self-balancing federated learning. This method, through gradient monitoring, data augmentation, and mediator training scheduling techniques, effectively solves the data imbalance problem and improves the accuracy and stability of the fault diagnosis model. Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of this invention and not intended to limit it. Although the invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of this invention, and these modifications or equivalent substitutions should not deviate from the spirit and scope of the technical solutions of this invention.

Claims

1. A method for diagnosing imbalanced data faults based on self-balancing federated learning, characterized in that, Includes the following steps: S1. Obtain client data and perform data preprocessing; the monitor performs gradient monitoring on the client data. S2. If a data imbalance is found in the client data, data augmentation is performed. S3. Improve the imbalance in model training by sharing generator weights and discriminator weights; S4. Employ intermediate training scheduling to reduce the impact of data imbalance; S5. The intermediary server sends the model parameters to be updated to the client, performs local training, and returns the updated model parameters to the intermediary server. S6. After receiving the updated model parameters, the intermediary server sends the updated model parameters to the waiting training clients. After all clients have completed one round of training, the intermediary server sends the updated model parameters to the federated learning server for model parameter aggregation. S7. The federated learning server uses the federated averaging algorithm to aggregate the model parameters obtained in S6, and sends the aggregated model parameters to the intermediary server to start the next round of training. S8. After training is complete, the client obtains the classification accuracy results.

2. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that, In S1, the data preprocessing includes normalization.

3. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that, In S1, the gradient monitoring infers the composition of each round of training data based on the gradient of the client data and detects whether there is a global imbalance. The specific steps are as follows: S101. In round t+1, the monitor downloads the global model G for round t. t And provide it with samples of client data; S102. For each class, the monitor obtains the corresponding gradient and updates the global model G. t These updates are then compared with the global model G. t By comparing with +1, the monitor obtains the composition of the training data in round t+1; S103. If imbalanced components are continuously detected in the data, data augmentation and mediator training scheduling are used for further processing.

4. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 3, characterized in that, In S102, each class of samples in round t+1 is individually fed into the global model in round t to obtain weight updates. By calculating the ratio of each weight composition, the weight with the larger update magnitude is selected.

5. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that, In S2, the data augmentation is based on a conditional generative adversarial network model that is aware of data label distribution bias. The conditional generative adversarial network model consists of a generator and a discriminator. The generator is a conditional generator with random labels.

6. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that, In S4, the mediation training scheduling is based on the KL divergence-based mediation training scheduling algorithm, and the KL divergence formula is as follows: Where p(k) represents the true distribution of the client data, and q(k) represents a uniform distribution.

7. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 6, characterized in that, The specific steps of the KL divergence-based mediation training scheduling algorithm are as follows: S401, Initial set of intermediary servers S mediator =φ, its number of elements |S mediator | = 0, Client set S client = {1, 2, ..., n}, where the set S of clients to be assigned to the intermediary server is... clients The number of elements | S clients If |≠0, proceed to the next step; S402. Create intermediary server m; S403, if |S clients |≠0 and|S mediator If |<γ, then the value of k is calculated using the following formula: Among them, P m P represents the distribution of data on the intermediary server. u D represents a uniform distribution. KL P represents m With P u KL divergence between them; S404, Client k joins intermediary server m; S405, Remove client k from client set S client Remove from; S406, Output the set of intermediary servers S mediator .

8. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that, In S5, the local training refers to training the model for several local rounds using the mini-batch gradient descent method.

9. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that, The client initiates and participates in the federated learning process. The client stores local data and uses it to train a local model, and transmits the model parameters to the cloud server via a wireless network.

10. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that, The client is an edge server, which is equipped with sensors and control data acquisition devices for industrial production. The client aggregates the collected data to obtain client data and processes the client data.

Images