Unbalanced data fault diagnosis method based on self-balancing federated learning

Abstract

The invention discloses an unbalanced data fault diagnosis method based on self-balancing federal learning, and belongs to the technical field of combination of artificial intelligence and edge computing, and the method comprises the following steps: S1, obtaining client data, and monitoring whether the client data has an unbalanced phenomenon; s2, data enhancement; s3, sharing the weights of the generator and the discriminator; s4, performing intermediary training scheduling; s5, scheduling the client training model by the intermediary server; s6, aggregating model parameters by the federal learning server; s7, returning the aggregated model parameters to the intermediary server, and starting the next round of synchronous training; and S8, the client obtains an accuracy result. According to the unbalanced data fault diagnosis method based on self-balancing federated learning, the problem of data imbalance can be effectively solved through gradient monitoring, data enhancement and intermediary training scheduling technologies, and the precision and stability of a fault diagnosis model are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of the combination of artificial intelligence and edge computing, and specifically relates to an unbalanced data fault diagnosis method based on self-balancing federated learning. Background Art

[0002] Fault diagnosis using condition monitoring data is of great significance in the modern industrial Internet, which can improve the reliability of equipment, increase operational safety, and reduce maintenance costs. In recent years, with the rapid development of artificial intelligence, data-driven fault diagnosis methods have achieved great success and have been widely used in many applications. Deep learning-based fault diagnosis technology has received widespread attention from the industry and academia due to its powerful data representation learning and analysis capabilities, thereby achieving more automated and effective intelligent process control.

[0003] In the field of artificial intelligence, the traditional data processing model is often centralized, and the data collected by all parties will be concentrated in the same place for preprocessing, cleaning and modeling. However, with the development and application of big data related technologies and artificial intelligence, data security and privacy are increasingly valued by relevant departments and enterprises, and restrictions on the circulation and sharing of private data have created the problem of "data islands". In practical applications, training data is scattered in institutions or enterprises, and is privacy-sensitive, so it is impossible to concentrate on the server for training.

[0004] Class imbalance of data is another problem in fault diagnosis. Some types of faults, such as catastrophic faults and unexpected mechanical failures, are rare. Deep learning requires a sufficient number of data samples for representation learning. At present, a large number of studies have shown that the more high-quality data is used to train the model, the better the classification effect. Since the model is more dependent on the quality and quantity of data, when the amount of data is relatively small, the deep learning algorithm model cannot show high performance, which can easily lead to the model falling into overfitting and collapse. Especially when the number of some categories is significantly lower than that of other categories, the model classification results will be greatly tilted. As the number gap increases, this tilt will increase, greatly affecting the model performance. In the actual training process, the amount of data for some data types is insufficient, and this situation may occur both globally and locally, and it may be difficult to collect, such as consuming a lot of time and manpower. Therefore, it is crucial to effectively use unbalanced data sets for industrial equipment fault diagnosis. Summary of the invention

[0005] The purpose of the present invention is to provide an unbalanced data fault diagnosis method based on self-balancing federated learning, which can effectively solve the data imbalance problem and improve the accuracy and stability of the fault diagnosis model through gradient monitoring, data enhancement and intermediate training scheduling technology.

[0006] To achieve the above object, the present invention provides an unbalanced data fault diagnosis method based on self-balancing federated learning, comprising the following steps:

[0007] S1, obtaining client data and performing data preprocessing, and the monitor performs gradient monitoring on the client data;

[0008] S2. If data imbalance is found in the client data, data enhancement is performed;

[0009] S3, using the method of sharing generator weights and discriminator weights to improve the imbalance of model training;

[0010] S4, using intermediate training scheduling to reduce the impact of data imbalance;

[0011] S5. The intermediary server sends the model parameters to be updated to the client, performs local training, and returns the updated model parameters to the intermediary server;

[0012] S6. After receiving the updated model parameters, the intermediary server sends the updated model parameters to the waiting training clients. After all clients complete a round of training, the intermediary server sends the updated model parameters to the federated learning server for model parameter aggregation.

[0013] S7, the federated learning server aggregates the model parameters obtained in S6 using the federated averaging algorithm, and sends the aggregated model parameters to the intermediary server to start the next round of synchronous training;

[0014] S8. After the training is completed, the client obtains the classification accuracy result.

[0015] Preferably, in S1, the data preprocessing includes normalization.

[0016] Preferably, the gradient monitoring is to infer the composition of each round of training data according to the gradient of the client data and detect whether there is a global imbalance. The specific steps are as follows:

[0017] S101, in round t+1, the monitor downloads the global model G of round t t , and provide it with a sample of the client data;

[0018] S102, for each class, the monitor obtains the corresponding gradient to update the global model G t , and combine these updates with the global model G t +1, the monitor obtains the composition of the t+1 round of training data;

[0019] S103: If unbalanced components are detected continuously in the data, data enhancement and intermediate training scheduling are adopted for further processing.

[0020] Preferably, in S102, each type of sample in the t+1th round is sent separately to the global model of the tth round to obtain weight update, and the weight with a large update amplitude is selected by calculating the ratio of each weight component.

[0021] Preferably, in S2, the data enhancement is a conditional generative adversarial network model based on data label distribution bias perception, the conditional generative adversarial network model consists of a generator and a discriminator, and the generator is a conditional generator with random labels.

[0022] Preferably, in S4, the intermediate training scheduling is an intermediate training scheduling algorithm based on KL divergence, and the KL divergence formula is as follows:

[0023]

[0024] Wherein, p(k) represents the real distribution of the client data, and q(k) represents uniform distribution.

[0025] Preferably, the specific steps of the intermediate training scheduling algorithm based on KL divergence are as follows:

[0026] S401, initial intermediary server set S mediator =0, client set S kclient ={1,2,…,n}, when S clients ≠0, go to the next step;

[0027] S402, creating an intermediary server m;

[0028] S403, if S clients ≠0 and S mediator <γ, then calculate the value of k, the formula is as follows:

[0029]

[0030] Among them, P m represents the distribution of intermediary server data, P u represents uniform distribution, D KL Indicates P m With P u The KL divergence between

[0031] S404, client k joins the intermediary server m;

[0032] S405: Remove client k from client set S client Removed;

[0033] S406: Output the intermediary server set S mediator .

[0034] Preferably, in S5, the local training is to train the model for several local rounds using a mini-batch gradient descent method.

[0035] Preferably, the client initiates and participates in a federated learning process, the client stores local data and uses it to train a local model, and transmits model parameters to a cloud server via a wireless network.

[0036] Preferably, the client is an edge server, and the client is provided with industrial production sensors and control collectors. The client aggregates the collected data to obtain the client data, and processes the client data.

[0037] Therefore, the present invention adopts the above-mentioned unbalanced data fault diagnosis method based on self-balancing federated learning. Compared with the prior art, the present invention has the following significant beneficial effects:

[0038] (1) Solving global and local data imbalance: The present invention considers both global and local data imbalance problems and provides a complete technical solution to effectively improve the accuracy and stability of the fault diagnosis model;

[0039] (2) Data enhancement to improve generalization ability: The present invention adopts the LCGAN model based on data label distribution deviation perception to generate more minority data samples, improve the imbalance of model training, and improve the generalization ability of the model;

[0040] (3) Intermediary scheduling optimizes performance and efficiency: The present invention uses KL divergence to redistribute the client training order, balance local data imbalance, further improve the accuracy and stability of the model, and effectively control communication overhead;

[0041] (4) Protecting privacy and achieving self-balancing federated learning: The present invention realizes self-balancing federated learning of fault diagnosis models under the premise of protecting data privacy, providing an effective solution for complex unbalanced fault diagnosis.

[0042] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is an architecture diagram of an unbalanced data fault diagnosis method based on self-balancing federated learning of the present invention;

[0044] Figure 2A conditional generative adversarial network model architecture diagram of an unbalanced data fault diagnosis method based on self-balancing federated learning of the present invention;

[0045] Figure 3 This is a comparison diagram of real data and generated data of an unbalanced data fault diagnosis method based on self-balancing federated learning of the present invention, wherein a-1 is a real sample distribution diagram of outer ring fault; a-2 is a generated sample distribution diagram of outer ring fault; b-1 is a real sample distribution diagram of inner ring fault; b-2 is a generated sample distribution diagram of inner ring fault; c-1 is a real sample distribution diagram of ball bearing fault; c-2 is a generated sample distribution diagram of ball bearing fault;

[0046] Figure 4 This is a diagram showing the effect of improving the classification accuracy of an intermediate training scheduling algorithm of an imbalanced data fault diagnosis method based on self-balancing federated learning in the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with general skills in the field to which the present invention belongs.

[0048] Embodiment 1

[0049] like Figure 1-Figure 4 As shown, a method for diagnosing unbalanced data faults based on self-balancing federated learning of the present invention runs on multiple local user nodes and a server aggregation node, comprising the following steps:

[0050] S1. The client collects fault data from its local device, which is the client data. The client data is preprocessed. Data preprocessing includes denoising, normalization, and feature extraction. Normalization is to map the sample data to the range of 0 to 1. The formula is:

[0051]

[0052] Where x is the training data, x min is the minimum value of the training data, x max is the maximum value of the training data.

[0053] The client initializes the model weights and optimizer at the beginning of each round of training, and downloads the global model of the previous round. The client then feeds each category sample in the client data into the global model and calculates the gradient update for each category. The monitor monitors the gradient of the client data. By analyzing the gradient update, the monitor infers the composition of the current training data and determines whether there is a global data imbalance. If data imbalance is detected, the monitor triggers the data enhancement step.

[0054] The gradient monitoring method is that in the t+1th round, the monitor will send each type of sample in the client data to the same global model G in the tth round separately. t The corresponding weight update is obtained Each of these The gradient corresponding to the sample of category p.

[0055] Typically, not all weights of the model are updated significantly, some weights increase very little and are therefore easily offset by negative updates from other classes. So a filter is designed to select weights with larger updates. Specifically, for class p, from Get the weight update of the pth output node Then calculate the ratio of each weight component. The specific steps are as follows:

[0056] S101, in round t+1, the monitor downloads the global model G of round t t , and provide it with a sample of the client data;

[0057] S102, for each class, the detector obtains the corresponding gradient update g L , and these updates are combined with G t +1, the monitor can obtain the composition of the t+1 round of training data;

[0058] S103. If similar imbalanced components are detected continuously, it proves that the global model has learned the imbalanced data, and data enhancement and intermediate training scheduling will be adopted to mitigate its impact.

[0059] S2. Data augmentation is a conditional generative adversarial network model based on data label distribution bias perception. The conditional generative adversarial network model consists of a generator and a discriminator. By designing a conditional generator with random labels and conducting adversarial training with the classifier, the number of minority class data samples can be effectively increased.

[0060] S3. A weight sharing scheme is adopted between the generator and the discriminator to realize collaborative adversarial training among the generator, the discriminator and the classifier, thereby improving the imbalance of model training, reducing the number of training parameters and improving training efficiency.

[0061] The classifier in S3 is a CNN network, whose structure is an input layer, a 3*3 convolutional layer with 32 filters, a 2*2 pooling layer, followed by another 3*3 convolutional layer with 64 filters, again a 2*2 pooling layer, then a 3*3 convolutional layer with 128 filters, accompanied by a 2*2 pooling layer, followed by a fully connected layer with 2048 to 1024 neurons, and finally an output layer with Sigmoid activation function.

[0062] The conditional generator in S3 uses a generative adversarial network (GAN) model, which consists of a generator and a discriminator. The generator receives conditional labels to generate samples with specific labels, while the discriminator determines whether the input sample is a real sample from a real dataset or a sample generated by the generator. In this model, both the generator and the discriminator are built based on a three-layer long short-term memory network (LSTM).

[0063] Adversarial training based on shared feature weights is set up between the generator and the classifier, which can gradually correct the distribution deviation of each generated data during the data enhancement process, thereby effectively solving the problem of insufficient data for minority classes.

[0064] S4. In the model aggregation stage, the federated learning server dynamically adjusts the weight of each client according to the imbalance of the client data to ensure that the important data sources with small data volume will not be ignored. The intermediary training scheduling based on KL divergence is used to create an intermediary server to reschedule the client training to average out the local imbalance and further improve the accuracy and stability of the model. The specific formula for calculating the KL divergence of each client distribution is:

[0065]

[0066] Among them, p(k) represents the true distribution of the local data set of client k, and q(k) represents the probability distribution of the balanced data set, that is, uniform distribution.

[0067] The present invention designs an intermediary training scheduling algorithm based on KL divergence, which is used to assign clients to intermediary servers. The algorithm traverses the data distribution of all unassigned clients and selects the client whose data distribution can make the client data distribution in the intermediary server group closest to the uniform distribution. Assume that the initial intermediary server set S mediator =0, client set S kclient ={1,2,…,n}, the specific steps are as follows:

[0068] S401, when S clients ≠0, go to the next step;

[0069] S402, creating an intermediary server m;

[0070] S403, if S clients ≠0 and S mediator <γ, then calculate the value of k, the formula is as follows:

[0071]

[0072] Where D KL Represents the data distribution P of the intermediary server m With uniform distribution P u The KL divergence between

[0073] S404, client k joins the intermediary server m;

[0074] S405: Remove client k from client set S client Removed;

[0075] S406: Output the intermediary server set S mediator .

[0076] When the intermediary server reaches the preset maximum number of clients γ, the federated learning server will create a new intermediary server and repeat the above steps until all clients have been assigned.

[0077] S5. Each intermediary server sends the model parameters to be updated to the client for local training, uses the mini-batch gradient descent method to train the model for several local rounds, and returns the updated model parameters to the corresponding intermediary server.

[0078] S6. The intermediary server receives the updated model parameters and sends them to the next waiting training client. After all clients complete a round of training, all intermediary servers send the updated model parameters to the federated learning server for model parameter aggregation.

[0079] S7. The federated learning server aggregates the model parameters obtained in S6 using the federated averaging algorithm. The federated server then sends the aggregated model parameters to the intermediary server to start the next round of synchronous training.

[0080] S8. After the training is completed, each client obtains the data generator and fault classifier in S3, obtains a local fault diagnosis classifier that solves the category imbalance including local imbalance problems, and obtains the classification accuracy result.

[0081] Embodiment 2

[0082] like Figure 1 As shown, the workflow of the present invention is shown in this embodiment, and the steps include:

[0083] 1. Gradient monitoring stage:

[0084] After the federated learning server initializes the weights and optimizer of the classification network model, it starts training. In the t+1th round, the monitor downloads the global model G of the tth round. t , and provide it with samples of client data. For each type of sample, the monitor obtains the corresponding gradient update g L , and update these with G t +1 for comparison. Through this monitoring, the monitor is able to identify the composition of the training data in the t+1th round. If similar imbalanced components are detected continuously, it means that the global model has learned the imbalanced data, so subsequent data enhancement and intermediary scheduling measures will be taken to mitigate the impact of imbalance;

[0085] 2. Data rebalancing phase:

[0086] When implementing federated data augmentation, we used a label-distribution-bias-aware conditional generative adversarial network (LCGAN), and adopted an aggregation scheme of the federated average algorithm. We incorporated complementary classifiers into the generative adversarial network (GAN) model, and designed a conditional generator with random labels. Through adversarial training and the classifier, we effectively increased the number of minority class samples. At the same time, we adopted a method of sharing generator weights and discriminator weights to achieve collaborative adversarial training between the generator, discriminator, and classifier, thereby alleviating the global imbalance problem of training data.

[0087] After obtaining the global LCGAN model, each client uses the model for data enhancement to train a more accurate classification model. However, this process may aggravate local imbalances and affect the stability of model weights and classification accuracy. Therefore, an intermediary training scheduling algorithm based on KL divergence is adopted. The federated learning server reschedules the client by creating an intermediary server to achieve local balance of the model. Each intermediary server is only responsible for coordinating γ clients to control communication overhead.

[0088] 3. Model training and updating:

[0089] In the training and updating process of the federated learning model, at the beginning of each round of communication, the intermediary server sends the model to be updated to its subordinate client. The subordinate client then uses the mini-batch gradient descent algorithm to perform E rounds of local training on the model to be updated, and returns the updated model to the intermediary server. This process does not increase additional communication overhead because the number of local training rounds only affects the training time of the client.

[0090] After receiving the updated model, the intermediary server forwards the updated model to the next client waiting for training, and the cycle continues until all clients complete a round of training.

[0091] After several such cycles, the intermediary will send the collected updated model to the federated learning server for aggregation. The federated learning server aggregates the model data using the federated averaging algorithm and then sends the aggregated model to the intermediary server, thus starting the next round of synchronous training.

[0092] The self-balancing federated learning method is designed to perform fault diagnosis on unbalanced data, especially for motor bearing data with a sampling frequency of 12kHz, which covers three states: normal state, external fault and internal fault. The specific steps include:

[0093] 1. Gradient monitoring stage:

[0094] At the beginning of federated learning training, the federated learning server is responsible for initializing the model weights and continuously monitoring the model gradients in each subsequent round of training. At the t+1 round, the monitor uses the client data samples to evaluate the global model G. t Test and calculate the gradient update g for each category L If the monitor finds that a certain type of data exhibits gradient imbalance in multiple consecutive rounds of training, the monitor will determine that the global model has data bias and initiate a data rebalancing strategy accordingly;

[0095] 2. Data rebalancing phase:

[0096] To solve the problem of data imbalance, local LCGAN is used for data enhancement. The conditional generator uses a 128-dimensional random noise vector as input and is trained using the Adam optimizer with a learning rate of 0.0002. The classifier network consists of three convolutional layers, each with a convolution kernel size of 3*3 and a step size of 1, followed by a pooling layer with a pooling kernel of 2*2. At the same time, a random label conditional generation strategy is applied to conduct adversarial training on minority class data. The parameters of the generator, discriminator, and classifier are coordinated through a weight sharing mechanism to increase the proportion of minority class samples in the training set. The amount of enhanced data for each client is twice the amount of the original data, and the consistency of the generated data on local nodes is ensured through an intermediate training scheduling algorithm based on KL divergence.

[0097] 3. Model training and updating:

[0098] In the model training and updating process of federated learning, at the beginning of each round of communication, the intermediary server sends the model to be updated to up to 5 subordinate clients (γ=5), and these subordinate clients execute 10 times (E=10) of the mini-batch gradient descent algorithm to train the model to be updated locally, with a batch size of 32 each time. After the local training is completed, the subordinate client returns the updated model to the intermediary server;

[0099] The intermediary server will send the collected updated model to the federated learning server for aggregation. The federated learning server uses the federated average algorithm to update the global model. The federated learning server will send the updated model to the intermediary server again to start the next round of training.

[0100] After 100 rounds of training, the updated model reached convergence and the classification accuracy was steadily improved. The specific data generation effect and scheduling algorithm effect are shown in Figure 3 and Figure 4 middle.

[0101] It is worth noting that the contents not elaborated in detail in the present invention are all prior art and are well known to those skilled in the art.

[0102] Therefore, the present invention adopts the above-mentioned unbalanced data fault diagnosis method based on self-balancing federated learning. This method can effectively solve the data imbalance problem and improve the accuracy and stability of the fault diagnosis model through gradient monitoring, data enhancement and intermediate training scheduling technology.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for fault diagnosis of unbalanced data based on self-balancing federated learning, characterized in that: The following steps are involved: S1, obtaining client data and performing data preprocessing, and the monitor performs gradient monitoring on the client data; S2. If data imbalance is found in the client data, data enhancement is performed; S3, using the method of sharing generator weights and discriminator weights to improve the imbalance of model training; S4, using intermediate training scheduling to reduce the impact of data imbalance; S5. The intermediary server sends the model parameters to be updated to the client, performs local training, and returns the updated model parameters to the intermediary server; S6. After receiving the updated model parameters, the intermediary server sends the updated model parameters to the waiting training clients. After all clients complete a round of training, the intermediary server sends the updated model parameters to the federated learning server for model parameter aggregation. S7, the federated learning server aggregates the model parameters obtained in S6 using the federated average algorithm, and sends the aggregated model parameters to the intermediary server to start the next round of training; S8. After the training is completed, the client obtains the classification accuracy result.

2. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that: In S1, the data preprocessing includes normalization.

3. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that: In S1, the gradient monitoring is to infer the composition of each round of training data based on the gradient of the client data and detect whether there is a global imbalance. The specific steps are as follows: S101, in round t+1, the monitor downloads the global model G of round t t , and provide it with a sample of the client data; S102, for each class, the monitor obtains the corresponding gradient to update the global model G t , and combine these updates with the global model G t +1, the monitor obtains the composition of the t+1 round of training data; S103: If unbalanced components are detected continuously in the data, data enhancement and intermediate training scheduling are adopted for further processing.

4. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 3, characterized in that: In S102, each type of sample in the t+1th round is sent separately to the global model of the tth round to obtain weight update, and the weight with the largest update amplitude is selected by calculating the ratio of each weight component.

5. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that: In S2, the data enhancement is a conditional generative adversarial network model based on data label distribution bias perception, the conditional generative adversarial network model consists of a generator and a discriminator, and the generator is a conditional generator with random labels.

6. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that: In S4, the intermediate training scheduling is an intermediate training scheduling algorithm based on KL divergence, and the KL divergence formula is as follows: Wherein, p(k) represents the real distribution of the client data, and q(k) represents uniform distribution.

7. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 6, characterized in that: The specific steps of the intermediate training scheduling algorithm based on KL divergence are as follows: S401, initial intermediary server set S mediator =0, client set S k client ={1,2,…,n}, when S clients ≠0, go to the next step; S402, creating an intermediary server m; S403, if S clients ≠0 and S mediator <γ, then calculate the value of k, the formula is as follows: Among them, P m represents the distribution of intermediary server data, P u represents uniform distribution, D KL Indicates P m With P u The KL divergence between S404, client k joins the intermediary server m; S405: Remove client k from client set S client Removed; S406: Output the intermediary server set S mediator .

8. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that: In S5, the local training uses the mini-batch gradient descent method to train the model for several local rounds.

9. The method for diagnosing imbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that: The client initiates and participates in the federated learning process, the client stores local data and uses it to train a local model, and transmits model parameters to a cloud server via a wireless network.

10. The method for diagnosing unbalanced data faults based on self-balancing federated learning according to claim 1, characterized in that: The client is an edge server, and the client is provided with industrial production sensors and control collectors. The client aggregates the collected data to obtain the client data, and processes the client data.

Images