Long-tail distribution-oriented generative federated learning fault diagnosis method

Abstract

The application discloses a long-tail distribution-oriented generative federated learning fault diagnosis method, which comprises the following steps: a participant interacts with a center server to obtain model parameters and hyperparameters, calculates a model classifier gradient and optimizes a local model by using a local data set, and uploads the model parameters and the classifier gradient to the center server. The server aggregates the classifier gradient and the model parameters, then iteratively calculates the updated balanced features F and the binary loss of the global classifier, and optimizes the feature extractor aggregation parameters and the global classifier parameters. The participant downloads the global classifier and the feature extractor parameters after the server iteration, updates the local model and takes the updated local model as the starting point of the next iteration. When the communication times between the center server and the participant reach the preset iteration times, the model training is completed. The method can effectively solve the long-tail distribution problem of data and improve the performance of a diagnosis model.

Description

Technical Field

[0001] This invention relates to the field of fault diagnosis of rotating machinery and equipment, and more particularly to a generative federated learning fault diagnosis method for long-tail distribution. Background Technology

[0002] With the development of technologies such as cloud computing and the Internet of Things, and the improvement of theories such as machine learning and deep learning, the field of fault diagnosis and predictive maintenance has gradually evolved towards intelligence, benefiting from deep learning systems and developing a mature and complete field of intelligent fault diagnosis. Research on intelligent fault diagnosis for rotating machinery is beneficial for improving production efficiency, reducing maintenance costs, and enhancing equipment safety. To effectively utilize data scattered across various locations, a natural approach is to integrate data from multiple sources into a shared, large-scale dataset and use it to train a deep network model, thereby increasing the model's performance by expanding the data scale. Federated learning allows multiple parties to collaborate without disclosing their local raw data, jointly training a good network model and sharing the results. This satisfies data privacy requirements while achieving superior model performance.

[0003] In deep learning, class imbalance in training data—where some label categories have a large number of data points while others have a small number—is known as a long-tailed distribution. This long-tailed distribution causes the trained model to favor the category with more data and neglect the category with less data, negatively impacting classification accuracy. In fault diagnosis, because mechanical equipment mostly operates under normal conditions and only malfunctions occasionally, the collected data often consists largely of normal samples. This makes long-tailed distribution even more common in federated learning fault diagnosis, where multiple data sources are combined. The long-tailed distribution significantly limits the performance of federated learning and hinders its practical application.

[0004] The invention disclosed in Chinese invention patent CN115438714A, "Method, Device and Medium for Mechanical Fault Diagnosis Driven by Clustered Federated Learning", does not take into account the huge difference in the number of normal samples and fault samples in the training data of each client, and also has the problem of long-tail distribution, resulting in low accuracy of diagnosis results. Summary of the Invention

[0005] To address the adverse effects of long-tailed distribution of training data on model training, this invention provides a binary-loss generation-federated learning (BG-FL) fault diagnosis method. By introducing the idea of ​​generating balanced features, a balanced feature generation method driven by a binary loss function is designed to train an unbiased classifier, effectively solving the problem of long-tailed data distribution in federated learning scenarios.

[0006] To achieve the objectives of this invention, a generative federated learning fault diagnosis method for long-tailed distributions is provided, comprising the following steps:

[0007] A self-attention network fault diagnosis model is built on the central server. The hyperparameters in the federated learning process are determined. The parameters and hyperparameters of the fault diagnosis model are sent to all participants. After downloading the parameters and hyperparameters, the participants initialize the model, calculate the classifier gradient, optimize the local self-attention network fault diagnosis model, and upload the local model parameters and classifier gradient to the central server.

[0008] The central server aggregates the classifier gradients and model parameters. Then, it performs binary optimization to balance the features and the global classifier, resulting in updated balanced features F and the binary loss of the global classifier. Finally, it optimizes the feature extractor aggregation parameters U. agg and global classifier parameters V;

[0009] Participants download the global classifier and the averaged feature extractor parameters from the server after the iteration to update their local classifier and feature extractor, which serve as the starting point for the next iteration.

[0010] When the central server communicates with the participants a preset number of iterations, model training is complete. At this point, the central server obtains the client model parameters and feature representation vectors under the final stable condition. The trained self-attention network fault diagnosis model is then used for fault diagnosis.

[0011] Furthermore, in the preprocessing stage, the participants interact with the server to obtain the model and training hyperparameters. The model parameters include the feature extractor parameters U. agg The training hyperparameters include global classifier parameters, global iteration count, local iteration count, and local learning rate.

[0012] Imbalance between normal and faulty samples is a very common data acquisition problem, manifesting as a long-tailed data distribution with a large number of normal samples and a small number of faulty samples. Figure 2As shown, when different participants collaborate on training, their independently collected datasets will result in a larger number of normal samples. Therefore, assuming that the participants' data sources are from the same equipment and under the same operating conditions, this study mainly investigates the impact of different degrees of imbalance between the number of normal and faulty samples on the federated learning fault diagnosis model.

[0013] Furthermore, during joint training, assuming there are N participants, each holding a dataset, and each participant's dataset contains two parts: one part is the training set D used for collaborative training. i train The corresponding tag category is C. i train The number of samples in each category is Similarly, another part of the test set D is used to evaluate the model performance. i test The corresponding tag category is C. i test The number of samples in each category is

[0014] Furthermore, the method of the present invention can be understood as adding two steps to the federated learning fault diagnosis model: calculating the classifier gradient and performing binary optimization to balance features and the global classifier.

[0015] Furthermore, the local fault diagnosis model for participating parties includes a feature extraction module and a classifier module. The feature extraction module includes a signal encoding module and a self-attention encoding module (feature extractor), and the classification module includes a fully connected layer and a softmax layer. A schematic diagram of the diagnostic model is shown below. Figure 1 As shown.

[0016] The signal encoding module uses positional encoding and block encoding to convert one-dimensional data into a sequence format suitable for self-attention mechanisms. Based on a given sample length, the data is divided into fixed-length samples, denoted as x. The label y of sample data x is obtained by combining the corresponding health status of the sample data, thus obtaining a sample pair (x, y). The samples are then divided into fixed-length segments, typically 1028 bytes long, with appropriate segment lengths such as 4, 8, or 16 bytes. To obtain more usable information, high-dimensional feature representations of the original samples are added, along with additional learnable position vectors to encode the positional relationships between segments, and the dimensions of the position vectors and segment vectors are kept consistent. After positional encoding and segment encoding, data sample x yields position vectors and segment vectors. The result of the summation operation is called a token, thus obtaining a new representation of the sample {T0, T1...T...}. n}. Where T0 represents the sum of the position vector and segment vector obtained after the first sample x0 is encoded by position and segment, which can be understood as the new form of sample x0 after processing.

[0017] After receiving sample data in token form, the self-attention module calculates attention scores to reconstruct the data and then transforms it. During reconstruction, the input token matrix undergoes three different linear transformations to obtain three matrices: a query matrix (Q), a key matrix (K), and a value matrix (V). Then, the query matrix Q and the key matrix K are multiplied to obtain a relevance score matrix. This matrix is ​​then normalized using softmax to obtain the attention score matrix, which represents the weight scores of other tokens relative to the target token. Finally, the attention score matrix is ​​multiplied by the value matrix to obtain a new form of the original tokens after the self-attention operation extracts their feature information.

[0018] In the classifier module, a fully connected layer is first used to map the data fault categories, such as... Figure 3 As shown in the diagram. Using fully connected layers for function transformation allows the model to learn more information. The fully connected layer consists of two layers. The first layer has parameters that are 2 or 4 times the token dimension, while the second layer is equal to the token dimension to restore the data's dimensionality and maintain token consistency. The number of neurons in the input layer is equal to the token dimension, and the number of neurons in the output layer is equal to the number of label categories C in the dataset. After passing through the fully connected layer, the score indicating whether a sample belongs to a certain category is input into the softmax layer, normalized to the corresponding probability, and the category with the highest probability is used as the predicted fault category for the sample, as shown in the equation.

[0019]

[0020] In the formula, x' represents the sample obtained after passing through the fully connected layer, and y l To determine the possible categories to which sample x' belongs, input p(y) l |x') is used to classify x' into category y. l The probability value, h(x',y) l ) represents the value of the l-th neuron in the fully connected output layer.

[0021] Furthermore, the methods for measuring the imbalance of long-tailed data distributions addressed in this invention can be broadly categorized into exponential and proportional types. In exponential data distributions, the imbalance is controlled by the imbalance factor (IF); a larger IF value indicates a greater difference in the number of samples across different categories. In proportional data distributions, the imbalance is determined by the ratio factor (ra).

[0022] The exponential long-tail distribution uses the number of normal samples as a benchmark, and the number of fault category samples decreases exponentially with respect to the IF value, as shown in the formula.

[0023]

[0024] In the formula, n C n is the number of samples in category c. normal Let C be the number of normal state samples, and C be the total number of categories in the dataset. Generally, the sample label value of the normal category is set to 0.

[0025] Similarly, the proportional long-tailed distribution is shown in the equation.

[0026]

[0027] Furthermore, the classifier gradient calculation takes participant k as an example, such as... Figure 4 As shown in step 1, the gradient calculation method is as shown in the equation.

[0028]

[0029] In the formula, This indicates that p samples are drawn with replacement from class c data in participant k's dataset, x j y j Let represent the subsets of the balanced feature matrix and their respective labels, and let L function represent the loss function, which is the cross-entropy loss function. This formula aims to calculate the gradient of the classifier under the sample subset of class c. After performing the same operation on all participating classes, the set of classifier gradients under different classes is obtained. In this process, the model parameters are only used to calculate the classifier gradient and are not used for parameter optimization.

[0030] Furthermore, the local model optimization refers to using gradient descent to download the feature extractor aggregation parameters U. agg The local network is optimized after combining the global classifier parameters V, as shown in the equation.

[0031]

[0032] In the formula, w i r+1 Let η1 be the model parameters of participant i in the r-th optimization, η1 be the participant's learning rate, D be the local dataset, and w be the feature extractor parameters U in the local model. i With classifier V i Parameter set.

[0033] Furthermore, unlike traditional federated learning (FL), the method requires the server to maintain a global classifier and a randomly initialized balanced feature matrix. The global classifier structure is consistent with the classifier structure of the local model, while the balanced feature matrix F contains three parameters: the number of data categories, the number of samples per category, and the feature dimension. It can be considered that the balanced feature matrix F is a matrix composed of data features with a balanced number of samples.

[0034] Furthermore, in the aggregation stage of classifier gradient and model parameters in the method, the classifier gradient matrix is ​​averaged by category, while the model parameter matrix is ​​directly averaged, as shown in the equation.

[0035]

[0036]

[0037] In the formula, Ui represents the feature extractor parameters of participant i, and Uagg represents the global feature extractor parameters after aggregation. Let the classifier gradient be the classifier gradient of classifier c for participant i. These are the global true gradient parameters.

[0038] Furthermore, in the binary loss optimization step of the method, the cross-entropy loss of the balanced feature F and the global classifier is first calculated, as shown in the equation.

[0039]

[0040] In the formula, M represents the total number of balanced feature sets, V is the global classifier, and CELoss is the calculated cross-entropy loss. During this process, the gradient G of the global classifier is preserved. global .

[0041] Subsequently, the cosine loss of the difference between the classifier gradient and the true gradient matrix is ​​calculated, as shown in the equation.

[0042]

[0043] In the formula, G global G represents the gradient of the global classifier. agg For the true class gradient, cosine similarity can be used as a good measure of the similarity between quantities in a high-dimensional space.

[0044] Combining cross-entropy loss and cosine loss, we obtain the binary loss for updating the balanced feature F and the global classifier, as shown in the equation.

[0045] Loss=αCosineLoss+(1-α)CELoss

[0046] In the formula, α is the balance factor, and the first term, CosineLoss, represents the cosine similarity loss between the global classifier gradient and the average class gradient under balanced features. CELoss represents the cross-entropy loss of the balanced features, reflecting the classifier's ability to correctly determine the feature category. The larger the loss, the more likely the classifier is to fail to correctly determine the true category of the sample.

[0047] Furthermore, based on the optimization direction of the binary loss function, the balanced feature F and the global classifier parameter V will be optimized and updated. The cosine similarity loss will control the gradient of the global classifier to optimize towards the average gradient direction of the real data, and the cross-entropy loss will enable the parameters of the global classifier to have the correct classification ability.

[0048] Furthermore, in the actual optimization process, the method draws on the optimization approach of GANs, employing an alternating optimization method for the balancing feature F and the global classifier. During one iteration of the server, the balancing feature is first updated in round E1, then the global classifier is optimized in round E2. After the server iterates the balancing feature F and the global classifier for round E, the optimized global classifier and balancing feature are obtained. Finally, the participants download the global classifier and the averaged feature extractor parameters from the server iteration to update their local classifier and feature extractor, which serve as the starting point for the next iteration. This training process is repeated until the maximum number of global iterations is reached.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] (1) The fault diagnosis using federated learning in this invention can not only fully protect the privacy needs of each participant for local data, but also solve the "data silo" problem faced by model training. Based on the idea of ​​"the model moves while the data does not move", the model parameters and hyperparameters are improved through repeated parameter interaction and aggregation iterations between the central server and each participant, and finally a highly accurate diagnostic model is built.

[0051] (2) The present invention applies the self-attention model to the feature extractor module of the diagnostic model of each participant, so that the local network of the participants can better capture the data features of multiple parties and effectively improve the performance of the diagnostic model.

[0052] (3) This invention proposes a novel generative federated learning fault diagnosis method for long-tail distribution. Starting from the perspective of generating balanced features, it uses a binary loss composed of the cosine loss of the global classifier optimization gradient and the real class gradient combined with the cross-entropy loss of the balanced features to jointly optimize and generate a set of balanced class features and an unbiased classifier. The introduced generation method effectively solves the problem of data class imbalance in federated learning scenarios. Attached Figure Description

[0053] Figure 1This is a schematic diagram of the local self-attention fault diagnosis model of the participants in an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of the long-tailed data distribution that this invention aims to address.

[0055] Figure 3 This is a schematic diagram of the classifier module provided in an embodiment of the present invention.

[0056] Figure 4 This is a diagram of the overall algorithm architecture of the embodiment of the present invention (BG-FL).

[0057] Figure 5 This is a schematic diagram showing the diagnostic accuracy results of the model test set under different ImF values ​​in scenarios 1 and 3 of this invention.

[0058] Figure 6 This is a flowchart illustrating the steps of a generative federated learning fault diagnosis method for long-tail distribution provided in an embodiment of the present invention. Detailed Implementation

[0059] The embodiments of the present invention are described in detail below. The step numbers in the following embodiments are only set for ease of explanation and do not limit the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0060] This invention addresses the problem of long-tailed distribution of local data among participating parties under data privacy protection requirements. It proposes a generative federated learning method based on binary loss (BG-FL) to solve this problem. The key feature of BG-FL is that it generates a set of class-balanced features by optimizing the binary loss, and then trains an unbiased classifier based on these balanced features. By utilizing the idea of ​​balanced data generation, it effectively solves the problem of long-tailed data distribution. In embodiments of this invention, the effectiveness of the proposed method under various imbalance conditions was verified on the CNC Lab dataset and the MFPT public dataset. Experimental results show that the proposed BG-FL method has a diagnostic accuracy advantage of approximately 10% compared to the optimal federated learning method, effectively solving the problem of low model diagnostic accuracy under long-tailed distribution of participating party data.

[0061] This invention provides a generative federated learning fault diagnosis method for long-tailed distributions, comprising the following steps:

[0062] S1. A self-attention network fault diagnosis model is built on the central server, and the hyperparameters in the federated learning process are determined. After interacting with the central server, the participants obtain the self-attention network fault diagnosis model and its parameters and hyperparameters. Subsequently, the participants use their local datasets to calculate the classifier gradients and optimize the model. After all classifier gradients have been calculated, the participants optimize their local models and upload the local model parameters and classifier gradients to the central server.

[0063] Participants use their local private datasets to calculate classifier gradients for the received network model and perform stochastic gradient descent to optimize the model. After iterating a given number of times R, they send the optimized model parameters and the gradients of each classifier to the server.

[0064] The model's parameters include the feature extractor parameters U. agg The training hyperparameters include global classifier parameters V, global iteration count, local iteration count, and local learning rate.

[0065] In some embodiments of the present invention, the participants use a portion of the actual CNC machine tool dataset and the MFPT public dataset, with each participant maintaining a consistent data size and the test set and training set of each participant maintaining consistent test set and training set of test set, thereby eliminating the impact of differences in data size and test set on the method and verifying the effectiveness of the method more fairly.

[0066] The local self-attention network fault diagnosis model of the participants includes a feature extraction module and a classifier module.

[0067] The feature extraction module includes a signal encoding module and a self-attention encoding module.

[0068] The signal encoding module is used to convert one-dimensional data into a token sequence format suitable for self-attention mechanisms using positional encoding and block encoding. For example... Figure 1 As shown, in the signal encoding module, the data is divided into fixed-length samples, denoted as x, according to the given sample length. The label y of sample data x is obtained by combining the corresponding health status of the sample data, thus obtaining a sample pair (x, y). The samples are then divided into fixed-length segments, typically 1028 bytes long, with the segment length chosen to be appropriate (e.g., 4, 8, 16, etc.). To obtain more usable information and high-dimensional feature representations of the original samples, additional learnable position vectors are added to encode the positional relationships between segments, and the dimensions of the position vectors and segment vectors are kept consistent. After positional encoding and segmental encoding, sample data x yields position vectors and segment vectors. The result of the summation operation is called a token, thus obtaining a new representation of the sample {T0, T1...T...}. n}

[0069] After receiving sample data in the form of a token sequence, the self-attention encoding module calculates attention scores to reconstruct the data and transforms it to obtain a new form after extracting the tokens' feature information through self-attention operations. For example... Figure 1 As shown, in the process of reconstructing the data by calculating the attention score, the input data token matrix is ​​first subjected to three different linear transformations to obtain three matrices: the query matrix (Query, Q), the key matrix (Key, K), and the value matrix (Value, V). Then, the query matrix Q and the key matrix K are multiplied to obtain the relevance score matrix. After softmax normalization, this matrix yields the attention score matrix, which is the weight score matrix of other tokens relative to the target token. Finally, the attention score matrix is ​​multiplied by the value matrix to obtain a new form of the original tokens after the self-attention operation extracts the tokens' feature information.

[0070] The classifier module consists of two fully connected layers and one Softmax layer. See also... Figure 3 Using fully connected layers for function transformation allows the model to learn more information. A fully connected layer consists of two layers. The parameters of the first layer are 2 or 4 times the dimension of the token, and the parameters of the second layer are equal to the dimension of the token to restore the dimension of the data and maintain the consistency of the token dimension.

[0071] The number of neurons in the input layer of the classifier module is equal to the size of the token dimension, and the number of neurons in the output layer is equal to the number of label categories C in the dataset. The data in token sequence form is processed by a fully connected layer to obtain the score of the sample belonging to a certain category, and then input into a softmax layer to normalize it into the corresponding probability. The category with the highest probability is taken as the predicted fault category of the sample, as shown in the formula.

[0072]

[0073] In the formula, x' represents the sample obtained after passing through the fully connected layer, and y l To determine the possible categories to which sample x' belongs, input p(y) l |x') is used to classify x' into category y. l The probability value, h(x',y) l ) represents the value of the l-th neuron in the fully connected output layer.

[0074] The gradient calculation of the classifier, taking participant k as an example, is shown in the following formula.

[0075]

[0076] In the formula, This indicates that p samples are drawn with replacement from class c data in participant k's dataset, x j y j These represent the balanced feature matrix subsets and their respective labels, respectively. The L-function represents the loss function, which is typically the cross-entropy loss function for diagnostic tasks. After performing the same operation on all categories of the participants, we obtain the set of classifier gradients for different categories.

[0077] The optimized local model refers to using gradient descent to download the feature extractor aggregation parameters U. agg The local network is optimized after combining the global classifier parameters V, as shown in the equation.

[0078]

[0079] In the formula, w i r+1 Let η1 be the model parameters of participant i in the r-th optimization, η1 be the participant's learning rate, D be the local dataset, and w be the feature extractor parameters U in the local model. i With classifier V i Parameter set.

[0080] S2. The central server aggregates the classifier gradients and model parameters. Then, the central server performs binary optimization to balance the features and the global classifier, obtaining updated balanced features F and the binary loss of the global classifier, and optimizes the feature extractor aggregation parameters U. agg And the global classifier parameter V.

[0081] In the classifier gradient and model parameter aggregation stage, the classifier gradient matrix is ​​averaged by category, while the model parameter matrix is ​​directly averaged, as shown in the equation.

[0082]

[0083]

[0084] In the formula, Ui represents the feature extractor parameters of participant i, and Uagg represents the global feature extractor parameters after aggregation. Let the classifier gradient be the classifier gradient of classifier c for participant i. These are the global true gradient parameters.

[0085] In the process of performing binary optimization of the balanced feature and the global classifier, the central server first optimizes the balanced feature F using the gradient descent optimization algorithm, and then continues to optimize the global classifier using the gradient descent method. These are shown in the following equations:

[0086]

[0087]

[0088] In the formula, e1 represents the number of iterations for the central server to optimize the balanced features, e2 represents the number of iterations for the central server to optimize the global server, η2 represents the server learning rate, and the Loss function is the binary loss of the balanced features F and the global classifier. e1+1 V represents the balanced eigenvalue after the (e1+1)th iteration optimization. e2+1 These are the parameters of the global classifier after the e2+1th iteration optimization.

[0089] In the binary loss optimization step involving the balancing feature F and the global classifier, the cross-entropy loss of the balancing feature is calculated as follows:

[0090]

[0091] In the formula, M represents the total number of balanced feature sets, V is the global classifier, and CELoss is the calculated cross-entropy loss. During this process, the gradient G of the global classifier is preserved. global .

[0092] Subsequently, the cosine loss of the difference between the classifier gradient and the true gradient matrix is ​​calculated, as shown in the equation.

[0093]

[0094] In the formula, G global G represents the gradient of the global classifier. agg For the true class gradient, cosine similarity can be used as a good measure of the similarity between quantities in a high-dimensional space.

[0095] Combining cross-entropy loss and cosine loss, we obtain the binary loss for updating the balanced feature F and the global classifier, as shown in the equation.

[0096] Loss=αCosineLoss+(1-α)CELoss

[0097] In the formula, α is the balance factor, and the first term, CosineLoss, represents the cosine similarity loss between the global classifier gradient and the average class gradient under the balanced features. CELoss represents the cross-entropy loss of the balanced features.

[0098] In the actual optimization process, the balancing feature F and the global classifier are optimized alternately. During one iteration of the central server, the balancing feature is first updated in round E1, and then the global classifier is optimized in round E2. After the server iterates the balancing feature F and the global classifier in round E, the optimized global classifier and balancing feature are obtained. The values ​​of E, E1, and E2 are not directly related to each other and are specified by the central server as needed.

[0099] S3. The participants download the parameters of the global classifier and the feature extraction module after the server iteration and the mean value, update the participants' local classifier and feature extractor, and use them as the starting point for the next iteration.

[0100] S4. When the number of communications between the central server and the participants reaches the preset number of iterations, the communication between the participants and the central server is stopped, and the model training is completed. At this time, the central server obtains the participant model parameters and feature representation vectors under the final stable condition.

[0101] The methods for measuring the imbalance of long-tailed data distributions addressed in this invention can be broadly categorized into exponential and proportional types. In exponential data distributions, the imbalance is controlled by the imbalance factor (IF); a larger IF value indicates a greater difference in the number of samples across different categories. In proportional data distributions, the imbalance is determined by the ratio factor (r).

[0102] The exponential long-tail distribution uses the number of normal samples as a benchmark, and the number of fault category samples decreases exponentially with respect to the IF value, as shown in equation (2).

[0103]

[0104] In the formula n C n is the number of samples in category c. normal Let C be the number of normal state samples and C be the total number of categories in the dataset. In some embodiments of the present invention, the sample label value of the normal category is set to 0.

[0105] Similarly, the proportional long-tailed distribution is shown in the equation.

[0106]

[0107] The specific algorithm flow of the method provided in the foregoing embodiments of the present invention is shown in the following table:

[0108]

[0109]

[0110] In some embodiments of the present invention, the above methods will be explained in detail with reference to the accompanying drawings and specific examples.

[0111] The experiment collected data on the CNC machine tool under actual service conditions on the production line. The working loads included no-load, light-load (cutting aluminum), and heavy-load (cutting steel), and the operating speeds included 6000, 8000, and 10000 rpm. Other experimental conditions for the collected vibration data included a static load of 7950 N, a dynamic load of 10600 N, a sampling frequency of 25000 Hz, and health states including normal (N), inner fault (IF), outer fault (OF), and cage fault (CF).

[0112] The CNC machine tool spindle bearing vibration dataset was organized according to the different speeds and loads under different working conditions, and can be divided into 9 specific working conditions, as shown in the table.

[0113] Table 1 Data Acquisition Conditions

[0114]

[0115] The MFPT dataset is a publicly available dataset consisting of bearing vibration data in three states: normal, outer race fault, and inner race fault. The bearing type is NICE, with 8 rolling elements and a contact angle of 0°. The relevant data settings are shown in Table 2.

[0116] Table 2 shows the MFPT dataset.

[0117]

[0118] As shown in the table, the MFPT dataset contains data under five different acquisition conditions. Under a constant load of 270 lbs, data on normal and outer race fault states were collected. Under variable load conditions, data on inner and outer race faults were collected under seven different load conditions ranging from 25 to 300 lbs. The spindle speed for all data was kept constant at 25 Hz.

[0119] The hyperparameters of the BG-FL model in this experiment are shown in Table 3:

[0120] Table 3 BG-FL Method Parameter Settings

[0121]

[0122]

[0123] In addition, to more intuitively demonstrate the superiority of this method, this embodiment also includes a control group consisting of experiments performed using the FedAeg method, FedProx method, and Fed-Focal Loss method, respectively, in the same manner as the BG-FL method of this application.

[0124] To fully verify the effectiveness of the BG-FL method, experiments were conducted using actual CNC machine tool datasets and some working conditions from the MFPT public dataset. Two working conditions were selected for each dataset, and the specific experimental scenario settings are shown in the table.

[0125] Table 4 Experimental Scenario Settings

[0126]

[0127] To verify the effectiveness of the BG-FL algorithm under different degrees of long-tail distribution without loss of generality, for scenarios 1 and 3, based on an exponential long-tail distribution, we compare the model performance under different ImF values. For scenarios 2 and 4, based on a proportional long-tail distribution, we compare the model performance under different r values.

[0128] In the experiment, the data scale of each participant was kept consistent, and the test set conditions of each participant were kept consistent with the training set conditions in order to eliminate the impact of differences in data scale and conditions on the method and to verify the effectiveness of the method more fairly. The relevant experimental tasks are shown in Table 5.

[0129] Table 5 Data Long-tailed Distribution Experimental Tasks

[0130]

[0131] Taking ImF equal to 5 and ra equal to 5 as an example, the data volume of different category labels is compared, and the results are shown in Table 6.

[0132] Table 6 shows the data volume of different category labels when ImF equals 15 and ra equals 15.

[0133]

[0134] The table compares the overall sample distribution of the participants when both the imbalance factor and the scaling factor are 15. N, IF, OF, and CF represent the health status of the data samples, with corresponding label values ​​of 0, 1, 2, and 3.

[0135] The experimental results for scenarios 2 and 4 are similar to those for scenarios 1 and 3. The following analysis uses scenarios 1 and 3 as examples. Using scenarios 1 and 3 as experimental conditions, the diagnostic accuracy of the model test set under different ImF values ​​is compared, as follows: Figure 5 As shown, from left to right, they are the FedAeg, FedProx, Fed-Focal Loss, and BG-FL methods. Figure 5It can be seen that, firstly, as the ImF imbalance factor gradually increases, the imbalance of the data deepens, and the accuracy of different methods decreases. Relatively speaking, the performance decline of the BG-FL method is slower, indicating better stability, and it still maintains high accuracy even under extreme imbalance. Specifically, in scenario 1, when ImF equals 5, the BG-FL method shows no significant difference from other methods, exhibiting excellent test accuracy in this case where the class imbalance is not significant. However, as the class imbalance deepens, when ImF equals 15, the performance of the FedAvg and FedProx methods drops significantly, while the FedFocal method maintains an accuracy of around 92%, but still 5% lower than the BG-FL method. When the imbalance continues to deepen, reaching 25 and 50, the performance of FedFocal decreases to the same level as FedAvg and FedProx. This is because the sample size of some classes is extremely small, making it impossible to accurately identify the data of those classes. The BG-FL method can fully utilize information such as data gradients for this category to generate features, thus enabling it to better identify these samples. When ImF equals 25, its test accuracy is about 15% higher than other methods; when ImF equals 50, it also has a performance advantage of about 10%.

[0136] Secondly, a horizontal comparison of scenarios reveals that scenario 1 generally exhibits higher diagnostic performance than scenario 3. This is because scenario 1 involves 6 participants, while scenario 3 only involves 3. Although the imbalance factor is the same, the total number of samples in scenario 3 is lower than in scenario 1, leading to generally lower diagnostic performance for the latter. However, the proposed BG-FL method still outperforms similar methods in its diagnostic performance. The horizontal comparison results further demonstrate that the proposed method performs excellently across different scales of participant and sample sizes.

[0137] In summary, this embodiment has the following advantages and beneficial effects compared to the prior art:

[0138] (1) This invention adopts federated learning fault diagnosis to fully protect the privacy requirements of each participant for local data and build a highly accurate diagnostic model.

[0139] (2) The present invention applies the self-attention model to the feature extractor module of the diagnostic model of each participant, so that the local network of the participants can better capture the data features of multiple parties and effectively improve the performance of the diagnostic model.

[0140] (3) This invention proposes a novel generative federated learning fault diagnosis method for long-tail distribution. Starting from the perspective of generating balanced features, it uses a binary loss composed of the cosine loss of the global classifier optimization gradient and the real class gradient combined with the cross-entropy loss of the balanced features to jointly optimize and generate a set of balanced class features and an unbiased classifier. The introduced generation method effectively solves the problem of data class imbalance in federated learning scenarios.

[0141] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A generative federated learning fault diagnosis method for long-tail distribution, characterized in that, Includes the following steps: A self-attention network fault diagnosis model is built on the central server. The hyperparameters in the federated learning process are determined. The parameters and hyperparameters of the fault diagnosis model are sent to all participants. After downloading the parameters and hyperparameters, the participants initialize the model, calculate the classifier gradient, optimize the local self-attention network fault diagnosis model, and upload the local model parameters and classifier gradient to the central server. The central server aggregates the classifier gradients and model parameters. Then, it performs binary optimization to balance the features and the global classifier, resulting in updated balanced features F and the binary loss of the global classifier. Finally, it optimizes the feature extractor aggregation parameters U. agg and global classifier parameters V; Participants download the global classifier and the averaged feature extractor parameters from the server after the iteration to update their local classifier and feature extractor, which serve as the starting point for the next iteration. When the number of communications between the central server and the participants reaches the preset number of iterations, the model training is completed, and the trained self-attention network fault diagnosis model is used for fault diagnosis. The optimized local self-attention network fault diagnosis model refers to using gradient descent to download the feature extractor aggregation parameters U. agg The local model is optimized after incorporating the global classifier parameters V, as shown in the equation: : In the formula, w i r As a participant i The model parameters obtained in the r-th optimization, Let U be the participant learning rate, D be the local dataset, and w be the feature extractor parameter U in the local model. i With classifier V i Parameter set; In the classifier gradient and model parameter aggregation stage, the classifier gradient matrix is ​​averaged by category, while the model parameter matrix is ​​directly averaged, as shown in the equation: In the formula U i For the feature extractor parameters of participant i, U agg These are the parameters for the aggregated global feature extractor. Let the classifier gradient be the classifier gradient of classifier c for participant i. These are the global true gradient parameters. N Number of participants; The binary loss optimization process for balancing features F and the global classifier begins by calculating the cross-entropy loss of the balancing features, as shown in the equation: In the formula, M is the total number of balanced feature sets, V is the global classifier parameter, and x j y j Let G represent the subsets of the balanced feature matrix and their respective labels, and let CELows be the calculated cross-entropy loss. During this process, the gradient G of the global classifier is preserved. global ; Subsequently, the cosine loss, which is the difference between the classifier's gradient and the true gradient matrix, is calculated, as shown in the equation: In the formula, For the global classifier gradient, The gradient of the true class; Combining cross-entropy loss and cosine loss, we obtain the binary loss for updating the balanced feature F and the global classifier, as shown in the equation: In the formula, α is the balance factor, the first term CosineLoss represents the cosine similarity loss between the global classifier gradient and the true class gradient under the balance feature, and CELoss represents the cross-entropy loss of the balance feature.

2. The generative federated learning fault diagnosis method for long-tail distribution according to claim 1, characterized in that, The local self-attention network fault diagnosis model includes a feature extraction module and a classifier module. The feature extraction module includes a signal encoding module and a self-attention encoding module. The signal encoding module is used to convert one-dimensional data into a token sequence format suitable for processing by the self-attention mechanism using positional encoding and block encoding. The self-attention encoding module is used to receive token-formatted sample data, calculate attention scores to reconstruct the data, and transform the reconstructed data to obtain a new form after extracting token feature information through self-attention operation. The number of neurons in the input layer of the classification module is equal to the token dimension, and the number of neurons in the output layer is equal to the number of label categories C in the dataset. It includes a fully connected layer and a softmax layer. After the token-formatted data is processed by the fully connected layer to obtain the score of the sample belonging to a certain category, it is input into the softmax layer, normalized to the corresponding probability size, and the category with the highest probability is used as the predicted fault category of the sample.

3. The generative federated learning fault diagnosis method for long-tail distribution according to claim 2, characterized in that, The steps for obtaining the new form of tokens feature information after the self-attention encoding module includes: the input data token matrix undergoes three different linear transformations to obtain three matrices, namely the query matrix, the key matrix, and the value matrix; then, the query matrix and the key matrix are multiplied to obtain the relevance score matrix; the relevance score matrix is ​​normalized by softmax to obtain the attention score matrix, which is the weight score matrix of other tokens relative to the target token; the attention score matrix and the value matrix are multiplied to obtain the new form of the original tokens after the self-attention operation extracts the tokens feature information.

4. The generative federated learning fault diagnosis method for long-tail distribution according to claim 1, characterized in that, Assuming that the participants' data sources are the same equipment and under the same operating conditions, this study investigates the impact of different degrees of imbalance between the number of normal and faulty samples on the federated learning fault diagnosis model.

5. The generative federated learning fault diagnosis method for long-tail distribution according to claim 1, characterized in that, During joint training, assume there are N participants, each holding a dataset. Each participant's dataset consists of two parts: a training set used for collaborative training and a training set. The corresponding tag category is The number of samples in each category is ; Another part is the test set used to evaluate model performance. The corresponding tag category is The number of samples in each category is .

6. The generative federated learning fault diagnosis method for long-tail distribution according to claim 1, characterized in that, The methods for measuring the imbalance of the long-tail distribution of data include exponential and proportional types. In exponential data, the imbalance is controlled by the imbalance factor IF. The larger the IF value, the greater the difference in the number of samples in different categories. In proportional data, the imbalance is determined by the proportional factor ra. The long-tail distribution of exponential data uses the number of normal samples as a baseline, and the number of fault category samples decreases exponentially with respect to the IF value, as expressed by the formula: The long-tail distribution of proportional data is shown in the following formula: In the formula, n C n is the number of samples in category c. normal C represents the number of samples in the normal state, and C represents the total number of categories in the dataset.

7. The generative federated learning fault diagnosis method for long-tail distribution according to claim 1, characterized in that, The classifier gradient calculation is based on the participants k For example, its gradient calculation method is as follows: In the formula, Indicates from the participating parties k Categories in the dataset c Data is extracted with replacement p Samples, x j y j These represent the balanced feature matrix subsets and their respective labels, respectively. The L-function represents the loss function. After performing the same operation on all categories of the participants, we obtain the set of classifier gradients for different categories. C represents the total number of categories in the dataset.

8. A generative federated learning fault diagnosis method for long-tail distribution according to any one of claims 1-7, characterized in that, In the actual optimization process, the method employs an alternating optimization approach between the balancing feature F and the global classifier. During one iteration of the central server, the balancing feature is updated first. E One round, then global classification classifier optimization. E After two rounds of iterative balancing of the server-side features F and the global classifier E, the optimized global classifier and balanced features are obtained.

Images