Fault diagnosis method of federated learning based on dynamic weighted voting

Abstract

The invention relates to a fault diagnosis method for federated learning based on dynamic weighted voting, which comprises the following steps: in a production control system, similar equipment is clients and is controlled by a server; the client comprises a new access device of the target client and an original device of the source client; deploying a fault diagnosis model at the client; and inputting the to-be-detected sample into the local diagnosis model to obtain a fault classification result. The fault diagnosis network model is constructed by the following steps of: distributing sample data and initializing the model; feature extraction and adversarial learning; feature comparison and pseudo label generation; and incremental migration and optimization. According to the method, the sensitivity of a traditional method to data heterogeneity and noise is overcome. The method not only can reduce the dependence of the target client on a large amount of local data, but also can improve the prediction accuracy and generalization ability of the target client in a small sample learning scene.

Description

TECHNICAL FIELD

[0001] The present application relates to machine learning, distributed learning, and the application of multi-client data fusion in industrial production. Specifically, but not limited to, it relates to a dynamic weighted voting method based on source client feature similarity, used to improve the classification accuracy and fault diagnosis model performance of the target client in the case of label scarcity. BACKGROUND

[0002] In industrial production, the fault diagnosis accuracy and timeliness of key dynamic equipment are particularly important. For example, in the petroleum hydrocracking process, the compressor is a key dynamic equipment, and its operation reliability directly determines the continuous production capacity and safety of the device. Due to the high cost of equipment procurement (up to tens of millions for a single unit), refineries often deploy compressors in a leasing mode, resulting in equipment being scattered in different plant areas and complex operation and maintenance responsibilities. To deal with the risk of unplanned shutdown caused by typical faults, real-time fusion monitoring of multi-source sensor data such as compressor vibration, temperature, and oil pressure is required through industrial Internet of Things (IoT) technology to achieve early fault diagnosis. However, data privacy constraints under the leasing mode, noise suppression of high-frequency dynamic signals, and decoupling analysis of multi-dimensional fault features are still core challenges for current technology deployment. Taking the CMHS dataset (Condition Monitoring of Hydraulic Systems Dataset) as an example, this dataset comprehensively records the operating status of key components of hydraulic systems (pumps, valves, accumulators) under different working conditions through multi-sensor time series data (such as pressure, temperature, flow, and vibration signals), and labels typical fault modes such as pump wear, valve core blockage, and cooler efficiency decline. These data provide rich physical features and label information for developing fault diagnosis models, but traditional centralized learning methods still face serious challenges in the face of multi-client (hydraulic equipment) distributed scenarios.

[0003] Under the federated learning framework, the fault diagnosis of hydraulic systems needs to address the heterogeneity of data distribution between clients and the label scarcity problem of the target end. For example, hydraulic equipment from different factories or leasing parties may have significant deviations in local data features (such as pressure fluctuation patterns and vibration spectra) from the global model due to differences in operating load, environmental temperature, or maintenance strategies. At the same time, newly connected hydraulic equipment (target client) often has difficulty in training a robust model independently due to insufficient fault samples. However, traditional federated learning methods (such as FedAvg) or simple voting fusion strategies usually ignore the quality differences of client data and the inconsistency of feature spaces, resulting in insufficient adaptability of the global model at the target end. SUMMARY

[0004] To solve the above problems, the federated fault diagnosis method based on weighted voting has gradually become the research focus. The method quantifies the feature similarity between the target client (newly connected device) and multiple source clients (original devices), and assigns voting weights, thereby optimizing the global decision.

[0005] Specifically, in the application scenario of the CMHS dataset, the time-frequency feature similarity of the multi-sensor signals of the hydraulic system can be analyzed to give higher weights to high-similarity clients. This mechanism not only alleviates the influence of data distribution deviation, but also improves the accuracy of fault classification (such as distinguishing pump wear and cooler failure of hydraulic equipment) by using heterogeneous data of source clients when the target client label is scarce.

[0006] The present application proposes a fault diagnosis method based on dynamic weighted voting federated learning for equipment fault diagnosis. The method calculates the feature similarity between the target client and multiple source clients, dynamically adjusts the weights of each source client, and overcomes the sensitivity of traditional methods to data heterogeneity and noise. The method not only reduces the dependence of the target client on a large amount of local data, but also improves the prediction accuracy and generalization ability of the target client in a small sample learning scenario.

[0007] The main steps of the present application are as follows:

[0008] S1. Sample data distribution and model initialization

[0009] The fault sample dataset is distributed to the source client and the target client. First, a representative source client is selected, and its labeled sample data is used for local model initialization, and the classification loss is minimized by training;

[0010] After the initialization model is completed, the initialization model is sent to other source clients and target clients, and all source clients further pre-train the local model based on local data. The local model consists of a feature extractor, a reconstructor, and a classifier. The pre-training process uses an autoencoder structure, which reconstructs the original data through the reconstructor and minimizes the reconstruction error.

[0011] S2. Feature extraction and adversarial learning

[0012] The feature extractor is used to extract the fault features of each client and send them to the central server. The server distinguishes whether the features come from the source client or the target client through the discriminator, optimizes the client feature extractor according to the classification result, and gradually extracts the shared features. The prediction loss of the client is based on cross-entropy.

[0013] The client feature extractor is optimized through multiple rounds of adversarial learning, realizing the sharing and unification of different client features, and enhancing the migration ability of the model.

[0014] S3. Feature comparison and pseudo-label generation

[0015] To further improve the diagnostic accuracy, the cosine similarity between the target client and each source client feature is calculated to measure the feature.

[0016] After calculating the cosine similarity, all similarities are normalized, and the normalized similarities are used for weighted voting. For each test sample of the target client, the prediction results of the source client and the similarity weight are used for weighted voting to generate pseudo-labels. Finally, the prediction class of each sample is the class with the maximum weighted sum.

[0017] S4. Incremental migration and optimization

[0018] After the target client receives the pseudo-labels, it is trained with the local data set, aiming to minimize the difference between the pseudo-labels and the model prediction results. The cross-entropy loss function can be used to measure this difference, and the local model parameters are optimized by gradient descent method to update the target client model. The optimized target client model can adapt to the characteristics of the local data, improving the accuracy and reliability of fault diagnosis.

[0019] The technical principles and corresponding technical effects of the present application are as follows:

[0020] The weighted voting method of the present application can improve the classification accuracy of the target client. To solve the challenges of data scarcity of the target client and inconsistent distribution of cross-domain data, the present application establishes a transferable weighted voting framework that can dynamically adjust the weights of the source clients by calculating the feature similarity between the target client and multiple source clients, thereby improving the robustness and accuracy of the prediction. This method can effectively capture the heterogeneity between the source clients and the target client by introducing a feature similarity calculation mechanism, reducing the impact of data inconsistency.

[0021] The weighted voting method of the present application combines source client feature similarity and weighted voting strategy, automatically adjusts the contribution of each client to the prediction results of the target client by calculating the cosine similarity between each source client and the target client. Using a weight calculation method based on similarity normalization, the data fusion efficiency and prediction accuracy in a multi-client scenario are further improved. To enhance the generalization ability of the diagnostic classification model, the Minimax game theory in adversarial learning is adopted to help the model better handle measurement noise and system disturbances, improving the adaptability of the diagnostic classification model in actual industrial environments.

[0022] In addition, the application also designs a framework based on transfer learning, which improves the performance of the target client in small sample learning through the knowledge transfer of the source client, greatly reducing the demand of the target client for a large amount of local data. The framework combines deep transfer learning and weighted voting technology, so that the target client can obtain reliable and efficient prediction results even in the case of data scarcity. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a structural schematic diagram of the fault diagnosis method.

[0024] Figure 2 is a feature alignment schematic diagram.

[0025] Figure 3 is the accuracy of the cooling fault model.

[0026] Figure 4 is the accuracy of the pump fault model.

[0027] Figure 5 is a flowchart of the method of the application. DETAILED DESCRIPTION

[0028] The application will be further described below in combination with the drawings and specific embodiments.

[0029] The construction of the small sample learning dynamic weighted voting method under the federated learning framework of the application includes the following steps:

[0030] S1, the sample data set is sent to the source client and the target client respectively, the local model is initialized through the representative source client, the initialized model is sent to other source and target clients, and the local model is further pre-trained by all source clients, and the local model is composed of a feature extractor, a reconstructor and a classifier.

[0031] S2, the feature extractor of each client extracts fault features from the distributed fault sample data respectively, and the features are sent to the server;

[0032] The server discriminates the received features (source client or target client) through the discriminator, and feeds back the loss signal generated based on the discrimination result to the client; the feature extractor of the client updates the parameters through the gradient inversion strategy, forcing the feature extractors of different clients to output more general features after multiple rounds of adversarial training.

[0033] S3, the features of each source client and the target client are subtracted, and the weighted voting of the fault classification results is performed according to the difference from small to large to obtain pseudo labels.

[0034] S4, the pseudo labels are sent to the target client, and the given sample data set is used for local training.

[0035] In step S1,

[0036] S1-1, select a representative source client to implement local initialization, consider using the first source client, use the labeled data to train the model, so that the classification loss is minimized,

[0037]

[0038] wherein represents the number of fault clients, represents the last layer of the first client classifier, the output value.

[0039] S1-2, in the first source client, the sample data set is given, wherein is the input data of the first sample, is the corresponding fault diagnosis label, represents the number of source clients, represents the number of samples in the first source client, is the dimension of the sample. In the target client, the sample data set is given, wherein is the number of samples of the target client.

[0040] S1-3, use the labeled local data to train the model, adopt the structure of autoencoder, and the reconstructor reconstructs the original data sample from the high-level feature, and the reconstruction error is minimized:

[0041]

[0042] wherein and represent the feature extractor and the reconstructor of the first client, respectively.

[0043] In step S2,

[0044] S2-1, the server receives the features uploaded from all clients, and returns the prediction loss of the corresponding client to the corresponding client; the client optimizes the local model parameters according to the returned prediction loss.

[0045] The client prediction loss based on cross entropy is defined as:

[0046]

[0047]

[0048]

[0049] in and Representing the first Prediction loss for both source and target clients, Representing the The sample at the th Tags for each source client, Representing the The labels of each sample on the target client;

[0050] Indicates the first In the source client, the last layer of the discriminator is the first... The output corresponds to the first output. One input sample; This indicates the output of the target client, where dis represents the discriminator.

[0051] S2-2, The server feeds back the loss Ld to the client, and the feature extractor updates the parameters through the gradient inversion layer (GRL).

[0052]

[0053] in, These are the feature extractor parameters, where η is the learning rate. λ is the sum of the source and client losses, and λ is the adversarial weight coefficient ∈ [0.5, 1.5].

[0054] S2-3. Repeat the above process of feature extraction, feature discrimination, and parameter update to realize the process of adversarial learning.

[0055] In step S3,

[0056] S3-1. Each client (including the target client and the source client) extracts features from the input data through its feature extractor.

[0057] S3-2, Calculate the cosine similarity between the feature vectors of the target client and each source client. For the target client... and the Feature vectors of individual source clients and The cosine similarity is:

[0058]

[0059] in Indicates the first The dot product of the feature vectors of each sample between the target client and the source client represents their similarity. and These are the norms of the feature vectors of the target client and the source client, respectively, representing their lengths.

[0060] S3-3. The calculated similarity scores need to be normalized to ensure that the weights of each source client can be compared within the same range. Normalized similarity matrix. Normalize each column.

[0061]

[0062] The normalized similarity Indicates the source client For the The weights of each sample.

[0063] S3-4. For each test sample from the target client, a weighted vote is performed using the prediction results from the source client and their similarity weights. Assume that for the... One test sample, The prediction result of the individual source client is Each of them It is the source client. For the The predicted category of each sample.

[0064] For each test sample from the target client, the steps for weighted voting using the predictions from the source clients and their similarity weights include:

[0065] S3-4-1, For each sample , No. The weights of the predicted categories for each client are: ,

[0066] S3-4-2. For each category c, calculate the weighted sum:

[0067] in: It is an indicator function, if the first... The source client for the first If the predicted class of a sample is c, the function value is 1; otherwise, it is 0.

[0068] S3-4-3, For each sample The final predicted category is the category with the largest weighted sum:

[0069]

[0070] wherein, is the final predicted class of the th sample.

[0071] In step S4, the construction step of the incremental fine-tuning transfer learning IRTL framework is:

[0072] S4-1, after the target client receives the pseudo label, it combines its local data set and the corresponding pseudo label to perform local training. The training goal of the target client is to minimize the loss between the prediction result and the pseudo label, and the cross-entropy loss or mean square error loss is usually used to measure the difference between the prediction result and the pseudo label. For the th sample, the training loss function of the target client can be expressed as:

[0073]

[0074] wherein is the prediction of the target client model for input .

[0075] S4-2, the target client trains the model using the pseudo label by minimizing the loss function. The optimization process adjusts the model parameters to update the model of the target client. The training step of the target client can be expressed as:

[0076]

[0077] wherein is the parameter of the target client model, and η is the learning rate; is the gradient of the loss function with respect to the model parameter.

[0078] In the fault diagnosis classification model of the present example, the data loading and preprocessing method is as follows:

[0079] The data set contains different types of fault signals, and the labels are divided into cooler state, valve state, pump leakage, accumulator pressure, and stability flag.

[0080] In terms of data processing:

[0081] The data is standardized by the scaler = StandardScaler() function, ensuring that the mean of the features is 0 and the variance is 1, which helps to improve the convergence speed and performance of the model.

[0082] The features of the signal are extracted by wavelet transform, and the number of wavelet decomposition layers is set to 6, and the wavelet base used is "db4" wavelet. This setting can better extract the high and low frequency components in the signal, providing more rich feature information for subsequent fault classification.

[0083] The features of each component extracted include mean, standard deviation, maximum value, etc., which help describe the key characteristics of the signal.

[0084] In terms of label processing, due to the class imbalance of the dataset, the labels of the fault classes are weighted. Specifically:

[0085] The class_weight='balanced' parameter is applied in the support vector machine SVM classifier, which automatically adjusts the loss function according to the number of samples in each class, thus compensating for the sample imbalance and avoiding model bias towards classes with large sample sizes.

[0086] For the classification model, the SVC classifier is used, and the regularization parameter C is set to 1.0, and the kernel uses the "rbf" kernel function, which has good performance for handling nonlinear problems. Through cross-validation (10-fold cross-validation), the performance of the model under different training data is evaluated, and in addition, the AUC value is also used to evaluate the overall classification ability of the model, and the AUC reaches 0.95, indicating that the model has high discrimination ability.

[0087] In the overall diagnostic process: first, the data is standardized and feature extraction is performed; then, the label imbalance problem is handled; finally, the SVM classifier is used to complete the fault diagnosis task, and excellent classification results are obtained.

[0088] By adjusting the hyperparameters, especially selecting the appropriate wavelet decomposition layer number and feature extraction method, the efficiency and accuracy of the model are ensured.

Claims

1. A fault diagnosis method based on federated learning with dynamic weighted voting, characterized in that, In a production control system, similar devices are clients, which are controlled by a server. Clients include newly connected devices for target clients and existing devices for source clients. A global model is deployed on the server side, and a local model is deployed on the client side as a fault diagnosis model. The local model is a local model. The fault diagnosis method is to obtain fault classification results by inputting the sample to be tested into the local diagnostic model on the client side. The sample under test is a detected local fault signal; The steps for constructing a fault diagnosis network model include: S1. Sample Data Distribution and Model Initialization First, the server distributes the fault sample dataset to all clients; a representative source client initializes the local model; then, the initialized local model is sent to other clients; all source clients use the fault dataset distributed to them by the server to further perform local pre-training on their local models. The sample dataset distributed to the source client is labeled, while the sample dataset distributed to the target client is unlabeled. The local model consists of a feature extractor, a reconstructor, and a classifier; Is the representative source client the one with the smallest classification loss during local model training? S2, Feature Extraction and Adversarial Learning Each client's feature extractor extracts fault features from the distributed fault sample data, and these features are sent to the server. The server uses a discriminator to determine whether the received features originate from the source client or the target client, and feeds back the loss signal generated based on the discrimination result to the client; the client's feature extractor updates its parameters through a gradient inversion strategy, forcing the feature extractors of different clients to output more general features after multiple rounds of adversarial training; S3. Feature Comparison and Pseudo-Label Generation The fault features extracted from each source client are subtracted from the fault features extracted from the target client. The fault classification results are then weighted and voted on according to the difference values ​​from smallest to largest to obtain pseudo-labels. S4, Incremental Migration and Optimization The pseudo-labels are sent to the target client, which uses the sample dataset distributed by the server to train its local fault diagnosis model, resulting in the final local fault diagnosis model, which is used to diagnose faults in newly connected devices and classify them.

2. The fault diagnosis method based on dynamically weighted voting federated learning according to claim 1, characterized in that, The feature extractor and reconstructor are autoencoder structures; the classifier is a support vector machine (SVM).

3. The fault diagnosis method based on dynamically weighted voting federated learning according to claim 2, characterized in that, In step S1: S1-1, A representative source client is set as the first... A source client, which uses a given sample dataset to train a local model, with the training objective being to minimize the classification loss. : ， in Indicates the number of fault categories. Indicates the first The corresponding client of the first The k-th output value of the last layer of the classifier for each sample data; hi stands for classifier; S1-2. All source clients use the fault dataset distributed locally by the server to further pre-train their local models: In the In each source client, given a sample dataset ,in It is the first The first source client input One sample data, These are the corresponding fault diagnosis labels. Indicates the number of source clients. Indicates the first Number of samples in each source client It is the dimension of the sample; No. The client uses labeled, given sample data to train a model. The steps are as follows: First, a feature extractor is used to extract high-level features from the samples; then, a reconstructor reconstructs the original data samples from the high-level features. The training objective is to minimize the reconstruction error. : ， in and They represent the first Feature extractor and reconstructor for each source client.

4. The fault diagnosis method based on dynamically weighted voting federated learning according to claim 1, characterized in that, In step S2, S2-1. The server receives features uploaded from all clients and returns the prediction loss of each client to the corresponding client; the client optimizes the local model parameters based on the returned prediction loss. The client prediction loss based on cross-entropy is defined as: ， ， ， in and Representing the first Prediction loss for both source and target clients, Representing the The sample at the th Tags for each source client, Representing the The labels of each sample on the target client; Indicates the first In the source client, the last layer of the discriminator is the first... The output corresponds to the first output. One input sample; This indicates the output of the target client, where dis represents the discriminator; S2-2, The server feeds back the loss Ld to the client, and the feature extractor updates its parameters through the gradient inversion layer GRL. ， in, These are the feature extractor parameters, where η is the learning rate. It is the sum of the source and client losses, and λ is the adversarial weight coefficient ∈ [0.5, 1.5]. S2-3, Repeat the process of feature extraction, feature discrimination, and parameter update to achieve adversarial learning.

5. The fault diagnosis method based on dynamically weighted voting federated learning according to claim 1, characterized in that, In step S3, S3-1, Each client extracts features from the input data through its feature extractor; S3-2. Calculate the cosine similarity between the feature vector of the target client and the feature vector of each source client: For the target client and the Feature vectors of individual source clients and The cosine similarity is: ， in, Indicates the first The dot product of the feature vectors of the target client and the source client for each sample reflects their similarity; and These are the norms of the feature vectors of the target client and the source client, respectively, reflecting their lengths; S3-3. The calculated cosine similarity needs to be normalized to ensure that the weights of each source client can be compared within the same range: Using a normalized similarity matrix Normalize each column. ， The normalized similarity This indicates that the i-th source client is related to the i-th... The weights of each sample; m represents the traversal range covering all source clients (m=1,2,...,N). client ), used to calculate the normalized denominator; S3-4. For each test sample of the target client, a weighted vote is performed using the prediction result of the source client and the similarity weight of the source client. Suppose for the th One sample, The prediction result of the individual source client is Each of them It is the i-th source client For the The prediction results and predicted fault categories for each sample; Step S3-4 includes the following steps: S3-4-1, For the first The nth sample, the nth The weights of the predicted categories for each client are: ； S3-4-2. For any predicted category c, calculate the weighted sum: ， in: It is an indicator function, if the first... The source client for the first If the predicted class of a sample is c, the function value is 1; otherwise, it is 0. S3-4-3, For the first For each sample, the final predicted category is the category with the largest weighted sum: ， in, It is the first The final predicted category of each sample is used as a pseudo-label for the target client to train the local model.

6. The fault diagnosis method based on federated learning with dynamic weighted voting according to claim 5, characterized in that, In step S4, the target client adopts the incremental fine-tuning transfer learning IRTL training strategy, and the steps are as follows: S4-1. After receiving the pseudo-label, the target client combines it with the sample dataset given by the server. and corresponding pseudo tags Perform local training; The first input from the target client One sample data, It is the first The pseudo-labels corresponding to each sample data point The number of samples for the target client; The training objective of the target client is to minimize the loss between the prediction result and the pseudo label, i.e., the training loss. Cross-entropy loss or mean squared error loss is used to measure the difference between the prediction result and the pseudo label. For the For each sample data point, the training loss function for the target client is expressed as: ， in The target client's local model responds to the input data. The prediction; S4-2. The target client trains the local model using pseudo-labels by minimizing the training loss function; the optimization process adjusts the model parameters using gradient descent. To update the local model on the target client; the training representation on the target client is: ， in These are the parameters of the target client model, and η is the learning rate; It is the gradient of the loss function with respect to the model parameters.

Images