Learning method, system, medium and equipment for heterogeneous rotary machine operation and maintenance data

Abstract

A semi-supervised federated learning method, system, medium and equipment for heterogeneous rotating machinery operation data, in the method, rotating machinery operation data of a plurality of clients is collected and normalized, part of the data is labeled, a training set and a test set of the plurality of clients are divided, and a plurality of clients are randomly selected to participate in training in each round of communication of federated learning; a server constructs a global model and a parameter optimizer, the global model includes a feature extractor and a label encoder, and the global model parameters are initialized; the server collects local labels of the plurality of clients to form global labels and distributes the global labels to each client, and the clients and the server adopt the same model architecture; a plurality of rounds of communication are performed between the clients and the server, the clients complete training based on local data and upload model parameters to the server, the server performs weighted aggregation on the model parameters as global model parameters, and distributes the global model parameters to each client, and the global model parameters are distributed to each client until the server reaches a convergence condition, and the effectiveness is verified by a classification task.

Description

Technical Field

[0001] This invention relates to the field of rotating machinery operation and maintenance technology, and in particular to a semi-supervised federated learning method, system, medium and device for heterogeneous rotating machinery operation and maintenance data. Background Technology

[0002] In the field of intelligent operation and maintenance (O&M) of high-end equipment, a large amount of labeled data is required for training. However, fault data is relatively scarce, and the types of fault data obtained from a single device are limited, resulting in low generalization performance of the trained O&M model. Traditional high-end equipment O&M typically relies on centralized data analysis and processing methods, meaning that all data needs to be collected in a central server. Due to data privacy protection among various parties, centralized intelligent O&M becomes increasingly difficult. In actual engineering, the data distribution among different clients is different, which is not suitable for traditional distributed learning with independent and identically distributed data. Therefore, how to efficiently aggregate the trained models from various clients is a problem that needs to be solved. To protect data privacy and increase training data, federated learning can enable multi-party collaborative training of a global model without infringing on user privacy. Its core idea is to use the same model architecture for local and global models. Local models are trained on local data and their parameters are uploaded to the global model for weighted aggregation. This can effectively avoid users' concerns about data privacy, enhance the robustness of the O&M model, and improve the generalization ability of the global model.

[0003] The information disclosed in the background section is only for enhancing the understanding of the background of this invention, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] This invention provides a semi-supervised federated learning method, system, medium, and device for heterogeneous operation and maintenance data of rotating machinery, which solves the problems of insufficient operation and maintenance data and privacy protection. This invention enables multi-party collaboration without infringing on data privacy and realizes collaborative model training under the condition that the data of multiple parties are not independent and are identically distributed.

[0005] A semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data includes:

[0006] Step S1, collect The operation and maintenance data of rotating machinery from each client was collected and normalized, with some data already labeled and divided. The training and test sets for each client are randomly selected in each round of federated learning communication. One client participated in the training;

[0007] Step S2: The server builds a global model and a parameter optimizer. The global model includes a feature extractor. and tag encoder Initialize global model parameters ;

[0008] Step S3, server collects Local tags for each client Form global tags It is then distributed to each client, and the client and server use the same model architecture;

[0009] Step S4 involves multiple rounds of communication between the client and the server. The client completes training based on local data and uploads the model parameters to the server. The server weights and aggregates the model parameters as global model parameters and distributes the global model parameters to each client until the server reaches the convergence condition, thus obtaining the optimized global model.

[0010] In the aforementioned semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the rotating machinery operation and maintenance data is normalized, and the data normalization calculation method is as follows:

[0011]

[0012] in It is normalized data. It is the raw data. It is the mean of the original data. It is the standard deviation of the original data.

[0013] In the aforementioned semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the tag encoder... The process maps global labels to label encodings. The label encoder adds the word embeddings, sentence embeddings, and position embeddings of the global labels to obtain its input representation. It uses a 12-layer Transformer encoder, with 768 hidden units and 12 self-attention heads in each layer. Then, it uses Bi-LSTM layers and fully connected layers to perform multi-level abstraction and representation learning on the input global labels to obtain the label encoding. The number of bidirectional LSTM units is 512 each, and the output dimension of the connected fully connected layers is 256 dimensions.

[0014] In the aforementioned semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the feature extractor First, the monitoring data is mapped into a 256-dimensional latent space through a fully connected layer. Then, the summation with the location embedding is fed into a multi-head attention layer. The output and input are summed and layer normalized. This result is then fed into a feedforward connection layer, where the output and input are summed and layer normalized again. Finally, 256-dimensional monitoring data features are extracted. This indicates the input of rotating machinery operation and maintenance data in the multi-head attention layer, where express The d-dimensional monitoring value at time t, and the subsequent data feature extraction process are represented as follows:

[0015]

[0016]

[0017] in and These represent the outputs of the layer normalization, This indicates the self-attention mechanism. This represents the computation of a feedforward neural network. Representation layer normalization.

[0018] After encoding the data features and labels, the dot product operation is performed, and the probability of the data belonging to each label is obtained through the softmax activation function.

[0019] In the aforementioned semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the client parameter optimization process is as follows: first, freeze the parameters of the label encoder, optimize and update the parameters of the feature extractor, then freeze the parameters of the feature extractor, optimize and update the parameters of the label encoder, thereby realizing the alternating update of the feature extractor and the label encoder.

[0020] In the aforementioned semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the process is carried out within the federated learning framework. Round communication, each round of communication selects Each client performs The training cycle consists of 10 periods, with the set of clients participating in the training being 100. The client m records N labeled data points based on the global label W. ,in It is a client Data, It is data The labels are One-Hot encoded; during the client-side training process, local data features and global label encodings are aligned, and the alignment process is as follows:

[0021] Generate global tag encoding, client From the The updated label encoder is trained to obtain the global label encoding. , Indicates the tag name.

[0022] Calculate the similarity distance between data features and label encoding. Client From the The updated feature extractor is used to obtain data features in each training round. Calculate the similarity distance between data features and global label encoding. :

[0023]

[0024] in This represents the dot product operation. Describing the L2 norm,

[0025] Assign pseudo-labels to unlabeled data Unlabeled data is assigned pseudo-labels based on similarity distance, according to the following rules: local label Determining its affiliation range based on the similarity distance of labeled data. For non-local tags, the top 2% of similar distances across all local data are considered as their category. If the minimum similarity distance of unlabeled data falls within one of the two aforementioned categories, then a pseudo-label is assigned. Otherwise, it will not participate in this round of local model training.

[0026] In the aforementioned semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the training loss during local data training includes labeled data. and pseudo-labeled data training loss The training loss uses cross-entropy loss. and The specific calculation process is as follows:

[0027]

[0028]

[0029]

[0030]

[0031] in Representing data Predicted probability for global labels This represents the softmax function. Indicates the data from the dataset Randomly selected samples Perform the expectation operation, which represents the expectation operation on the entire training set. The loss of all samples is averaged.

[0032] After the e-th cycle of training in the h-th round, the client Feature extractor in the local model and tag encoder parameters The alternating update process is as follows:

[0033]

[0034]

[0035] in Indicates the learning rate. Represents the gradient.

[0036] Client The training objective is

[0037]

[0038] The training objective of the server is

[0039] .

[0040] A system for implementing the method includes:

[0041] The acquisition module is used for data acquisition. The operation and maintenance data of rotating machinery from each client was collected and normalized, with some data already labeled and divided. The training and test sets for each client are randomly selected in each round of federated learning communication. One client participated in the training;

[0042] The server builds the global model and parameter optimizer; the global model includes a feature extractor. and tag encoder Initialize global model parameters Server collection Local tags for each client Form global tags It is then distributed to each client, and the client and server use the same model architecture;

[0043] Heterogeneous units are used for multi-round communication between clients and servers. Clients complete training based on local data and upload model parameters to the server. The server weights and aggregates the model parameters as global model parameters and distributes the global model parameters to each client until the server reaches the convergence condition and obtains the optimized global model.

[0044] A computer storage medium including computer instructions that, when run on a computer, cause the computer to perform the method.

[0045] An electronic device, the electronic device comprising:

[0046] Memory, processor, and computer programs stored in memory and executable on the processor, wherein,

[0047] The processor implements the method when executing the program.

[0048] Compared with existing technologies, this invention has the following advantages: This invention can achieve collaborative training of the global model when the data of each client is not independent and identically distributed, thereby enhancing the robustness of the global model; since most of the data labels are Chinese descriptions, a label encoder is constructed to generate label codes; the model includes a feature extractor and a label encoder, and global labels are formed by sharing the labels of each user terminal to guide the alignment of data features, avoiding the performance degradation of the global model caused by local model drift due to the heterogeneity of data from each user terminal, and ultimately solving the problems of imbalance and scarcity of operation and maintenance data. Attached Figure Description

[0049] Various other advantages and benefits of the present invention will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0050] In the attached diagram:

[0051] Figure 1 This is a schematic diagram of the computation process of semi-supervised federated learning according to an embodiment of the present invention;

[0052] Figure 2 This is a local model structure diagram of an embodiment of the present invention. The global model only has a feature extractor and a label encoder, and its parameters are obtained by aggregating the local model parameters.

[0053] Figure 3 This is a schematic diagram of the classification results of client data that are independently and identically distributed according to an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the classification results of client data that are not independently and identically distributed according to an embodiment of the present invention;

[0055] Figure 5 This is an F1 score curve of the global model training process in an embodiment of the present invention.

[0056] The present invention will be further explained below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0057] Specific embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While specific embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0058] It should be noted that certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that different terms may be used to refer to the same component. This specification and claims do not distinguish components based on differences in terminology, but rather on differences in function. The terms "comprising" or "including" used throughout the specification and claims are open-ended and should be interpreted as "comprising but not limited to." The following descriptions are preferred embodiments for carrying out the invention; however, these descriptions are for the purpose of understanding the general principles of the specification and are not intended to limit the scope of the invention. The scope of protection of this invention is determined by the appended claims.

[0059] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. The accompanying drawings do not constitute a limitation on the embodiments of the present invention.

[0060] like Figures 1 to 5 As shown, the semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data includes the following steps:

[0061] Step S1, collect The operation and maintenance data of rotating machinery from each client was collected and normalized, with some data already labeled and divided. The training and test sets for each client are randomly selected in each round of federated learning communication. One client participated in the training;

[0062] Step S2: The server builds a global model and a parameter optimizer. The global model includes a feature extractor. and tag encoder Initialize global model parameters ;

[0063] Step S3, server collects Local tags for each client Form global tags It is then distributed to each client, and the client and server use the same model architecture;

[0064] Step S4 involves multiple rounds of communication between the client and server. The client completes training based on local data and uploads the model parameters to the server. The server weights and aggregates the model parameters as global model parameters and distributes them to each client until the server reaches the convergence condition, thus obtaining the optimized global model.

[0065] In a preferred embodiment of the semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the rotating machinery operation and maintenance data is normalized, and the data normalization calculation method is as follows:

[0066]

[0067] in It is normalized data. It is the raw data. It is the mean of the original data. It is the standard deviation of the original data.

[0068] In a preferred embodiment of the semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, a tag encoder is used. It is to use global tags Mapped to tag encoding, the tag encoder will convert global tags The input representation is obtained by adding the word embeddings, sentence embeddings, and position embeddings. This representation is then processed through a 12-layer Transformer encoder, with 768 hidden units and 12 self-attention heads per layer. Finally, a Bi-LSTM layer and a fully connected layer are used to process the global labels of the input. Label encoding is obtained through multi-level abstraction and representation learning. The number of bidirectional LSTM units is 512, and the output dimension of the fully connected layers is 256.

[0069] In a preferred embodiment of the semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the feature extractor... First, the monitoring data is mapped into a 256-dimensional latent space through a fully connected layer. Then, the summation with the location embedding is fed into a multi-head attention layer. The output and input are summed and layer normalized. This result is then fed into a feedforward connection layer, where the output and input are summed and layer normalized again. Finally, 256-dimensional monitoring data features are extracted. This indicates the input of rotating machinery operation and maintenance data in the multi-head attention layer, where express The d-dimensional monitoring value at time t, and the subsequent data feature extraction process are represented as follows:

[0070]

[0071]

[0072] in and These represent the outputs of the layer normalization, This indicates the self-attention mechanism. This represents the computation of a feedforward neural network. Representation layer normalization.

[0073] After encoding the data features and labels, the dot product operation is performed, and the probability of the data belonging to each label is obtained through the softmax activation function.

[0074] In a preferred embodiment of the semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the client parameter optimization process is as follows: first, freeze the parameters of the label encoder, optimize and update the parameters of the feature extractor, then freeze the parameters of the feature extractor, optimize and update the parameters of the label encoder, thereby realizing the alternating update of the feature extractor and the label encoder.

[0075] In a preferred embodiment of the semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the process is performed within a federated learning framework. Round communication, each round of communication selects Each client performs The training cycle consists of 10 periods, with the set of clients participating in the training being 100. The client m records N labeled data points based on the global label W. ,in It is a client Data, It is data The labels are One-Hot encoded; during the client-side training process, local data features and global label encodings are aligned, and the alignment process is as follows:

[0076] Generate global tag encoding, client From the The updated label encoder is trained to obtain the global label encoding. , Indicates the tag name.

[0077] Calculate the similarity distance between data features and label encoding. Client From the The updated feature extractor is used to obtain data features in each training round. Calculate the similarity distance between data features and global label encoding. :

[0078]

[0079] in This represents the dot product operation. Represents the L2 norm.

[0080] Assign pseudo-labels to unlabeled data Unlabeled data is assigned pseudo-labels based on similarity distance, according to the following rules: local label Determining its affiliation range based on the similarity distance of labeled data. For non-local tags, the top 2% of similar distances across all local data are considered as their category. If the minimum similarity distance of unlabeled data falls within one of the two aforementioned categories, then a pseudo-label is assigned. Otherwise, it will not participate in this round of local model training.

[0081] In a preferred embodiment of the semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data, the training loss during local data training includes labeled data. and pseudo-labeled data training loss The training loss uses cross-entropy loss. and The specific calculation process is as follows:

[0082]

[0083]

[0084]

[0085]

[0086] in Representing data Predicted probability for global labels This represents the softmax function. Indicates the data from the dataset Randomly selected samples Perform the expectation operation, which represents the expectation operation on the entire training set. The loss of all samples is averaged.

[0087] After the e-th cycle of training in the h-th round, the client Feature extractor in the local model and tag encoder parameters The alternating update process is as follows:

[0088]

[0089]

[0090] in Indicates the learning rate. Represents the gradient.

[0091] Client The training objective is

[0092]

[0093] The training objective of the server is

[0094] .

[0095] A system for implementing the method includes:

[0096] The acquisition module is used for data acquisition. The operation and maintenance data of rotating machinery from each client was collected and normalized, with some data already labeled and divided. The training and test sets for each client are randomly selected in each round of federated learning communication. One client participated in the training;

[0097] The server builds the global model and parameter optimizer; the global model includes a feature extractor. and tag encoder Initialize global model parameters Server collection Local tags for each client Form global tags It is then distributed to each client, and the client and server use the same model architecture;

[0098] Heterogeneous units are used for multi-round communication between clients and servers. Clients complete training based on local data and upload model parameters to the server. The server weights and aggregates the model parameters as global model parameters and distributes the global model parameters to each client until the server reaches the convergence condition and obtains the optimized global model.

[0099] In one embodiment, the client Based on global tags Will Each labeled data point is denoted as . ,in It is a client Data, It is data The tag is One-Hot encoded.

[0100] In one embodiment, during the aggregation of label encoder parameters in the global model, it is necessary to summarize the label distribution of the clients participating in the training, perform weighted aggregation based on the amount of data under each category, and perform weighted aggregation of feature extractor parameters based on the amount of data from the clients. After the aggregation is completed, the global model redistributes the model parameters to each local model, and finally completes one round of communication.

[0101]

[0102]

[0103]

[0104] in Indicates the client participating in the training. During the aggregation process, tags are targeted. Aggregate weights, Indicates the client belong The amount of data in the class and These represent the clients participating in the training in the current round. And the amount of data for all clients.

[0105] In one embodiment, in practical engineering, the rotating machinery operation and maintenance data from various clients exhibit non-independent and identically distributed characteristics and missing labels. Furthermore, local data cannot be shared due to privacy concerns, leading to low generalization performance of the operation and maintenance model. This invention proposes a semi-supervised federated learning method. The core idea is to aggregate local labels from each client to form a global label, which guides data feature alignment, avoiding model drift caused by heterogeneous client data. Additionally, unlabeled data is used as a supplement to model training, achieving semi-supervised learning. In this method, the global model (server) and local models (clients) are based on the same model architecture, which includes a feature extractor and a label encoder. Local models complete training using local data and upload model parameters to the server. The global model weightedly aggregates the parameters of each local model and shares them with each local model, completing one round of model communication. The global model reaches convergence through multiple rounds of communication. A classification task verifies that this method can improve the generalization performance of the operation and maintenance model even with heterogeneous client operation and maintenance data.

[0106] In one embodiment, to enable those skilled in the art to better understand the technical solution of the present invention, a bearing fault diagnosis dataset is used as an example, which includes five types of data: ball cracks, inner ring cracks, outer ring cracks, inner and outer ring cracks, and healthy data. The present invention will be further described in detail below with reference to the accompanying drawings.

[0107] A semi-supervised federated learning method for heterogeneous operation and maintenance data of rotating machinery includes the following steps:

[0108] S1: Load and normalize the rotating machinery operation and maintenance data from 5 clients. Some of the data has been labeled. Divide the data into training, test and validation sets for the 5 clients. In each round of communication, 3 clients are randomly selected to participate in training. In the field of federated learning, the client represents each node participating in model training in the federated learning framework. This client is the various organizations that have rotating machinery operation and maintenance data.

[0109] Furthermore, the loaded data is normalized. The data normalization calculation method is as follows:

[0110]

[0111] in It is normalized data. It is the raw data. It is the mean of the original data. It is the standard deviation of the original data.

[0112] The bearing dataset is continuous time-series data, requiring a sliding window method to partition the data, with a window length of 100. In this specific embodiment, the dataset is divided into two types of samples: independent and identically distributed (IMDG) and non-independent and identically distributed (MIDDG) samples, for model training. This data partitioning method is only used for method verification of this invention; the specific allocation is shown in Table 1. After loading the data, 20% of the local data from each client is aggregated into a test set, and the remaining 80% of the data is used as the training set for the local model.

[0113] Table 1 Two data partitioning methods

[0114]

[0115] Where * represents unlabeled data.

[0116] S2, the server builds the global model and parameter optimizer; the global model includes a feature extractor. and tag encoder Initialize global model parameters ;

[0117] In the realm of federated learning, the server represents the central node that coordinates and manages the participating nodes; this server acts as a third-party organization managing the various institutions. Further, the tag encoder... It is to use global tags Mapped to tag encoding, the tag encoder will convert global tags The input representation is obtained by adding the word embeddings, sentence embeddings, and position embeddings. This representation is then processed through a 12-layer Transformer encoder, with 768 hidden units and 12 self-attention heads per layer. Finally, a Bi-LSTM layer and a fully connected layer are used to process the global labels of the input. Label encoding is obtained through multi-level abstraction and representation learning. The number of bidirectional LSTM units is 512, and the output dimension of the fully connected layers is 256.

[0118] Furthermore, feature extractor First, the monitoring data is mapped to a 256-dimensional latent space through a fully connected layer. Then, the summation with the position embedding is fed into a multi-head attention layer (4 attention heads). The output and input are summed and layer normalized before being fed into a feedforward connection layer. This feedforward connection layer consists of two fully connected layers. The first fully connected layer has input and output dimensions of 256 and 64, respectively, while the second fully connected layer has input and output dimensions of 64 and 256. Layer normalization is then performed, ultimately extracting 256-dimensional monitoring data features. This indicates the input of rotating machinery operation and maintenance data in the multi-head attention layer, where express The d-dimensional monitoring value at time t, and the subsequent data feature extraction process are represented as follows:

[0119]

[0120]

[0121] in and These represent the outputs of the layer normalization, This indicates the self-attention mechanism. This represents the computation of a feedforward neural network. Representation layer normalization.

[0122] After encoding the data features and labels, the dot product operation is performed, and the probability of the data belonging to each label is obtained through the softmax activation function.

[0123] Furthermore, the local model parameter optimization process is as follows: first, freeze the parameters of the label encoder, optimize and update the parameters of the feature extractor, then freeze the parameters of the feature extractor, optimize and update the parameters of the label encoder, thereby realizing the alternating update of the feature extractor and the label encoder.

[0124] S3, server collection Local tags for each client Form global tags It is then distributed to each client, with the local model (client) and the global model (server) using the same model architecture;

[0125] Furthermore, the client Based on global tags Will Each labeled data point is denoted as . ,in It is a client Data, It is data The tag is One-Hot encoded.

[0126] S4 involves multiple rounds of communication between the local and global models. Local models are trained using local data and upload their parameters to the global model. The global model weights and aggregates the local model parameters and distributes them to the various local models until the global model reaches convergence. The effectiveness of this method is verified using a classification task. Federated learning is a distributed machine learning method that allows multiple clients to collaboratively train a model without exchanging original data, ultimately learning a global model with strong generalization ability. Given the significant differences in data distribution among clients, a classification task was chosen as the application scenario for the global model. Experiments were conducted under both independent and identically distributed (i.i.d.) and non-independent i.i.d. data conditions to verify that this patent can still achieve high-accuracy classification even with heterogeneous data.

[0127] Furthermore, within the federated learning framework, 50 rounds of communication are required. In each round, three clients are selected for five training cycles. The set of clients participating in the training is as follows: The client m records N labeled data points based on the global label W. ,in It is a client Data, It is data The label is One-Hot encoded; the local model training process requires aligning local data features with global label encodings, and the alignment process is as follows:

[0128] Step 1: Generate global tag encoding. (Client) From the The updated label encoder is trained to obtain the global label encoding. , Indicates the tag name.

[0129] Step 2: Calculate the similarity distance between data features and label codes. Client From the The updated feature extractor is used to obtain data features in each training round. Calculate the similarity distance between data features and global label encoding. :

[0130]

[0131] in This represents the dot product operation. This represents the L2 norm.

[0132] Step 3: Assign pseudo-labels to unlabeled data Unlabeled data is assigned pseudo-labels based on similarity distance, according to the following rules: Local Label Determining its affiliation range based on the similarity distance of labeled data. For non-local tags, the top 2% of similar distances across all local data are considered as their category. If the minimum similarity distance of unlabeled data falls within one of the two aforementioned categories, then a pseudo-label is assigned. Otherwise, it will not participate in this round of local model training.

[0133] Furthermore, the training loss during local training includes labeled data. and pseudo-labeled data training loss The training loss uses cross-entropy loss. and The specific calculation process is as follows:

[0134]

[0135]

[0136]

[0137]

[0138] in Representing data Predicted probability for global labels This represents the softmax function. Indicates the data from the dataset Randomly selected samples Perform the expectation operation, which represents the expectation operation on the entire training set. The loss of all samples is averaged.

[0139] After the e-th cycle of training in the h-th round, the client Feature extractor in the local model and tag encoder parameters The alternating update process is as follows:

[0140]

[0141]

[0142] in Indicates the learning rate. The gradient is represented by a learning rate of 0.001 in this specific implementation.

[0143] Client The training objective is

[0144]

[0145] The training objective of the global model is

[0146]

[0147] Furthermore, during the aggregation of label encoder parameters in the global model, it is necessary to summarize the label distribution of the clients participating in the training, and perform weighted aggregation based on the amount of data under each category. The feature extractor parameters are then weighted and aggregated based on the amount of data from each client. After the aggregation is completed, the global model redistributes the model parameters to each local model, thus completing one round of communication.

[0148]

[0149]

[0150]

[0151] in Indicates the client participating in the training. During the aggregation process, tags are targeted. Aggregate weights, Indicates the client belong The amount of data in the class and These represent the clients participating in the training in the current round. And the amount of data for all clients.

[0152] The performance of the proposed method under two data distribution scenarios is evaluated using five indicators, namely: , ACC, Recall, and Recall_weighted are calculated using the following formulas:

[0153]

[0154]

[0155]

[0156]

[0157] Where P represents precision and R represents recall. Indicates a real example, Indicates a false positive example. Indicates a false negative. This indicates a true counterexample.

[0158]

[0159]

[0160] in It is a category The number of samples, where N is the total number of samples. It is a category F1 score, It is a category Recall rate.

[0161] The fault classification results are shown in Table 2:

[0162] Table 2 Federated Learning Results for Two Client Data Distributions

[0163]

[0164] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A semi-supervised federated learning method for heterogeneous operation and maintenance data of rotating machinery, characterized in that, Includes the following steps: Step S1, collect The operation and maintenance data of rotating machinery from each client was collected and normalized, with some data already labeled and divided. The training and test sets for each client are randomly selected in each round of federated learning communication. One client participated in the training; Step S2: The server builds a global model and a parameter optimizer. The global model includes a feature extractor. and tag encoder Initialize global model parameters ; Step S3, server collects Local tags for each client Form global tags It is then distributed to each client, and the client and server use the same model architecture; Step S4 involves multiple rounds of communication between the client and the server. The client completes training based on local data and uploads the model parameters to the server. The server weights and aggregates the model parameters as global model parameters and distributes the global model parameters to each client until the server reaches the convergence condition, thus obtaining the optimized global model.

2. The semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data according to claim 1, characterized in that, Preferably, the operation and maintenance data of rotating machinery is normalized, and the data normalization calculation method is as follows: ， in It is normalized data. It is the raw data. It is the mean of the original data. It is the standard deviation of the original data.

3. The semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data according to claim 1, characterized in that, Tag encoder It is to use global tags Mapped to tag encoding, the tag encoder will convert global tags The input representation is obtained by adding the word embeddings, sentence embeddings, and position embeddings. This representation is then processed through a 12-layer Transformer encoder, with 768 hidden units and 12 self-attention heads per layer. Finally, a Bi-LSTM layer and a fully connected layer are used to process the global labels of the input. Label encoding is obtained through multi-level abstraction and representation learning. The number of bidirectional LSTM units is 512, and the output dimension of the fully connected layers is 256.

4. The semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data according to claim 1, characterized in that, Feature extractor First, the monitoring data is mapped into a 256-dimensional latent space through a fully connected layer. Then, the summation with the location embedding is fed into a multi-head attention layer. The output and input are summed and normalized. This result is then fed into a feedforward connection layer, where the output and input are summed and normalized again. Finally, 256-dimensional monitoring data features are extracted. This indicates the input of rotating machinery operation and maintenance data in the multi-head attention layer, where express The d-dimensional monitoring value at time t, and the subsequent data feature extraction process are represented as follows: ， ， in and These represent the outputs of the layer normalization, This indicates the self-attention mechanism. This represents the computation of a feedforward neural network. Representation layer normalization; After encoding the data features and labels, the dot product operation is performed, and the probability of the data belonging to each label is obtained through the softmax activation function.

5. A semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data according to claim 1, characterized in that, The client parameter optimization process is as follows: First, freeze the parameters of the label encoder and optimize and update the parameters of the feature extractor. Then, freeze the parameters of the feature extractor and optimize and update the parameters of the label encoder, thereby realizing the alternating update of the feature extractor and the label encoder.

6. A semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data according to claim 1, characterized in that, Conducted within the federated learning framework Round communication, each round of communication selects Each client performs The training cycle consists of 10 periods, with the set of clients participating in the training being 100. The client m records N labeled data points based on the global label W. ,in It is a client Data, It is data The labels are One-Hot encoded; during the client-side training process, local data features and global label encodings are aligned, and the alignment process is as follows: Generate global tag encoding, client From the The updated label encoder is trained to obtain the global label encoding. , Indicates the tag name. Calculate the similarity distance between data features and label encoding. Client From the The updated feature extractor is used to obtain data features in each training round. Calculate the similarity distance between data features and global label encoding. : ， in This represents the dot product operation. Represents the L2 norm. Assign pseudo-labels to unlabeled data Unlabeled data is assigned pseudo-labels based on similarity distance, according to the following rules: local label Determining its affiliation range based on the similarity distance of labeled data. Non-local tags Then, the top 2% of all local data with similar distances are considered as their respective ranges. If the minimum similarity distance of unlabeled data falls within one of the two aforementioned categories, then a pseudo-label is assigned. Otherwise, it will not participate in this round of local model training.

7. A semi-supervised federated learning method for heterogeneous rotating machinery operation and maintenance data according to claim 6, characterized in that, Training loss including labeled data during local data training and pseudo-labeled data training loss The training loss uses cross-entropy loss. and The specific calculation process is as follows: ， ， ， ， in Representing data Predicted probability for global labels This represents the softmax function. Indicates the data from the dataset Randomly selected samples Perform the expectation operation, which represents the expectation operation on the entire training set. The loss of all samples is averaged. After the e-th cycle of training in the h-th round, the client Feature extractor in the local model and tag encoder parameters The alternating update process is as follows: ， ， in Indicates the learning rate. Represents the gradient. Client The training objective is: ， The training objective of the server is: 。 8. A system for implementing the method of any one of claims 1-7, characterized in that, It includes: The acquisition module is used for data acquisition. The operation and maintenance data of rotating machinery from each client was collected and normalized, with some data already labeled and divided. The training and test sets for each client are randomly selected in each round of federated learning communication. One client participated in the training; The server builds the global model and parameter optimizer; the global model includes a feature extractor. and tag encoder Initialize global model parameters Server collection Local tags for each client Form global tags It is then distributed to each client, and the client and server use the same model architecture; Heterogeneous units are used for multi-round communication between clients and servers. Clients complete training based on local data and upload model parameters to the server. The server weights and aggregates the model parameters as global model parameters and distributes the global model parameters to each client until the server reaches the convergence condition and obtains the optimized global model.

9. A computer storage medium, characterized in that, The storage medium includes computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-7.

10. An electronic device, characterized in that, The electronic device includes: Memory, processor, and computer programs stored in memory and executable on the processor, wherein, When the processor executes the program, it implements the method as described in any one of claims 1-7.

Images