Industrial equipment fault semi-supervised classification method based on multi-scale feature contrast learning

Abstract

The invention discloses an industrial equipment fault semi-supervised classification method based on multi-scale feature contrast learning, and the method comprises the steps: obtaining industrial equipment fault data, carrying out the preprocessing of the data, obtaining initial marked data and unmarked data, transmitting all the data into a semi-supervised time-frequency classification model for training, outputting data features through an encoder, and carrying out the training of the data features through an encoder; based on the features, multi-scale contrast learning is respectively executed on time domain and frequency domain branches to mine richer and hierarchical features, meanwhile, time domain and frequency domain branch collaborative supervision contrast learning is executed on unmarked data to improve the feature discrimination ability of the unmarked data, and finally, model parameters are optimized through multiple mixing losses, so that the unmarked data is more accurate. And a model with optimal performance is obtained. And for data to be classified, directly inputting the trained model to obtain a fault classification result. According to the method, a large amount of unmarked data can be fully utilized to improve the fault classification performance, the marked data dependence is reduced, and an efficient solution is provided for the industrial equipment fault diagnosis semi-supervised classification field.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial equipment fault diagnosis classification, and particularly to an industrial equipment fault semi-supervised classification method based on multi-scale feature contrast learning. BACKGROUND

[0002] Industrial equipment is rapidly iterating towards intelligence, large-scale and complexity, and the safe and stable operation of the equipment is directly related to production efficiency, operating cost and job safety. Whether it is a large industrial equipment such as a wind turbine or a power transformation equipment, or a general equipment such as a pump or a valve, in the long-term service process, it is easy to be affected by multiple factors such as external environmental impact, internal load loss and aging of parts, and to cause various potential faults. If the fault cannot be accurately identified and disposed in time, it will not only cause the production line to stop and the production efficiency to decline, but also may induce major safety accidents, causing huge economic losses and adverse social influence.

[0003] Traditional fault diagnosis methods mainly rely on artificial experience judgment or supervised learning models. The supervised learning model has strong feature fitting ability and shows certain advantages in fault classification tasks, but the demand for labeled samples is huge, and a large number of complete and clear fault samples are needed to complete model training. However, in the actual industrial scene, there are many difficulties in obtaining labeled fault data. Not only does the data labeling need the participation of professional technical personnel, but it is time-consuming, labor-intensive and costly, and some faults are occasional and hidden types, which are difficult to collect and label in large quantities, while unlabeled equipment operation data can be collected in large quantities through sensors and other equipment. This data supply and demand contradiction of the scarcity of labeled samples and the redundancy of unlabeled samples seriously restricts the engineering landing of supervised learning models in industrial fault diagnosis. Researchers have begun to explore semi-supervised learning technology to solve these problems. Semi-supervised learning does not need to rely on a large amount of labeled data, and can use a small amount of labeled data and a large amount of unlabeled data to train the model. By mining the potential distribution rules, feature associations and class information in the unlabeled data, it can make up for the defects of insufficient labeled samples, and then improve the generalization ability and fault classification accuracy of the model. At present, semi-supervised classification has been preliminarily explored and applied in the field of industrial equipment fault diagnosis, but the existing technology still has obvious shortcomings, such as difficulty in capturing the intrinsic association information of data, interference of noise in unlabeled data on model training, and potential performance degradation caused by introducing unlabeled data. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide an industrial equipment fault semi-supervised classification method based on multi-scale feature contrast learning, which solves the problems of noise interference of unmarked data, potential performance decline caused by introduction of unmarked data, and difficulty in capturing internal correlation information of data by single-scale feature extraction mechanism, fully learns useful feature information of a large amount of unmarked data, while avoiding introducing learning bias, strengthens the extraction and differentiation ability of the model to the fault features of the equipment under complex working conditions, effectively identifies weak and early faults, and significantly improves the classification accuracy, stability and generalization ability.

[0005] To achieve the above-mentioned purpose, the technical scheme provided by the present application is: an industrial equipment fault semi-supervised classification method based on multi-scale feature contrast learning, which is based on a semi-supervised time-frequency classification model to realize industrial equipment fault classification, the semi-supervised time-frequency classification model is composed of a time domain branch and a frequency domain branch, each branch is based on the basic structure of an encoder and a classifier to add a multi-scale contrast learning module, a collaborative supervision contrast learning module and a multiple hybrid loss learning module; the encoder is used to output deep features, shallow features and data features normalized at the last layer of the data; the multi-scale contrast learning module of the time domain branch is used to predict the future step length on the time level of the shallow features with the deep features, and the multi-scale contrast learning module of the frequency domain branch is used to compare different feature information at the instance level, through hierarchical contrast learning on the time domain branch and the frequency domain branch, more rich and multi-level feature representation can be obtained; the collaborative supervision contrast learning module is used to train the feature information learned by the time domain branch and the frequency domain branch respectively, the time domain branch generates time domain pseudo labels for unmarked data based on the robust features learned by itself, the frequency domain branch generates frequency domain pseudo labels for unmarked data based on the robust features learned by itself, and then the pseudo labels are used for supervised contrast training of the other branch, through bidirectional supervision and collaborative optimization of the two branches, the feature discrimination of unmarked data is greatly improved; finally, the labeled data and the unmarked data are jointly trained through the multiple hybrid loss learning module, the model parameters are updated for multiple iterations until the loss value is minimum, and the semi-supervised time-frequency classification model with the best performance is obtained;

[0006] The specific implementation of the industrial equipment fault semi-supervised classification method includes the following steps:

[0007] 1) Obtain industrial equipment fault data and pre-process it to obtain initial labeled data and unmarked data, the labeled data refers to data containing classification labels, and the unmarked data refers to data not containing classification labels, there are a large amount of unmarked data and a small amount of labeled data in the semi-supervised scene;

[0008] 2) Labeled and unlabeled data are fed into a semi-supervised time-frequency classification model for training. The encoder and classifier in the time-domain branch are referred to as the time-domain encoder and classifier, respectively, and the encoder and classifier in the frequency-domain branch are referred to as the frequency-domain encoder and classifier, respectively. During training, all data are first processed by the time-domain encoder and frequency-domain encoder to obtain the deep features, shallow features, and final normalized output features of the labeled and unlabeled data in the time-domain and frequency-domain branches, respectively. The data features of the labeled data are then processed by the classifiers in both the time-domain and frequency-domain branches for cross-entropy classification learning to ensure basic classification performance. The deep and shallow features of the unlabeled data are processed by the classifiers in the time-domain and frequency-domain branches for cross-entropy classification learning to ensure basic classification performance. Multi-scale contrastive learning is performed on both the domain branch and the frequency branch. Through multi-scale feature extraction and hierarchical contrast constraints, the local details and global trend information of the data at different scales are fully explored. Subsequently, the temporal data features of the unlabeled data are processed by a temporal classifier to generate temporal pseudo-labels, and the frequency data features of the unlabeled data are processed by a frequency classifier to generate frequency pseudo-labels. The two pseudo-labels are used for collaborative supervised contrastive learning to improve the discriminativeness and stability of the unlabeled data feature learning. Finally, the labeled and unlabeled data are jointly trained through a multi-hybrid loss learning module, and the multi-hybrid loss function is optimized. The weight parameters of the encoder and classifier are continuously optimized during backpropagation to obtain the semi-supervised time-frequency classification model with the best performance.

[0009] 3) For the industrial equipment fault data to be classified, the data can be directly input into the best-performing semi-supervised time-frequency classification model to obtain the final fault classification result.

[0010] Furthermore, in step 1), the industrial equipment fault data is preprocessed, including missing value imputation, noise removal, and data splitting, to obtain labeled data. and unlabeled data ,in , , Representing the A set of multiple timestamp values ​​for a sample. Representing the The true class label of each sample Encoded as a one-dimensional vector, The length is the number of categories.

[0011] Furthermore, in step 2), for the initial labeled data... and unlabeled data Simultaneously, a semi-supervised time-frequency classification model is input for training; both the time-domain encoder and the frequency-domain encoder employ a fully convolutional network (FCN). The FCN contains three convolutional blocks, each consisting of a one-dimensional convolutional layer, a batch normalization layer, and a ReLU activation function, with the number of convolutional kernels being... , the convolution kernel sizes are ; first, all data passes through the time domain encoder and the frequency domain encoder to extract the corresponding features, and the labeled data and the unlabeled data obtain the respective deep features, shallow features and the last layer normalized output data features on the time domain branch and the frequency domain branch, which are expressed as follows:

[0012] ;

[0013] ;

[0014] In the formula, , represents a plurality of timestamp value sets of a certain sample input, represents a time domain encoder, represents a frequency domain encoder, respectively represent a time domain shallow feature, a time domain deep feature and a time domain data feature of the last layer normalized output, respectively represent a frequency domain shallow feature, a frequency domain deep feature and a frequency domain data feature of the last layer normalized output;

[0015] The time domain data features and the frequency domain data features of the labeled data pass through the time domain classifier and the frequency domain classifier respectively, and both correspond to output normalized class probability distribution, and then the real class label is converted into a one-hot encoding tensor, and the cross-entropy classification loss of the time domain branch and the frequency domain branch is calculated based on the class probability distribution and the one-hot encoding tensor;

[0016] The deep features and the shallow features of the unlabeled data on the time domain branch and the frequency domain branch are used for multi-scale contrastive learning, and the specific operation steps are as follows:

[0017] The time domain deep features and the time domain shallow features of the unlabeled data are used for multi-scale contrastive learning, which is specifically multi-scale hierarchical prediction contrastive learning using deep features and shallow features on the time domain branch. First, the deep features pass through a linear layer to obtain a context vector, and then the context vector of the deep features is used to predict the future time steps of the shallow features, and the contrastive loss tries to maximize the dot product between the deep feature prediction representation and the real representation of the same sample, while minimizing the dot product between the deep and shallow features of other samples in the batch, and through multiple max-pooling operations, features of multiple scales are obtained, and the prediction contrastive learning loss is calculated one by one and summarized; wherein the multi-scale contrastive loss on the time domain branch is expressed as follows:

[0018] ;

[0019] In the formula, a context vector obtained by passing the time-domain deep feature representing the same sample through a linear layer, a time-domain shallow feature representing the same sample, a total time step representing the prediction comparison, a time-domain shallow feature representing other samples in the same batch, a set of time-domain shallow feature representation vectors representing all samples in the batch; is a linear mapping function that projects the context vector into a vector of the same dimension as the time-domain shallow feature; by calculating the dot product between the vector and the vector, the predicted value of the deep feature of the same sample in the same batch and the true value of the shallow feature are closer;

[0020] The frequency domain deep feature and the frequency domain shallow feature of the unlabeled data are used for multi-scale contrastive learning, specifically, multi-scale hierarchical instance contrastive learning is performed on the frequency domain branch using the deep feature and the shallow feature, that is, the shallow feature and the deep feature of the same sample are taken as positive samples, and the corresponding deep and shallow features of different samples are taken as negative samples; instance contrastive learning makes positive samples closer in spatial representation, and as far apart as possible from each other; then, multiple scales of features are obtained through multiple max-pooling operations, and instance contrastive learning loss is calculated and summarized one by one; wherein, the multi-scale contrastive loss on the frequency domain branch is represented as follows:

[0021] ;

[0022] In the formula, represents the shallow feature representation of the i-th sample of the frequency domain encoder, represents the deep feature representation of the i-th sample of the frequency domain encoder, represents the shallow feature representation of the i-th sample of the frequency domain encoder in the same batch, represents the deep feature representation of the i-th sample of the frequency domain encoder in the same batch, represents the batch size;

[0023] After multi-scale contrastive learning, both the time domain branch and the frequency domain branch can learn rich and hierarchical feature information; in order to obtain more effective information of the unlabeled data, a time domain and frequency domain collaborative learning mechanism is further used, so that the time domain data features of the unlabeled data pass through the classifier to generate time domain pseudo labels, and the frequency domain data features pass through the classifier to generate frequency domain pseudo labels; the two pseudo labels are used as each other's pseudo labels of the unlabeled data of the other branch, and collaborative supervised contrastive learning is performed; the specific operation steps are as follows: ​​​​​​​

[0024] On the time domain branch, given unlabeled data , the time domain encoder encodes it to get the feature representation , and then processes it through the mapping layer of the time domain branch to output the nonlinear transformation representation Therefore, the collaborative supervision contrast loss on the time domain branch is , which is expressed as:

[0025] ;

[0026] In the formula, represents the pseudo label obtained by training unlabeled data on the frequency domain branch, which is used to train the time domain branch; represents the sample set in which has the same predicted pseudo label on the time domain branch, represents the nonlinear transformation representation of the th sample of the sample set on the time domain branch, represents the nonlinear transformation representation of the th sample of all unlabeled data on the time domain branch, is the temperature coefficient;

[0027] On the frequency domain branch, given unlabeled data , first convert to frequency domain data using Fourier transform, and then convert it to frequency domain feature representation using the frequency domain encoder, and then get the nonlinear transformation representation on the frequency domain branch using the mapping layer of the frequency domain branch Therefore, the collaborative supervision contrast loss on the frequency domain branch is , which is expressed as:

[0028] ;

[0029] In the formula, represents the pseudo label obtained by training unlabeled data on the time domain branch, which is used to train the frequency domain branch; represents the sample set in which has the same predicted pseudo label on the frequency domain branch, represents the nonlinear transformation representation of the th sample of the sample set on the frequency domain branch, represents the nonlinear transformation representation of the th sample of all unlabeled data on the frequency domain branch;

[0030] Finally, a multiple hybrid loss function is designed, which integrates multiple types of losses to optimize the performance of the model, the multiple hybrid loss function includes a cross-entropy classification loss of labeled data, a multi-scale contrastive loss and a co-supervised contrastive loss of unlabeled data, and the total loss of the time domain branch and the frequency domain branch is represented as and :

[0031] ;

[0032] ;

[0033] In the formula, represents the cross-entropy classification loss of labeled data on the time domain branch, represents the cross-entropy classification loss of labeled data on the frequency domain branch, represents the co-supervised contrastive loss of unlabeled data on the time domain branch, represents the co-supervised contrastive loss of unlabeled data on the frequency domain branch, represents the multi-scale contrastive loss of unlabeled data on the time domain branch, represents the multi-scale contrastive loss of unlabeled data on the frequency domain branch, represents the weight hyperparameter of the corresponding loss term; through multiple loss optimization, the model parameters are iteratively updated, the performance of the model is optimized, and finally the semi-supervised time-frequency classification model with the best performance is obtained.

[0034] Further, in step 3), the industrial equipment fault data to be classified is input into the semi-supervised time-frequency classification model with the best performance, and is processed through the time domain branch and the frequency domain branch, that is, the time domain prediction class label is obtained through the time domain encoder and the time domain classifier on the time domain branch, and the frequency domain prediction class label is obtained through the frequency domain encoder and the frequency domain classifier on the frequency domain branch, and finally one label with the maximum class probability prediction on the time domain branch and the frequency domain branch is selected as the final fault classification result.

[0035] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0036] 1. The present application designs multi-scale contrastive learning modules on the time domain branch and the frequency domain branch based on deep features and shallow features. Through cross-time step feature prediction contrastive learning on the time domain branch, the model's ability to capture dynamic trends and long-term and short-term dependencies of time series data is enhanced. Through instance-level differentiation and aggregation of frequency domain features on the frequency domain branch, the model's sensitivity to frequency distribution differences of data is strengthened. The two modules work together to achieve multi-level feature mining.

[0037] 2、The application introduces a cooperative supervision contrast learning module of time domain branch and frequency domain branch. The time domain branch and the frequency domain branch respectively predict the unlabeled data to generate pseudo labels, and perform cross-domain supervised contrast learning. The time domain pseudo label is used as a guide signal for supervised contrast training of the frequency domain branch, and the frequency domain feature is forced to align to the time domain discriminative information. Conversely, the frequency domain pseudo label is also used to guide the supervised contrast learning of the time domain branch, and the time domain feature is pushed to fuse the frequency domain key information. Through the bidirectional supervision and cooperative optimization of the two branches, the discriminative ability of the unlabeled data is greatly improved.

[0038] 3、The application uses the time domain branch and the frequency domain branch to jointly train the labeled data and the unlabeled data, and fuse the complementary information of the time domain and frequency domain features. At the same time, a multiple mixed loss function is designed for joint optimization, realizing accurate cooperative training of the encoder and the classifier, and further improving the generalization ability and classification stability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0039] Fig. 1 The framework diagram of the method of the application.

[0040] Fig. 2 The multi-scale contrast learning schematic diagram of the time domain branch.

[0041] Fig. 3 The multi-scale contrast learning schematic diagram of the frequency domain branch. DETAILED DESCRIPTION

[0042] The application will be further described below in conjunction with specific embodiments, but the implementation of the application is not limited thereto.

[0043] As Figs. 1 to 3As shown, the embodiment discloses an industrial equipment fault semi-supervised classification method based on multi-scale feature contrast learning. The method is based on a semi-supervised time-frequency classification model to realize industrial equipment fault classification. The semi-supervised time-frequency classification model is composed of a time domain branch and a frequency domain branch. Each branch is based on the basic structure of an encoder and a classifier to add a multi-scale contrast learning module, a collaborative supervised contrast learning module, and a multi-mixed loss learning module. The encoder is used to output deep features, shallow features, and data features of the last layer of normalized output of the data. The multi-scale contrast learning module of the time domain branch is used to predict the future step length on the time level of the shallow features with the deep features. The multi-scale contrast learning module of the frequency domain branch is used to contrast different feature information on the instance level. Through hierarchical contrast learning on the time domain branch and the frequency domain branch, more rich and multi-level feature representations can be obtained. The collaborative supervised contrast learning module is used to train the feature information learned by the time domain branch and the frequency domain branch. The time domain branch generates time domain pseudo labels by predicting unlabelled data based on the robust features learned by itself. The frequency domain branch generates frequency domain pseudo labels by predicting unlabelled data based on the robust features learned by itself. Then, the pseudo labels are used for supervised contrast training of the other branch. Through bidirectional supervision and collaborative optimization of the two branches, the feature discrimination of unlabelled data can be greatly improved. Finally, the multi-mixed loss learning module is used to jointly train the labelled data and the unlabelled data. After multiple iterations of updating the model parameters to the minimum loss value, the semi-supervised time-frequency classification model with the best performance is obtained.

[0044] The specific implementation of the industrial equipment fault semi-supervised classification method includes the following steps:

[0045] 1) Obtain industrial equipment fault data and perform preprocessing to obtain initial labelled data and unlabelled data. The labelled data refers to data containing classification labels, and the unlabelled data refers to data not containing classification labels. In a semi-supervised scenario, there are a large amount of unlabelled data and a small amount of labelled data. The preprocessing of the industrial equipment fault data includes missing value filling, noise removal, and data splitting to obtain labelled data and unlabelled data , , , represents a plurality of timestamp value sets of the first sample, represents the true class label of the first sample, is encoded into a one-dimensional vector, The length of is the number of categories.

[0046] 2) the labeled data and the unlabeled data are input into the semi-supervised time-frequency classification model for training, wherein the encoder and the classifier on the time domain branch are respectively referred to as a time domain encoder and a time domain classifier, and the encoder and the classifier on the frequency domain branch are respectively referred to as a frequency domain encoder and a frequency domain classifier; during training, first, all data is processed through the time domain encoder and the frequency domain encoder respectively to obtain respective deep features, shallow features and data features of the last layer of normalization output of the labeled data and the unlabeled data on the time domain branch and the frequency domain branch; the data features of the labeled data are processed through the classifiers on the time domain branch and the frequency domain branch respectively to perform cross-entropy classification learning, thereby guaranteeing basic classification performance; the deep features and the shallow features of the unlabeled data are subjected to multi-scale contrast learning on the time domain branch and the frequency domain branch respectively, thereby fully mining local details and global trend information of the data under different scales through multi-scale feature extraction and hierarchical contrast constraint; subsequently, the time domain data features of the unlabeled data are subjected to time domain classification to generate time domain pseudo labels, and the frequency domain data features of the unlabeled data are subjected to frequency domain classification to generate frequency domain pseudo labels, and the two pseudo labels are used for collaborative supervision contrast learning to improve discriminability and stability of feature learning of the unlabeled data; finally, the labeled data and the unlabeled data are jointly trained through a multi-mixed loss learning module to optimize a multi-mixed loss function, and the weight parameters of the encoder and the classifier are iteratively optimized in back propagation, thereby obtaining a semi-supervised time-frequency classification model with optimal performance; specifically as follows:

[0047] For the initial labeled data and the unlabeled data , the semi-supervised time-frequency classification model is simultaneously input for training; the time domain encoder and the frequency domain encoder both adopt a full convolutional network FCN, the FCN includes three convolutional blocks, each convolutional block is composed of a one-dimensional convolutional layer, a batch normalization layer and a ReLU activation function, and the number of convolutional kernels is , and the size of the convolutional kernel is ; first, all data is extracted through the time domain encoder and the frequency domain encoder to obtain respective corresponding deep features, shallow features and data features of the last layer of normalization output of the labeled data and the unlabeled data on the time domain branch and the frequency domain branch, which are expressed by the following formulas:

[0048] ;

[0049] ;

[0050] In the formulas, , , represents a plurality of timestamp value sets of a certain sample, represents the time domain encoder, represents the frequency domain encoder, respectively represent the time domain shallow layer feature, the time domain deep layer feature and the time domain data feature of the last layer normalization output, respectively represent the frequency domain shallow layer feature, the frequency domain deep layer feature and the frequency domain data feature of the last layer normalization output;

[0051] The time domain data feature and the frequency domain data feature of the labeled data are processed by a time domain classifier and a frequency domain classifier respectively, and both correspond to output normalized class probability distribution, and then the real class label is converted into a one-hot encoding tensor, and the cross-entropy classification loss of the time domain branch and the frequency domain branch is calculated based on the class probability distribution and the one-hot encoding tensor;

[0052] The deep layer feature and the shallow layer feature of the unlabeled data on the time domain branch and the frequency domain branch are used for multi-scale contrastive learning, and the specific operation steps are as follows:

[0053] The time domain deep layer feature and the time domain shallow layer feature of the unlabeled data are used for multi-scale contrastive learning, specifically, the deep layer feature and the shallow layer feature are used for multi-scale hierarchical prediction contrastive learning on the time domain branch, the deep layer feature is first processed by a linear layer to obtain a context vector, and then the context vector of the deep layer feature is used to predict the future time step of the shallow layer feature, the contrastive loss tries to maximize the dot product between the deep layer feature prediction representation and the real representation of the shallow layer feature of the same sample, while minimizing the dot product between the deep and shallow layer features of other samples in the batch, and through multiple max-pooling operations, features of multiple scales are obtained, and the prediction contrastive learning loss is calculated one by one and summarized; wherein the multi-scale contrastive loss on the time domain branch is represented as follows:

[0054] ;

[0055] wherein, represents the context vector obtained by the linear layer from the time domain deep layer feature, represents the time domain shallow layer feature of the same sample, the first representation vector, represents the total time step of the prediction comparison, represents the time domain shallow layer feature of other samples in the same batch, the first representation vector, represents the set of time domain shallow layer feature representation vectors of all samples in the batch; is a linear mapping function that projects the context vector into a vector of the same dimension as the time domain shallow layer feature; by calculating the dot product between the vector and the vector, the predicted value of the deep layer feature and the real value of the shallow layer feature of the same sample in the same batch are made closer;

[0056] The frequency domain deep features and the frequency domain shallow features of the unlabeled data are used for multi-scale contrast learning, specifically, multi-scale hierarchical instance contrast learning is performed on the frequency domain branch using the deep features and the shallow features, that is, the shallow features and the deep features of the same sample are taken as positive samples, and the corresponding deep and shallow features of different samples are taken as negative samples, instance contrast learning makes the positive samples closer in spatial representation, and makes the negative samples as far away from each other as possible, then a plurality of scales of features are obtained through a plurality of maximum pooling operations, and the instance contrast learning loss is calculated one by one and is summarized; wherein the multi-scale contrast loss on the frequency domain branch is represented as follows:

[0057] ;

[0058] In the formula, represents the shallow feature representation of the i-th sample of the frequency domain encoder, represents the deep feature representation of the i-th sample of the frequency domain encoder, represents the shallow feature representation of the i-th sample of the frequency domain encoder in the same batch, represents the deep feature representation of the i-th sample of the frequency domain encoder in the same batch, represents the batch size;

[0059] After multi-scale contrast learning, the time domain branch and the frequency domain branch can learn rich and hierarchical feature information; in order to obtain more effective information of the unlabeled data, a time domain and frequency domain collaborative learning mechanism is further used, so that the time domain data features of the unlabeled data generate time domain pseudo labels through the classifier, and the frequency domain data features generate frequency domain pseudo labels through the classifier, the two pseudo labels are used as each other's pseudo labels of the unlabeled data of the other branch, and collaborative supervised contrast learning is performed, and the specific operation steps are as follows:

[0060] On the time domain branch, given the unlabeled data , the time domain encoder encodes the data to obtain the feature representation , and then the mapping layer of the time domain branch is processed to output the nonlinear transformation representation , therefore the collaborative supervised contrast loss on the time domain branch is represented as:

[0061] ;

[0062] In the formula, represents the pseudo label obtained by training the unlabeled data on the frequency domain branch, which is used to train the time domain branch; represents the sample set in the time domain branch with the same predicted pseudo label, ​​​​​Representative sample set on the time-domain branch The Nonlinear transformation representation of a sample, Represents all unlabeled data on the time-domain branch. The Nonlinear transformation representation of a sample, It is the temperature coefficient;

[0063] In the frequency domain branch, given unlabeled data First, use Fourier transform to... The data is converted into frequency domain data, and then converted into frequency domain feature representation by a frequency domain encoder. Then, the nonlinear transform representation on the frequency domain branch is obtained by using the mapping layer of the frequency domain branch. Therefore, the cooperative supervised contrast loss on the frequency domain branch It is expressed as follows:

[0064] ;

[0065] In the formula, The pseudo-labels obtained from training unlabeled data in the time domain branch are used to train the frequency domain branch. Represents the frequency domain branch A sample set with the same predicted pseudo-label. Represents the sample set on the frequency domain branch The Nonlinear transformation representation of a sample, Represents all unlabeled data in the frequency domain branch. The Nonlinear transformation representation of a sample;

[0066] Finally, a multivariate loss function is designed, which integrates multiple types of losses to optimize model performance. This multivariate loss function includes cross-entropy classification loss for labeled data, multi-scale contrastive loss for unlabeled data, and collaborative supervised contrastive loss. The total losses for the time-domain branch and the frequency-domain branch are expressed as follows: and :

[0067] ;

[0068] ;

[0069] In the formula, The cross-entropy classification loss represents the labeled data in the time-domain branch. The cross-entropy classification loss represents the value of the labeled data on the frequency domain branch. The collaborative supervised contrastive loss represents the unlabeled data in the time-domain branch. a co-supervised contrast loss representing the unlabeled data on the frequency domain branch, a multi-scale contrast loss representing the unlabeled data on the time domain branch, a multi-scale contrast loss representing the unlabeled data on the frequency domain branch, a weight hyperparameter representing the corresponding loss term; through multi-loss optimization, the model parameters are iteratively updated, the model performance is optimized, and finally a semi-supervised time-frequency classification model with optimal performance is obtained.

[0070] 3) For the industrial equipment fault data to be classified, the performance-optimal semi-supervised time-frequency classification model is directly inputted, and the final fault classification result is obtained, specifically as follows:

[0071] The industrial equipment fault data to be classified are processed through the time domain branch and the frequency domain branch, that is, the time domain encoder and the time domain classifier are processed on the time domain branch to obtain the time domain predicted class label, the frequency domain encoder and the frequency domain classifier are processed on the frequency domain branch to obtain the frequency domain predicted class label, and finally the label with the maximum class probability prediction on the time domain branch and the frequency domain branch is selected as the final fault classification result.

[0072] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited by the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement modes and shall be included in the protection scope of the present application.

Claims

1. An industrial equipment fault semi-supervised classification method based on multi-scale feature contrast learning, characterized in that, The method is based on a semi-supervised time-frequency classification model to realize industrial equipment fault classification, the semi-supervised time-frequency classification model is composed of a time domain branch and a frequency domain branch, each branch is based on the basic structure of an encoder and a classifier to add a multi-scale contrast learning module, a collaborative supervision contrast learning module and a multiple hybrid loss learning module; the encoder is used to output deep features, shallow features and data features of the last layer of normalization output of the data; the multi-scale contrast learning module of the time domain branch is used to predict the future step length on the time level of the shallow features by using the deep features, and the multi-scale contrast learning module of the frequency domain branch is used to compare the different feature information on the instance level; through the hierarchical contrast learning on the time domain branch and the frequency domain branch, more rich and multi-level feature representation can be obtained; the collaborative supervision contrast learning module is used to train the feature information learned by the time domain branch and the frequency domain branch respectively, the time domain branch is used to predict and generate time domain pseudo labels for unmarked data based on the robust features learned by itself, the frequency domain branch is used to predict and generate frequency domain pseudo labels for unmarked data based on the robust features learned by itself, and then the pseudo labels are used for the supervised contrast training of the other branch; through the bidirectional supervision and collaborative optimization of the two branches, the feature discriminability of the unmarked data can be greatly improved; Finally, the labeled data and the unmarked data are jointly trained by the multiple hybrid loss learning module, the model parameters are updated for multiple iterations until the loss value is minimum, and the semi-supervised time-frequency classification model with the best performance is obtained; The specific implementation of the industrial equipment fault semi-supervised classification method includes the following steps: 1) Obtain industrial equipment fault data and perform preprocessing to obtain initial labeled data and unmarked data, the labeled data refers to data containing classification labels, and the unmarked data refers to data not containing classification labels, in the semi-supervised scene, there are a large amount of unmarked data and a small amount of labeled data; 2) The labeled data and unlabeled data are input into the semi-supervised time-frequency classification model for training, wherein the encoder and classifier on the time domain branch are respectively referred to as a time domain encoder and a time domain classifier, and the encoder and classifier on the frequency domain branch are respectively referred to as a frequency domain encoder and a frequency domain classifier; during training, first, all data are processed through the time domain encoder and the frequency domain encoder respectively to obtain respective deep features, shallow features and data features of the last layer normalization output of the labeled data and the unlabeled data on the time domain branch and the frequency domain branch; the data features of the labeled data are processed through the classifiers respectively on the time domain branch and the frequency domain branch for cross-entropy classification learning to ensure basic classification performance; the deep features and the shallow features of the unlabeled data are subjected to multi-scale contrastive learning respectively on the time domain branch and the frequency domain branch to fully mine local details and global trend information of the data under different scales through multi-scale feature extraction and hierarchical contrastive constraint; subsequently, the time domain data features of the unlabeled data are subjected to the time domain classifier to generate time domain pseudo labels, and the frequency domain data features of the unlabeled data are subjected to the frequency domain classifier to generate frequency domain pseudo labels, and the two pseudo labels are used for collaborative supervision contrastive learning to improve the discriminability and stability of the unlabeled data feature learning; finally, the labeled data and the unlabeled data are jointly trained through a multi-mixed loss learning module to optimize a multi-mixed loss function, and the weight parameters of the encoder and the classifier are iteratively optimized in back propagation to obtain a semi-supervised time-frequency classification model with optimal performance; 3) The industrial equipment fault data to be classified is directly input into the semi-supervised time-frequency classification model with optimal performance to obtain a final fault classification result.

2. The industrial equipment failure semi-supervised classification method based on multi-scale feature contrastive learning according to claim 1, characterized in that, In step 1), the industrial equipment failure data is preprocessed, including missing value filling, noise removal and data splitting, to obtain labeled data and unlabeled data , , , represent a plurality of timestamp value sets of the first sample, represent the true class label of the first sample, is encoded into a one-dimensional vector, The length of the category number is.

3. The industrial equipment failure semi-supervised classification method based on multi-scale feature contrastive learning according to claim 2, characterized in that, In step 2), for the initial labeled data and unlabeled data , a semi-supervised time-frequency classification model is inputted for training; the time domain encoder and the frequency domain encoder both adopt a full convolution network (FCN), the FCN comprises three convolution blocks, each convolution block is composed of a one-dimensional convolution layer, a batch normalization layer and a ReLU activation function, and the number of convolution kernels is , and the size of the convolution kernel is ; first, all data are extracted by the time domain encoder and the frequency domain encoder to obtain corresponding features, and the labeled data and the unlabeled data obtain respective corresponding deep features, shallow features and data features of the last layer normalization output on the time domain branch and the frequency domain branch, and the formula is as follows: ； ； In the formula, , denote a plurality of sets of time stamp values of a certain sample input, denote a time domain encoder, denote a frequency domain encoder, denote a time domain shallow layer feature, a time domain deep layer feature, and a time domain data feature of a normalized output of a last layer, respectively, denote a frequency domain shallow layer feature, a frequency domain deep layer feature, and a frequency domain data feature of a normalized output of a last layer, respectively; The time domain data features and the frequency domain data features of the labeled data are processed through the time domain classifier and the frequency domain classifier respectively, and normalized class probability distributions are output, and then the real class labels are converted into one-hot encoding tensors, and cross-entropy classification losses of the time domain branch and the frequency domain branch are calculated based on the class probability distributions and the one-hot encoding tensors; The deep features and the shallow features of the unlabeled data on the time domain branch and the frequency domain branch are used for multi-scale contrastive learning, and the specific operation steps are as follows: The time domain deep features and the time domain shallow features of the unlabeled data are used for multi-scale contrast learning, specifically, multi-scale hierarchical prediction contrast learning is performed on the time domain branch using the deep features and the shallow features, first, the deep features are processed through a linear layer to obtain a context vector, then the context vector of the deep features is used to predict the future of the shallow features of the same sample, while minimizing the dot product between the deep and shallow features of other samples in the batch, and through multiple max-pooling operations, features of multiple scales are obtained, and the prediction contrast learning loss is calculated one by one and summarized; wherein the multi-scale contrast loss on the time domain branch is represented as follows: ​ ； In the formula, represents the context vector obtained by the linear layer from the time domain deep feature, represents the time domain shallow feature of the same sample, represents the first representation vector, represents the total time step of the prediction comparison, represents the time domain shallow feature of other samples in the same batch, represents the first representation vector, represents the representation vector set of the time domain shallow feature of all samples in the batch; is a linear mapping function, which projects the context vector into a vector with the same dimension as the time domain shallow feature; by calculating the dot product between the vector and the vector, the predicted value of the deep feature of the same sample in the same batch is closer to the true value of the shallow feature. The frequency domain deep features and the frequency domain shallow features of the unlabeled data are used for multi-scale contrast learning, specifically, multi-scale hierarchical instance contrast learning is performed on the frequency domain branch using the deep features and the shallow features, that is, the shallow features and the deep features of the same sample are taken as positive samples, and the corresponding deep and shallow features of different samples are taken as negative samples, instance contrast learning makes the positive samples closer in spatial representation, and makes the negative samples as far away from each other as possible, then a plurality of scales of features are obtained through a plurality of maximum pooling operations, and the instance contrast learning loss is calculated one by one and summarized; wherein the multi-scale contrast loss on the frequency domain branch is as follows: is as follows: ； wherein, a shallow feature representation of the i-th sample of the frequency domain encoder, a deep feature representation of the i-th sample of the frequency domain encoder, a shallow feature representation of the i-th sample of the frequency domain encoder, a deep feature representation of the i-th sample of the frequency domain encoder, a shallow feature representation of the i-th sample of the frequency domain encoder within the same batch, a deep feature representation of the i-th sample of the frequency domain encoder within the same batch, a deep feature representation of the i-th sample of the frequency domain encoder within the same batch, a deep feature representation of the i-th sample of the frequency domain encoder within the same batch, represents a batch size; After multi-scale contrastive learning, the time domain branch and the frequency domain branch can learn rich and hierarchical feature information; in order to obtain more effective information of the unlabeled data, a time domain and frequency domain collaborative learning mechanism is further used to make the time domain data features of the unlabeled data generate time domain pseudo labels through the classifier, and the frequency domain data features generate frequency domain pseudo labels through the classifier, and the two pseudo labels are used as pseudo labels of the unlabeled data of the other branch for collaborative supervision contrastive learning, and the specific operation steps are as follows: On the time domain branch, given unlabeled data is encoded by the time domain encoder to obtain a feature representation After the mapping layer processing of the time domain branch, a nonlinear transformation representation is output Therefore, the collaborative supervised contrast loss on the time domain branch is represented as: ； In the formula, The pseudo-labels obtained from training unlabeled data in the frequency domain branch are used to train the time domain branch. Represents the time-domain branch A sample set with the same predicted pseudo-label. Representative sample set on the time-domain branch The Nonlinear transformation representation of a sample, Represents all unlabeled data on the time-domain branch. The Nonlinear transformation representation of a sample, It is the temperature coefficient; In the frequency domain branch, given unlabeled data , first use Fourier transform to convert into frequency domain data, and then pass through the frequency domain encoder to convert into frequency domain feature representation , and then use the mapping layer of the frequency domain branch to obtain the nonlinear transformation representation in the frequency domain branch Therefore, the collaborative supervision contrast loss in the frequency domain branch is as follows: ； In the formula, represents the pseudo label obtained by training the unlabeled data on the time domain branch, and is used for training the frequency domain branch; represents the pseudo label obtained by training the unlabeled data on the frequency domain branch, represents a sample set with the same predicted pseudo label in the middle, represents a sample set in the frequency domain branch represents a nonlinear transformation representation of the first sample of the sample set in the frequency domain branch, represents a nonlinear transformation representation of the first sample of all unlabeled data in the frequency domain branch; and represents a nonlinear transformation representation of the first Finally, a multi-mixed loss function is designed, which integrates multiple types of losses to optimize the model performance, including the cross-entropy classification loss of labeled data, the multi-scale contrastive loss and the co-supervised contrastive loss of unlabeled data, and the total losses of the time domain branch and the frequency domain branch are respectively represented as and : ； ； In the formula, a cross-entropy classification loss representing the labeled data on the time domain branch, a cross-entropy classification loss representing the labeled data on the frequency domain branch, a co-supervised contrast loss representing the unlabeled data on the time domain branch, a co-supervised contrast loss representing the unlabeled data on the frequency domain branch, a multi-scale contrast loss representing the unlabeled data on the time domain branch, a multi-scale contrast loss representing the unlabeled data on the frequency domain branch, a weight hyperparameter representing a corresponding loss term; through multi-loss optimization, the model parameters are iteratively updated, the model performance is optimized, and finally a semi-supervised time-frequency classification model with optimal performance is obtained.

4. The industrial equipment failure semi-supervised classification method based on multi-scale feature contrastive learning according to claim 3, characterized in that, In step 3), the industrial equipment fault data to be classified is input into the semi-supervised time-frequency classification model with optimal performance, and is processed through the time domain branch and the frequency domain branch respectively, that is, a time domain predicted class label is obtained through the time domain encoder and the time domain classifier on the time domain branch, and a frequency domain predicted class label is obtained through the frequency domain encoder and the frequency domain classifier on the frequency domain branch, and finally a label with the maximum class probability prediction on the time domain branch and the frequency domain branch is selected as a final fault classification result.

Images