Lightweight bearing fault diagnosis algorithm based on probability decoupling knowledge distillation and meta-learning

Abstract

The invention discloses a lightweight bearing fault diagnosis algorithm based on probability decoupling knowledge distillation and meta-learning. The algorithm comprises the following steps: 1) collecting and preprocessing bearing data; 2) constructing an algorithm model, wherein the model is composed of a teacher model and a student model; 3) performing parameter initialization and efficient training on the teacher model by means of an unknown meta learning algorithm so as to construct a teacher model with strong performance; 4) training the student model through a probability decoupling knowledge distillation method, so that the student model can converge rapidly under the condition of small samples; and 5) performing experimental verification on the trained student model. The lightweight bearing fault diagnosis algorithm based on probability decoupling knowledge distillation and meta learning is provided through meta learning and knowledge distillation methods, and the problems that in the field of bearing intelligent fault diagnosis, the number of samples is small, the diagnosis precision is lowered, and a large neural network model cannot be deployed are solved. The method is an efficient bearing fault diagnosis method.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of bearing fault diagnosis in mechanical health management, and discloses a lightweight bearing fault diagnosis algorithm based on probability decoupling knowledge distillation and meta-learning. BACKGROUND

[0002] Rolling bearings, as key components of rotating machinery in the industrial field, are widely used in many fields such as automobiles, aircraft, industrial machinery, motors and household appliances. Its main function is to support rotating parts, significantly reduce the friction coefficient, effectively improve the operating efficiency and reliability of the equipment, and further prolong the service life of the equipment. However, under complex and variable actual working conditions and harsh environmental conditions, rolling bearings are prone to damage, which will have a negative impact on the overall performance of the equipment. According to the statistical data of the authoritative agencies, the proportion of bearing failure in all mechanical failures is about 45%-70%. Therefore, it is necessary to carry out diagnosis for bearing failure. This not only can supervise the normal operation state of the equipment in real time, but also can find and replace the bearing in time when the failure occurs, so as to effectively avoid more serious losses caused by bearing failure and ensure the safety and stability of industrial production.

[0003] In order to effectively reduce the complexity of the model and improve the diagnosis accuracy of the model under small sample and certain interference conditions, a lightweight bearing fault diagnosis algorithm based on probability decoupling knowledge extraction and meta-learning is proposed. The whole algorithm process is as follows: first, a four-layer mixed convolutional network is used to build a teacher model, and a two-layer mixed convolutional network is used to build a student model. Then, the teacher model is meta-trained to obtain an initialized parameter, and the trained teacher model is used to distill the knowledge of the student model through general formal training, so that the student model can learn the small sample recognition ability of the teacher model. Finally, the trained student convolutional network is used to classify the faults. This method can effectively improve the small sample learning ability and fully excavate the key feature information, so as to realize accurate diagnosis under small sample condition. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a lightweight bearing fault diagnosis algorithm based on probability decoupling knowledge distillation and meta-learning, which provides an innovative idea for bearing fault diagnosis research and deployment model real-time monitoring under small sample condition.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: a lightweight bearing fault diagnosis algorithm based on probability decoupling knowledge distillation and meta-learning, characterized in that the steps are as follows:

[0006] 1) Data acquisition and preprocessing of laboratory bearings are carried out to form a.mat file dataset of bearing vibration signals;

[0007] 1.1) In the 2HP load working condition, the sensor is installed in the 12 o'clock direction of the bearing to collect diagnostic signals at a sampling frequency of 5KHz, and data containing four types of normal state, rolling element fault, inner ring fault and outer ring fault are collected at rotating speeds of 950r / min, 1000r / min and 1050r / min respectively; wherein each fault type corresponds to three damage diameters of 0.2mm, 0.3mm and 0.4mm, and after data collection, the data is normalized to finally form a.mat file dataset of bearing vibration signals;

[0008] 1.2) The processed data is set to have 2048 segment starting points, and the step is set to 28 to resample the data, and the data is divided according to the proportions of 20%, 10% and 70% of the training set, the validation set and the test set.

[0009] 2) Construct an algorithm model, which consists of a teacher model and a student model;

[0010] 2.1) For the original vibration signal, first extract the short-time feature using a wide kernel convolution filter, then use ReLU as the activation function after convolution operation, and then normalize the output value through batch normalization;

[0011] 2.2) Use deep separable convolution and residual network to construct teacher models and student models of different depths;

[0012] 2.3) Use a four-layer hybrid convolution network to construct the teacher model architecture, and a two-layer hybrid convolution network to construct the student model, and the convolution kernel size used by the wide convolution and the deep separable convolution is 64x1;

[0013] 3) Initialize and train the teacher model with the meta-unknowing learning algorithm;

[0014] Meta-unknowing learning randomly extracts n pseudo-training sets P1, P2, P3,..., P n from a large amount of known data, and corresponding pseudo-test sets are selected, and multiple pseudo-training sets are used to adjust the initial parameters of the teacher model;

[0015] The cross-entropy is used as the loss function in meta-training, and its formula is Where α is the learning rate of meta-training, θ is the randomly initialized network parameter before meta-training, and each meta-training set participates in the training of the model, starting from the initial parameter θ

[0016] of the model, after calculating the loss function, the gradient method is used to update the parameters of such pseudo-training sets

[0017] After each meta-training set is trained, their loss functions are added, as shown in the following formula:

[0018]

[0019] In the formula, L meta represents the cross-entropy loss after the i-th pseudo-training set is trained, and finally the model is updated by L meta The initial parameters of the model are updated to realize model initialization. The formula is as follows:

[0020]

[0021] Where β is the learning rate of updating the model parameters.

[0022] Subsequently, let Q = {q(x1), q(x2),..., q(x N )} represent the class label of a migration set x, and the teacher network is first trained by a hard label, and the Softmax layout is used as a soft label to guide the student network. When training the teacher network, its loss function L T (Q, P T ) is defined as the cross-entropy between the output probability distribution and the class label, which is represented as:

[0023]

[0024] In the formula, q(x i ) is the sample class label, and p T (x i ) is the output value of the teacher model.

[0025] 4) Train the student model by the probability decoupling knowledge distillation method;

[0026] The probability knowledge distillation can consider both the probability distribution and the class label of the sample. The probability distribution is used to maintain the geometric relationship of the teacher feature space in the low-dimensional space of the student network, and is used to enhance the mapping relationship between the classification label and other migration set data.

[0027] For the student network, the hard label provides a clear classification target, which gives the student model a clear optimization direction, and as a reliable initial learning basis, it can quickly establish basic discrimination ability; therefore, the loss function L S of the student model has both L hard and L soft , as shown below.

[0028] L S (Q, P T , P S ) = (1-α)·L hard (Q, P S)+ aL soft (P T , P S ).

[0029] where a is the weighting coefficient balancing the two loss functions, L hard (Q, P S ) is the cross-entropy loss between the class labels and the output probability distribution of the student network, and L soft (Q, P S ) is the KL divergence between the teacher and student networks.

[0030] The loss L hard (Q, P S ) is defined as the cross-entropy between the class labels and the output probability distribution of the student network, calculated as:

[0031]

[0032] where q(x i ) is the sample class label, and p S (x i ) is the output value of the student model.

[0033] The loss L soft (Q, P S ) uses the KL divergence to calculate the approximation degree of PD between the teacher and student networks, expressed as:

[0034]

[0035] where q(x i ) is the sample class label, p S (x i ) is the output value of the student model, and p T (x i ) is the output value of the teacher model.

[0036] The conditional probability represents the probability of each sample selecting each of its neighbors, and for the teacher and student networks, the conditional probability is defined as:

[0037]

[0038] where y S , y T are the outputs within the layers of the student and teacher models. K(·) is the kernel function, and here the cosine kernel function is selected:

[0039]

[0040] where a and b are the inputs of the teacher or student network, respectively.

[0041] Finally, the derived L PDKD expression is:

[0042]

[0043] wherein denotes the KL loss between the target class teacher and student probabilities, called target class knowledge distillation (TCKD); denotes the KL loss between the non-target class teacher and student probabilities, called non-target class knowledge distillation (NCKD), and two hyperparameters beta and gamma represent the weights of target class distillation and non-target class distillation respectively, used to balance the proportion between the two.

[0044] 5) After the model is trained, if the student model fails to successfully classify the fault or fails to learn small sample extraction ability, the parameters and the teacher model are adjusted, and the student model is retrained; if the student model successfully classifies and learns small sample extraction ability, a final classification model is obtained;

[0045] 6) The preprocessed bearing vibration data is input into the classification model obtained in step 5 for classification, so as to realize classification and identification of different fault types.

[0046] The beneficial effects of the present application are: the present application proposes a lightweight bearing fault diagnosis algorithm for probability decoupling knowledge distillation and meta-learning by the methods of meta-learning and knowledge distillation, solves the problems of decreased diagnosis precision and large neural network model that cannot be deployed in the field of bearing intelligent fault diagnosis, and is a high-efficiency bearing fault diagnosis method. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 It is a fault one-dimensional time sequence signal diagram.

[0048] Figure 2 It is an accuracy rate diagram of the model under different signal-to-noise ratios.

[0049] Figure 3 It is an architecture diagram of the teacher model and the student model.

[0050] Figure 4 It is an architecture diagram of a lightweight bearing fault diagnosis algorithm for probability decoupling knowledge distillation and meta-learning.

[0051] Figure 5 It is an accuracy rate diagram under different training samples.

[0052] Figure 6 It is a confusion matrix diagram of different fault type classification results. DETAILED DESCRIPTION

[0053] The present invention provides a new lightweight, small-sample, and deployable bearing fault diagnosis method, a lightweight bearing fault diagnosis algorithm that combines probabilistic decoupling of knowledge distillation and meta-learning. The steps are as follows:

[0054] A lightweight bearing fault diagnosis algorithm based on probabilistic decoupling knowledge distillation and meta-learning is proposed, which is characterized by the following steps:

[0055] 1) Data collection and preprocessing of laboratory bearings to form a .mat file dataset of bearing vibration signals;

[0056] 1.1) Under a 2HP load condition, a sensor was installed at the 12 o'clock position of the bearing to collect diagnostic signals at a sampling frequency of 5kHz. Data was collected at speeds of 950r / min, 1000r / min, and 1050r / min for four types of faults: normal state, rolling element fault, inner race fault, and outer race fault. Each fault type corresponded to three damage diameters: 0.2mm, 0.3mm, and 0.4mm. After collection, the data was normalized to form a .mat file dataset of the bearing vibration signal.

[0057] 1.2) Set 2048 segment starting points for the processed data and set the stride to 28 to resample the data and divide the data into training set, validation set and test set with a ratio of 20%, 10% and 70% respectively.

[0058] 2) Build an algorithm model, which consists of a teacher model and a student model;

[0059] 2.1) For the original vibration signal, a wide kernel convolution filter is first used to extract short-term features. After the convolution operation, ReLU is used as the activation function, and the output value is normalized through batch normalization.

[0060] 2.2) Use depthwise separable convolutional and residual networks to construct teacher and student models of different depths;

[0061] 2.3) A four-layer hybrid convolutional network is used to construct the teacher model architecture, and a two-layer hybrid convolutional network is used to construct the student model. The convolution kernel size used for wide convolution and depthwise separable convolution is 64×1;

[0062] 3) Initialize and train the teacher model parameters using an agnostic meta-learning algorithm;

[0063] Agnostic meta-learning randomly extracts n pseudo training sets P1, P2, P3, ..., P from a large amount of known data. n ,and Conduct corresponding pseudo test set The selection of the plurality of pseudo-training sets is used to adjust the initial parameters of the teacher model;

[0064] Cross-entropy is used as the loss function in meta-training, and its formula is Wherein, a is the learning rate of meta-training, and θ is the randomly initialized network parameter before meta-training. Each meta-training set participates in the training of the model, and the model is

[0065] The initial parameter θ starts training, and after calculating the loss function, the gradient method is used to update the parameters of such pseudo-training sets

[0066] After each meta-training set is trained, their loss functions are added, as shown in the following formula:

[0067]

[0068] In the formula, L meta represents the cross-entropy loss of the i-th pseudo-training set after training, and finally the L meta updates the initial parameters of the model to realize the initialization of the model. The formula is as follows:

[0069]

[0070] Wherein, β is the learning rate of updating the model parameters.

[0071] Subsequently, Q={q(x1),q(x2),...,q(x N )} represents the class label of a migration set x, and the teacher network is first trained by hard label, and the Softmax layout is used as soft label to guide the student network. When training the teacher network, its loss function L T (Q,P T ) is defined as the cross-entropy between the output probability distribution and the class label, which is expressed as:

[0072]

[0073] In the formula, q(x i ) is the sample class label, and p T (x i ) is the output value of the teacher model.

[0074] 4) Train the student model by the probability decoupling knowledge distillation method;

[0075] The probability knowledge distillation can consider the probability distribution and class label of the sample at the same time. The probability distribution is used to maintain the geometric relationship of the teacher feature space in the low-dimensional space of the student network, which is used to enhance the mapping relationship between the classification label and other migration set data.

[0076] For the student network, the hard label provides an explicit classification target, which makes the student model have a clear optimization direction, and as a reliable initial learning basis, it can quickly establish basic discrimination ability; therefore, the loss function L S of the student model in the hard teacher-student network is composed of L soft and L S , as shown below.

[0077] L T (Q, P S ) = (1-α)·L hard (Q, P S )+α·L soft (P T , P S ).

[0078] wherein α is a weighting coefficient for balancing the two loss functions, L hard (Q, P S ) is the class label, and L soft (Q, P S ) is the KL divergence of the teacher model and the student model.

[0079] The loss L hard (Q, P S ) is defined as the cross-entropy between the class label and the probability distribution output by the student network, and the calculation formula is as follows:

[0080]

[0081] wherein q(x i ) is the sample class label, and p S (x i ) is the output value of the student model.

[0082] The loss L soft (Q, P S ) uses the KL divergence to calculate the approximation degree of the PD between the teacher and student networks, and is expressed as:

[0083]

[0084] wherein q(x i ) is the sample class label, p S (x i ) is the output value of the student model, and p T (x i ) is the output value of the teacher model.

[0085] The conditional probability represents the probability of each sample selecting each neighbor, and for the teacher and student networks, the conditional probability is defined as:

[0086]

[0087] Where y S ,y T is the output of the student and teacher model layers. K(·) is the kernel function, and the cosine kernel function is selected here:

[0088]

[0089] Where a and b are the inputs to the teacher or student network respectively;

[0090] Finally, the obtained L PDKD The expression is:

[0091]

[0092] in Represents the KL loss between the target class teacher and student probabilities, called target class knowledge distillation TCKD; It represents the KL loss between the non-target class teacher and student probabilities, which is called non-target class knowledge distillation NCKD. The two hyperparameters β and γ represent the weights of target class distillation and non-target class distillation, respectively, and are used to balance the ratio between the two.

[0093] 5) Conduct experimental verification on the trained model. If the student model fails to successfully classify the fault or fails to learn the ability to extract small samples, adjust the parameters and teacher model and retrain the student model. If the student model successfully classifies the fault and learns the ability to extract small samples, the final classification model is obtained.

[0094] 6) Input the preprocessed bearing vibration data into the classification model obtained in step 5 for classification to achieve classification and identification of different fault types.

[0095] Example 1:

[0096] The teacher model is meta-trained using the bearing dataset of the University of Paderborn in Germany, and the bearing dataset and laboratory dataset of Case Western Reserve University in the United States are used for knowledge distillation training, validation, and testing.

[0097] Step 1: A dataset of artificial damage loads was selected from two operating conditions, N15M01F10 and N15M07F10, from the University of Paderborn in Germany for agnostic meta-learning training. A dataset of natural damage loads was selected from the N15M01F10 operating state for agnostic meta-learning testing. The Case Western Reserve University bearing dataset and laboratory dataset were preprocessed and normalized, resampled using a coded sliding window, and the training set was divided into 20, 10, and 100 partitions.

[0098] Step 2: Set the model's running environment and hyperparameter variables

[0099] All methods are coded in PyTorch 2.0 in Python 3.11 version and run on GPU NVIDIA GeForce RTX 4070Ti to ensure the consistency and repeatability of the experimental environment. Stochastic gradient descent (SGD) is used as the optimizer. The training rounds are set to 50. α = 0.2, β = 5, γ = 0.1.

[0100] Step 3: Construct a lightweight bearing fault diagnosis algorithm model that decouples the probability of knowledge distillation and meta-learning

[0101] The main modules include wide convolution module, deep convolution module, point convolution module, residual module, global average pooling, fully connected layer, etc., and the specific parameters of the model are as follows:

[0102] Table 1 Teacher model parameters

[0103]

[0104]

[0105] Table 2 Student model parameters

[0106]

[0107] Step 4: Compared with other lightweight models, the model proposed in this paper shows excellent performance under various noise levels and operating conditions. The evaluation indicators are accuracy and F1 score.

[0108]

[0109] In the formula, TP is the true positive, TN is the true negative, FP is the false positive, and FN is the false negative.

[0110] In order to reflect the anti-noise ability of the model, we added Gaussian white noise with different signal-to-noise ratios to the original bearing data, and the specific formula is as follows.

[0111]

[0112] P signal represents the power of the signal, P noise represents the noise power.

[0113] Table 3 Experimental results of different comparison models

[0114]

[0115] The application provides a lightweight bearing fault diagnosis algorithm for probability decoupling of knowledge distillation and meta-learning, realizes lightweight and small sample bearing fault diagnosis, has high accuracy and stability, and provides value in practical application.

Claims

1. A lightweight bearing fault diagnosis algorithm based on probabilistic decoupling knowledge distillation and meta-learning, characterized by: The steps are: 1) Data collection and preprocessing of laboratory bearings to form a .mat file dataset of bearing vibration signals; 2) Build an algorithm model, which consists of a teacher model and a student model; 2.1) For the original vibration signal, a wide kernel convolution filter is first used to extract short-term features. After the convolution operation, ReLU is used as the activation function, and the output value is normalized through batch normalization. 2.2) Use depthwise separable convolutional and residual networks to construct teacher and student models of different depths; 2.3) A four-layer hybrid convolutional network is used to construct the teacher model architecture, and a two-layer hybrid convolutional network is used to construct the student model. The convolution kernel size used for wide convolution and depthwise separable convolution is 64×1; 3) Initialize and train the teacher model parameters using an agnostic meta-learning algorithm; 4) Train the student model through probabilistic decoupling knowledge distillation method; 5) Conduct experimental verification on the trained model. If the student model fails to successfully classify the fault or fails to learn the small sample extraction capability, adjust the parameters and teacher model and retrain the student model. If the student model successfully classifies and learns the ability to extract small samples, the final classification model is obtained; 6) Input the preprocessed bearing vibration data into the classification model obtained in step 5 for classification to achieve classification and identification of different fault types.

2. A lightweight bearing fault diagnosis algorithm based on probabilistic decoupling knowledge distillation and meta-learning according to claim 1, characterized in that: In the step 1, the specific method is: 1.1) Under a 2HP load condition, a sensor was installed at the 12 o'clock position of the bearing to collect diagnostic signals at a sampling frequency of 5kHz. Data was collected at speeds of 950r / min, 1000r / min, and 1050r / min for four types of faults: normal state, rolling element fault, inner race fault, and outer race fault. Each fault type corresponded to three damage diameters: 0.2mm, 0.3mm, and 0.4mm. After collection, the data was normalized to form a .mat file dataset of the bearing vibration signal. 1.2) Set 2048 segment starting points for the processed data and set the stride to 28 to resample the data and divide the data into training set, validation set and test set with a ratio of 20%, 10% and 70% respectively.

3. The lightweight bearing fault diagnosis algorithm based on probabilistic decoupling knowledge distillation and meta-learning according to claim 1 is characterized in that: In the step 3), the specific method is: Agnostic meta-learning randomly extracts n pseudo training sets P1, P2, P3, ..., P from a large amount of known data. n ,and Conduct corresponding pseudo-test sets Selection of,using multiple pseudo training sets to adjust the initial parameters of the teacher model; Cross entropy is used as the loss function in meta-training, and its formula is Among them, α is the learning rate of meta-training, θ is the random initial network parameter before meta-training, each meta-training set participates in the training of the model, starting from the initial parameter θ of the model. After calculating the loss function, the parameters of this pseudo-training set are updated using the gradient method. After each meta-training set is trained, their loss functions are added together as shown below: Where L meta Represents the cross entropy loss after the i-th pseudo training set training is completed, and finally through L meta Update the initial parameters of the model to initialize the model; the formula is as follows: Among them, β is the learning rate for updating model parameters; Then, use Q = {q(x1),q(x2),...,q(x N )} represents the class label of a migration set x. The teacher network is first trained with hard labels, and its Softmax layout is used as a soft label to guide the student network. When training the teacher network, its loss function L T (Q,P T ) is defined as the cross entropy between the output probability distribution and the category label, expressed as: Where q(x i ) is the sample category label, p T (x i ) is the output value of the teacher model.

4. The lightweight bearing fault diagnosis algorithm based on probabilistic decoupling knowledge distillation and meta-learning according to claim 1 is characterized in that: The specific method of step 4) is: Probabilistic knowledge distillation can simultaneously consider the probability distribution and category labels of samples. The probability distribution is used to maintain the geometric relationship between the teacher feature space and the low-dimensional space of the student network, and to enhance the mapping relationship between the classification labels and other migration set data. For the student network, hard labels provide clear classification targets, giving the student model a clear optimization direction and serving as a reliable initial learning basis to quickly establish basic differentiation capabilities. Therefore, the loss function L of the student model is S There are both L hard There is also L soft , as shown below: 50 S (Q, P T ,P S )=(1-α)·L hard (Q, P S )+α·L soft (P T ,P S ). Where α is the weighted coefficient to balance the two loss functions, L hard (Q,P S ) is the category label, L soft (Q,P S ) is the KL divergence between the teacher model and the student model; Loss L hard (Q,P S ) is defined as the cross entropy between the class label and the probability distribution of the student network output, and is calculated as: Where q(x i ) is the sample category label, p S (x i ) is the output value of the student model; Loss L soft (Q,P S ) KL divergence is used to calculate the approximation of PD between the teacher and student networks, expressed as: Where q(x i ) is the sample category label, p S (x i ) is the output value of the student model, p T (x i ) is the output value of the teacher model; The conditional probability expresses the probability of each sample selecting each of its neighbors. For the teacher and student networks, the conditional probabilities are defined as: Where y S ,y T is the output of the student and teacher model layers; K(·) is the kernel function, and the cosine kernel function is selected here: Where a and b are the inputs to the teacher or student network respectively; Finally, the obtained L PDKD The expression is: in Represents the KL loss between the target class teacher and student probabilities, called target class knowledge distillation TCKD; It represents the KL loss between the non-target class teacher and student probabilities, which is called non-target class knowledge distillation NCKD. The two hyperparameters β and γ represent the weights of target class distillation and non-target class distillation, respectively, and are used to balance the ratio between the two.

Images