Rolling bearing intelligent fault diagnosis method based on integral knowledge distillation

Abstract

The invention discloses a rolling bearing intelligent fault diagnosis method based on integral knowledge distillation, and the method comprises the steps: 1, collecting rolling bearing vibration signals in different health states, carrying out the preprocessing of the rolling bearing vibration signals, and dividing the rolling bearing vibration signals into a training set and a test set; 2, establishing a teacher-student fault diagnosis model, pre-training a teacher model by using the training set, and obtaining and storing optimal parameters of the teacher model; 3, loading the teacher model and the optimal parameters stored in the step 2, and performing dimension raising on interlayer features of the teacher model and the student model; step 4, calculating a loss function of relational knowledge distillation; step 5, calculating a loss function of response knowledge distillation under relaxation matching; step 6, calculating a loss function of final integral knowledge distillation; and step 7, performing knowledge distillation to obtain an optimal student model, and storing the optimal student model as a final deployment model.

Description

Technical Field

[0001] The present invention belongs to the technical field of fault diagnosis, and relates to an intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation. Background Art

[0002] As one of the important components in industrial production, bearings are widely used in industrial equipment. Bearing faults are one of the main reasons for mechanical equipment failure. When a bearing fails, it often leads to major faults in rotating mechanical equipment, causing significant economic losses. Therefore, how to quickly, efficiently, and accurately judge the operating state of bearings has become the key to bearing fault diagnosis technology.

[0003] Currently, many technologies for bearing fault diagnosis have been developed at home and abroad, including those based on traditional signal processing and traditional machine learning. These methods have all achieved the fault classification and diagnosis tasks to a certain extent, but they all have their limitations. Therefore, scholars have proposed fault diagnosis technologies based on deep learning, which have been widely used because of their advantages such as not requiring in-depth professional knowledge, automatically extracting features, and accurate classification. However, deep learning models often have high requirements for hardware, so there are problems such as difficulty in actual deployment in industrial applications.

[0004] To solve this problem, researchers have proposed the knowledge distillation method, that is, the knowledge of a large teacher model is transferred to a small student network that occupies less space and is easier to deploy through a teacher-student model. Currently, the more popular distillation methods can be roughly divided into three categories according to the distillation content, namely response-based, feature-based, and relationship-based knowledge distillation. These methods prompt the student network to learn the knowledge learned by the teacher network during the diagnosis process, and to a certain extent, can improve the diagnosis accuracy of the student model.

[0005] However, the above methods all have the following problems: (1) Artificial data processing of the original data is required, such as feature extraction, time-frequency domain transformation, etc. to adapt to the two-dimensional convolutional layer. This requires prior knowledge in the professional field and increases the diagnosis time; (2) The student model only focuses on the output of a certain aspect of the teacher model, such as the end softmax output layer, the intermediate feature layer, etc. On the one hand, this makes the original data unable to be directly used as the input of the model, and on the other hand, it is easy to lose potential effective knowledge, so there is still a large room for improvement in the accuracy of the student model. Summary of the Invention

[0006] To solve the above technical problems, the purpose of the present invention is to provide an intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation.

[0007] The present invention provides an intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation, including:

[0008] Step 1: Collect the vibration signals of rolling bearings in different health states, preprocess them, and divide them into a training set and a test set;

[0009] Step 2: Establish a teacher-student fault diagnosis model, use the training set to pre-train the teacher model, obtain the optimal parameters of the teacher model, and save them;

[0010] Step 3: Load the teacher model and the optimal parameters saved in Step 2, and increase the dimension of the intermediate layer features of the teacher model and the student model;

[0011] Step 4: Calculate the loss function of relational knowledge distillation:

[0012] Step 5: Calculate the loss function of response-based knowledge distillation under relaxed matching:

[0013] Step 6: Calculate the loss function of the final holistic knowledge distillation:

[0014] Step 7: Conduct knowledge distillation to obtain the optimal student model, and save the optimal student model as the finally deployed model.

[0015] Further, the said Step 1 includes:

[0016] Step 1.1: Label the collected data set. The original label of the data set takes the positive label as 1 and the negative label as 0, and then normalize the data;

[0017] Step 1.2: Divide the original signals in the data set into samples with a length of 1000. The total sample capacity is 500, and the training set and the test set are divided according to a ratio of 4:1.

[0018] Further, the said Step 2 includes:

[0019] Step 2.1: Construct the teacher model: The teacher model is a deep residual network structure with a wide convolutional kernel in the first layer, including a first convolutional layer, three identical residual blocks, a second convolutional layer, a max pooling layer, and a fully connected layer connected in sequence;

[0020] The data is input into the teacher model. After passing through the first convolutional layer, it goes through batch normalization and an activation function, and then enters the three identical residual blocks in sequence. Each residual block consists of two convolutional layers and a corresponding shortcut mechanism. The data in the residual block is linearly added to itself after passing through the two convolutional layers. After passing through the three residual blocks, it enters the second convolutional layer, and finally, classification is performed through the max pooling layer and the fully connected layer to output the diagnosis result;

[0021] Step 2.2: Pre-train the teacher model: Input the training set obtained in Step 1 into the teacher model, and take the cross-entropy between the predicted output of the teacher model and the true label of the sample as the loss function:

[0022]

[0023] In the formula, score k is the k-th predicted output of the teacher model, target k is the true label of the k-th original data; K To predict the total number of outputs; use the back-propagation algorithm to update the network weights, and use the adaptive learning rate Adam algorithm and manually set learning rate decay to adjust the learning rate, effectively improving the convergence speed, preventing falling into local minima, and alleviating overfitting problems;

[0024] Step 2.3: Establish a student model: The network structure of the student model is similar to that of the teacher model, but the width of the convolution kernel of the first convolution layer of the student model is smaller than that of the teacher model; the width of the convolution kernel in the residual block remains unchanged, and the number of channels is reduced; the width of the convolution kernel and the number of channels of the second convolution layer are both smaller than those of the teacher model.

[0025] Furthermore, the step 3 is specifically as follows:

[0026]

[0027]

[0028] Where, Model for teachers The intermediate layer output under samples, Model for students The intermediate layer output under samples, It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model; It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model.

[0029] Furthermore, the step 4 is specifically as follows:

[0030] Step 4.1: Calculate the distillation loss between two points according to the following formula:

[0031]

[0032] in, Represents the distance distillation loss between two points, represents the i-th training sample, represents the jth training sample, Represents a set of relationships between two points. represents the Smooth L1 loss function, It represents the two-dimensional feature distribution after the one-dimensional features of the intermediate layer output of the $i$-th sample after inputting into the teacher model are dimensionally ascended. It represents the two-dimensional feature distribution after the one-dimensional features of the intermediate layer output of the $j$-th sample after inputting into the teacher model are dimensionally ascended; It represents the two-dimensional feature distribution after the one-dimensional features of the intermediate layer output of the $i$-th sample after inputting into the student model are dimensionally ascended, It represents the two-dimensional feature distribution after the one-dimensional features of the intermediate layer output of the $j$-th sample after inputting into the student model are dimensionally ascended; It represents and the distance between, It represents and the distance;

[0033]

[0034]

[0035] In the formula, and are distance normalization parameters, and the purpose is to solve the inconsistency in the scales of the outputs of the teacher model and the student model. Specifically:

[0036]

[0037]

[0038] In the formula, represents the set of two-point relationships;

[0039] Step 4.2: Calculate the angle distillation loss between three points according to the following formula:

[0040]

[0041] Among them, represents the angle distillation loss between three points, represents the $k$-th training sample, represents the set of three-point relationships, represents the three-point angle relationship of, represents the three-point angle relationship of, represents the two-dimensional feature distribution after the one-dimensional features of the intermediate layer output of the $k$-th sample after inputting into the teacher model are dimensionally ascended; represents the two-dimensional feature distribution after the one-dimensional features of the intermediate layer output of the $k$-th sample after inputting into the student model are dimensionally ascended;

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] Step 4.3: Obtain the following loss function for relational knowledge distillation:

[0049]

[0050] where and represent the weights of the distance distillation loss between two points and the angle distillation loss between three points, respectively.

[0051] Furthermore, the specific steps of step 5 are as follows:

[0052]

[0053]

[0054] In the formula, represents the loss function of response-based knowledge distillation under relaxed matching, represents the covariance of u and v, and represent the mean and standard deviation of u, respectively, and represent the results after calculating the deformed Softmax formula for the final responses of the teacher model and the student model, respectively. The formula is as follows:

[0055]

[0056]

[0057] where represents the response of the network model under the m-th category, represents the response of the network model under the n-th category, C represents the total number of categories in the classification task, represents the temperature of knowledge distillation.

[0058] Furthermore, the specific steps of step 6 are as follows:

[0059]

[0060] In the formula, α is the weight of the loss function of relational knowledge distillation.

[0061] Further, step 7 is specifically as follows:

[0062] Step 7.1: Call the loaded teacher model and the best parameters, input the training set data into the teacher model to obtain the intermediate layer features and perform dimensionality elevation, and at the same time obtain the final response;

[0063] Step 7.2: Input the training set data into the student model to obtain the intermediate layer features and perform dimensionality elevation, and at the same time obtain the final response;

[0064] Step 7.3: Use the loss function of the final holistic knowledge distillation for the obtained intermediate layer dimensionality elevation features and the final response to perform knowledge distillation to complete the training of the entire student model;

[0065] Step 7.4: Use the test set to test the performance of the student model, and save the best student model as the finally deployed model.

[0066] An intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation of the present invention has the following beneficial effects:

[0067] (1) In the existing bearing fault diagnosis methods based on knowledge distillation, the student model only learns the output of the Softmax at the end of the teacher model; or only considers the relationship of the intermediate layer features, and the cross entropy is forcibly used for the prediction of the student model and the actual target. In fact, there is an adversarial relationship between the two. These two schemes both limit the learning ability of the student model, resulting in low fault diagnosis accuracy. To address the above problems, the present invention proposes a new method that combines relational knowledge distillation and response-based knowledge distillation improved by using relaxed matching, reduces the antagonism between the relational distilled knowledge and the original knowledge of the student model, thereby achieving the purpose of holistic knowledge distillation and improving the fault diagnosis ability of the model.

[0068] (2) In the existing bearing fault diagnosis methods based on knowledge distillation, in order to improve the diagnosis ability of the student model, the original data is generally converted into a time-frequency diagram as the input of the model. Although this objectively improves the diagnosis ability of the model, it will cause all the convolutional layers of the student model to be two-dimensional, resulting in the inability to further compress the parameter quantity of the student model. To address the above problems, the present invention uses a student network with a one-dimensional convolutional network structure, and at the same time performs dimensionality elevation on the intermediate layer features during the distillation process to reduce the performance loss caused by using the one-dimensional convolutional network, so that the parameter quantity of the model can be further compressed during application. Description of the Drawings

[0069] Figure 1 is a flowchart of an intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation of the present invention;

[0070] Figure 2It is the network structure diagram of the teacher model in the embodiments of the present invention;

[0071] Figure 3 It is the network structure diagram of the student model in the embodiments of the present invention. Specific implementation manners

[0072] As Figure 1 shown, an intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation according to the present invention includes:

[0073] Step 1: Collect vibration signals of rolling bearings in different health states. These vibration signals represent the operating characteristics of the bearings in the axial direction, and perform preprocessing on them and divide them into a training set and a test set. The said step 1 includes:

[0074] Step 1.1: Label the collected data set. The original label of the data set takes the positive label as 1 and the negative label as 0, and then perform normalization processing on the data.

[0075] Step 1.2: Divide the original signals in the data set into samples with a length of 1000. The total sample capacity is 500, and the training set and the test set are divided according to a ratio of 4:1.

[0076] Step 2: Establish a teacher-student fault diagnosis model, use the training set to pre-train the teacher model, obtain the best parameters of the teacher model and save them. The said step 2 includes:

[0077] Step 2.1: Construct the teacher model: As Figure 2 shown, the teacher model is a deep residual network structure with a wide convolutional kernel in the first layer, including a first convolutional layer, three identical residual blocks, a second convolutional layer, a max pooling layer, and a fully connected layer connected in sequence.

[0078] The data is input into the teacher model. After passing through the first convolutional layer, it undergoes batch normalization and activation functions and then enters the three identical residual blocks in sequence. Each residual block consists of two convolutional layers and a corresponding shortcut mechanism. The data in the residual block is linearly added to itself after passing through the two convolutional layers, thus solving the problems of degradation and vanishing or exploding gradients exposed by conventional CNNs as the number of network layers deepens. After passing through the three residual blocks, it enters the second convolutional layer. The main purpose of the second convolutional layer is to align features with the student model and at the same time enable it to enter the corresponding fully connected layer classifier. Finally, classification is performed through the max pooling layer and the fully connected layer, and the diagnosis result is output.

[0079] Specifically in implementation, the convolutional kernel width of the first convolutional layer of the teacher model is 64 so that the model can focus on the global features of the data. The convolutional kernel width of the residual block is 3, and the number of channels is 32. The convolutional kernel width of the second convolutional layer is 10, and the number of channels is 32.

[0080] Step 2.2: Pre-train the teacher model: Input the training set obtained in Step 1 into the teacher model, and take the cross-entropy between the predicted output of the teacher model and the true label of the sample as the loss function:

[0081]

[0082] In the formula, score k is the k-th predicted output of the teacher model, and target k is the true label of the k-th original data; K is the total number of predicted outputs; Use the backpropagation algorithm to update the network weights, and at the same time use the adaptive learning rate Adam algorithm and manually set the learning rate decay to adjust the learning rate, effectively improving the convergence speed, preventing falling into local minima, and alleviating the overfitting problem.

[0083] Step 2.3: Build the student model: As Figure 3 shown, the network structure of the student model is similar to that of the teacher model, but the convolution kernel width of the first convolutional layer of the student model is reduced compared to the teacher model; the convolution kernel width within the residual block remains unchanged, and the number of channels is reduced; the convolution kernel width and the number of channels of the second convolutional layer are both reduced compared to the teacher model.

[0084] Specifically, when implemented, the convolution kernel width of the first convolutional layer of the student model is 8. The convolution kernel width of the residual block is 3, and the number of channels is 8. The convolution kernel width of the second convolutional layer is 8, and the number of channels is 8.

[0085] Step 3: Load the teacher model and the best parameters saved in Step 2, and perform dimensionality increase on the intermediate layer features of the teacher model and the student model. Specifically:

[0086]

[0087]

[0088] In the formula, is the intermediate layer output of the teacher model under the -th sample, is the intermediate layer output of the student model under the -th sample, represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature of the intermediate layer output after the i-th sample is input into the teacher model; represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature of the intermediate layer output after the i-th sample is input into the student model.

[0089] Step 4: Calculate the loss function of relational knowledge distillation, specifically:

[0090] Step 4.1: Calculate the distance distillation loss between two points according to the following formula:

[0091]

[0092] Among them, represents the distance distillation loss between two points, represents the i-th training sample, represents the j-th training sample, represents the set of two-point relationships, represents the Smooth L1 loss function, represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model, represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the teacher model; represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model, represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the student model; represents and the distance between, represents and the distance between.

[0093]

[0094]

[0095] In the formula, and are distance normalization parameters, and the purpose is to solve the inconsistency in the scales of the outputs of the teacher model and the student model. Specifically:

[0096]

[0097]

[0098] In the formula, represents the set of two-point relationships.

[0099] Step 4.2: Calculate the angle distillation loss between three points according to the following formula:

[0100]

[0101] Among them, represents the angle distillation loss between three points, represents the k-th training sample, represents the set of three-point relationships, represents The three-point angle relationship represents The three-point angle relationship represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature output by the intermediate layer after the k-th sample is input into the teacher model; represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature output by the intermediate layer after the k-th sample is input into the student model.

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108] Step 4.3: Obtain the following loss function for relational knowledge distillation:

[0109]

[0110] where and represent the weights of the distance distillation loss between two points and the angle distillation loss between three points, respectively.

[0111] Step 5: Calculate the loss function for response-based knowledge distillation under relaxed matching, specifically:

[0112]

[0113]

[0114] In the formula, represents the loss function for response-based knowledge distillation under relaxed matching, represents the covariance of u and v, and represent the mean and standard deviation of u, respectively, and represent the results calculated by the deformed Softmax formula for the final responses of the teacher model and the student model, respectively. The formula is as follows:

[0115]

[0116]

[0117] where represents the response of the network model under the m-th category, represents the response of the network model under the n-th category, and C represents the total number of categories in the classification task. represents the temperature of knowledge distillation.

[0118] Step 6: Calculate the loss function of the final holistic knowledge distillation, specifically:

[0119]

[0120] In the formula, α is the weight of the loss function of relational knowledge distillation.

[0121] Step 7: Perform knowledge distillation to obtain the best student model, and save the best student model as the finally deployed model, specifically:

[0122] Step 7.1: Call the loaded teacher model and the best parameters, input the training set data into the teacher model to obtain the intermediate layer features and perform dimensionality increase, and at the same time obtain the final response.

[0123] Step 7.2: Input the training set data into the student model to obtain the intermediate layer features and perform dimensionality increase, and at the same time obtain the final response.

[0124] Step 7.3: Use the loss function of the final holistic knowledge distillation for the obtained intermediate layer dimensionality-increased features and the final response to perform knowledge distillation to complete the training of the entire student model.

[0125] Step 7.4: Use the test set to test the performance of the student model, and save the best student model as the finally deployed model.

[0126] The specific effects of the present invention can be verified on the rolling bearing dataset publicly available at the University of Parderborn in Germany. This dataset provides the vibration signals and current signals of bearings. The vibration signals of the 6203 bearing are used in this verification, and 6 fault states are selected, including inner ring damage and outer ring damage. The processing methods include electric engraving knife, drilling, and electric discharge machining. The temperature is maintained between 45 and 50 °C. Each bearing is measured 20 times, 4 seconds each time, and the sampling frequency is 64 kHz. The original vibration signals are segmented into samples with a length of 1000, and the training set and the dataset are segmented according to 4:1. The initial learning rates of the teacher network and the student network are both 0.01. The learning rate is manually set to decay by 0.99 in each round. The Adam optimization algorithm is selected for iteration, the batch number is 32, and the hyperparameters are set as = 0.3, = 25, =50, the distillation temperature T = 10. The effects of the present invention are verified by comparing the teacher model Teacher, the student model Student before knowledge distillation, the traditional response-based knowledge distillation student model KD-Student (using the loss function of response-based knowledge distillation), the relationship-based knowledge distillation student model RKD-Student (using the loss function of relationship-based knowledge distillation), and the knowledge distillation student model CKD-Student of the present invention (using the loss function of holistic knowledge distillation). The experimental results are shown in Table 1. The CKD-Student model of the present invention ensures that the student model still has a high diagnostic accuracy while reducing the number of parameters compared with the teacher model Teacher. Compared with the student model Student before knowledge distillation, the traditional response-based knowledge distillation student model KD-Student and the relationship-based knowledge distillation student model RKD-Student, the classification performance of the student model has been effectively improved.

[0127] Table 1

[0128]

[0129] The above are only the preferred embodiments of the present invention and are not intended to limit the idea of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation, characterized in that, Including: Step 1: Collect vibration signals of rolling bearings in different health states, preprocess them, and divide them into a training set and a test set. Step 2: Establish a teacher-student fault diagnosis model, pre-train the teacher model using the training set, obtain the optimal parameters of the teacher model, and save them. Step 3: Load the teacher model and the optimal parameters saved in Step 2, and perform dimensionality increase on the intermediate layer features of the teacher model and the student model. Step 4: Calculate the loss function of relational knowledge distillation: Step 5: Calculate the loss function of response-based knowledge distillation under relaxed matching: Step 6: Calculate the final overall knowledge distillation loss function: Step 7: Perform knowledge distillation to obtain the optimal student model, and save the optimal student model as the finally deployed model.

2. The intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation according to claim 1, characterized in that The said Step 1 includes: Step 1.1: Label the collected data set. The original label of the data set takes the positive label as 1 and the negative label as 0, and then perform normalization processing on the data. Step 1.2: Divide the original signals in the data set into samples with a length of 1000. The total sample size is 500, and the training set and the test set are divided according to the ratio of 4:

1.

3. The intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation according to claim 1, characterized in that, The said Step 2 includes: Step 2.1: Construct the teacher model: The teacher model is a deep residual network structure with a wide convolutional kernel in the first layer, including a first convolutional layer, three identical residual blocks, a second convolutional layer, a max pooling layer, and a fully connected layer connected in sequence. The data is input into the teacher model. After passing through the first convolutional layer, it undergoes batch normalization and activation function, and then enters the three identical residual blocks in sequence. Each residual block consists of two convolutional layers and the corresponding shortcut mechanism. The data in the residual block is linearly added to itself after passing through the two convolutional layers. After passing through the three residual blocks, it enters the second convolutional layer, and finally, classification is performed through the max pooling layer and the fully connected layer to output the diagnosis result. Step 2.2: Pre-train the teacher model: Input the training set obtained in Step 1 into the teacher model, and take the cross-entropy between the predicted output of the teacher model and the true label of the sample as the loss function. where score k is the k-th predicted output of the teacher model, and target k is the true label of the k-th original data; K is the total number of predicted outputs; the network weights are updated using the backpropagation algorithm, and at the same time, the adaptive learning rate Adam algorithm and manually set learning rate decay are used to adjust the learning rate, effectively improving the convergence speed, preventing getting stuck in local minima, and alleviating the overfitting problem; Step 2.3: Establish the student model: The network structure of the student model is similar to that of the teacher model, but the width of the convolutional kernel of the first convolutional layer of the student model is reduced compared to the teacher model; the width of the convolutional kernel in the residual block remains unchanged, and the number of channels is reduced; the width and the number of channels of the second convolutional layer of the student model are both reduced compared to the teacher model.

4. The intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation according to claim 1, wherein The specific content of Step 3 is: Wherein, is the output of the intermediate layer of the teacher model for the th sample, is the output of the intermediate layer of the student model for the th sample, represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature of the output of the intermediate layer after the i-th sample is input into the teacher model; represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature of the output of the intermediate layer after the i-th sample is input into the student model.

5. The intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation according to claim 4, wherein The specific content of Step 4 is: Step 4.1: Calculate the distance distillation loss between two points according to the following formula: Among them, represents the distance distillation loss between two points, represents the i-th training sample, represents the j-th training sample, represents the set of two-point relationships, represents the Smooth L1 loss function, represents the two-dimensional feature distribution after dimension elevation of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model, represents the two-dimensional feature distribution after dimension elevation of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the teacher model; represents the two-dimensional feature distribution after dimension elevation of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model, represents the two-dimensional feature distribution after dimension elevation of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the student model; represents and the distance between, represents and the distance between; Wherein, and are distance normalization parameters, which are used to solve the problem of inconsistent scales of the outputs of the teacher model and the student model, specifically: In the formula, represents the set of two-point relationships; Step 4.2: Calculate the angle distillation loss between three points according to the following formula: Among them, represents the angular distillation loss between three points, represents the k-th training sample, represents the set of three-point relationships, represents the three-point angular relationship of represents the three-point angular relationship of represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature output by the middle layer after the k-th sample is input into the teacher model; represents the two-dimensional feature distribution after dimensionality increase of the one-dimensional feature output by the middle layer after the k-th sample is input into the student model; Step 4.3: Obtain the following loss function of relational knowledge distillation: Among them, and represent the weights of the distance distillation loss between two points and the angle distillation loss between three points respectively.

6. The intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation according to claim 5, characterized in that The specific content of Step 5 is: In the formula, represents the loss function of response-based knowledge distillation under loose matching, represents the covariance of u and v, and represent the mean and standard deviation of u respectively, and represent the results after the deformed Softmax formula calculation of the final responses of the teacher model and the student model respectively. The formula is as follows: Among them, represents the response of the network model under the m-th category, represents the response of the network model under the n-th category, and C represents the total number of categories in the classification task. represents the temperature of knowledge distillation.

7. The intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation according to claim 6, characterized in that, The specific content of Step 6 is: In the formula, α is the weight of the loss function of relational knowledge distillation.

8. The intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation according to claim 7, characterized in that The specific content of Step 7 is: Step 7.1: Call the loaded teacher model and the optimal parameters, input the training set data into the teacher model to obtain the intermediate layer features and perform dimensionality increase, and at the same time obtain the final response. Step 7.2: Input the training set data into the student model to obtain the intermediate layer features and perform dimensionality increase, and at the same time obtain the final response. Step 7.3: Use the loss function of the final holistic knowledge distillation for the obtained intermediate-level upsampled features and the final response to perform knowledge distillation to complete the training of the entire student model; Step 7.4: Use the test set to test the performance of the student model and save the best student model as the finally deployed model.

Images