An intelligent fault diagnosis method for rolling bearings based on integrity knowledge distillation

Abstract

The application discloses a rolling bearing intelligent fault diagnosis method based on overall knowledge distillation, which comprises the following steps: step 1, collecting vibration signals of rolling bearings in different health states, preprocessing the vibration signals and dividing the vibration signals into a training set and a test set; step 2, establishing a teacher-student fault diagnosis model, pretraining a teacher model by using the training set, obtaining optimal parameters of the teacher model and saving the optimal parameters; step 3, loading the teacher model and the optimal parameters saved in step 2, and upgrading the intermediate layer features of the teacher model and a student model; step 4, calculating a loss function of relational knowledge distillation; step 5, calculating a loss function of responsive knowledge distillation under loose matching; step 6, calculating a loss function of final overall knowledge distillation; and step 7, performing knowledge distillation, obtaining an optimal student model and saving the optimal student model as a finally deployed model.

Description

Technical Field

[0001] The present invention belongs to the technical field of fault diagnosis, and relates to an intelligent rolling bearing fault diagnosis method based on holistic knowledge distillation. Background Art

[0002] Bearings are one of the most important components in industrial production and are widely used in industrial equipment. Bearing failure is one of the main causes of mechanical equipment failure. When a bearing fails, it often causes major failures in rotating machinery and causes significant economic losses. Therefore, how to quickly, efficiently, and accurately determine the operating status of a bearing has become the key to bearing fault diagnosis technology.

[0003] Numerous technologies have been developed for bearing fault diagnosis, both domestically and internationally, including those based on traditional signal processing and machine learning. While these methods have achieved some success in fault classification and diagnosis, they all have limitations. Consequently, researchers have proposed fault diagnosis techniques based on deep learning, which have gained widespread application due to their advantages, such as requiring no in-depth expertise, automatic feature extraction, and accurate classification. However, deep learning models often have high hardware requirements, making their practical deployment in industrial applications difficult.

[0004] To address this issue, researchers have proposed a knowledge distillation method. This involves transferring knowledge from a large teacher model to a smaller, more easily deployed student network, using a teacher-student model. Currently, popular distillation methods can be broadly categorized into three categories based on the content of the distillation: response-based, feature-based, and relationship-based knowledge distillation. These methods encourage the student network to learn from the knowledge acquired by the teacher network during the diagnostic process, which can, to a certain extent, improve the student model's diagnostic accuracy.

[0005] However, all of the above methods have the following problems: (1) Manual data processing is required on the raw data, such as feature extraction and time-frequency domain transformation, to adapt to the two-dimensional convolutional layer. This requires prior knowledge in the professional field and increases the diagnosis time; (2) The student model only focuses on the output of a certain aspect of the teacher model, such as the terminal softmax output layer and the intermediate feature layer. This not only makes the raw data cannot be directly used as the input of the model, but also easily loses potential effective knowledge, leaving the accuracy of the student model with much room for improvement. Summary of the Invention

[0006] In order to solve the above technical problems, the purpose of the present invention is to provide an intelligent fault diagnosis method for rolling bearings based on holistic knowledge distillation.

[0007] The present invention provides a rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation, comprising:

[0008] Step 1: Collect vibration signals of rolling bearings in different health states, preprocess them and divide them into training and test sets;

[0009] Step 2: Establish a teacher-student fault diagnosis model, use the training set to pre-train the teacher model, obtain the optimal parameters of the teacher model and save them;

[0010] Step 3: Load the teacher model and optimal parameters saved in step 2, and perform dimension upgrade on the intermediate layer features of the teacher model and the student model;

[0011] Step 4: Calculate the loss function of relational knowledge distillation:

[0012] Step 5: Calculate the loss function of responsive knowledge distillation under relaxed matching:

[0013] Step 6: Calculate the loss function of the final holistic knowledge distillation:

[0014] Step 7: Perform knowledge distillation to obtain the best student model and save the best student model as the final deployed model.

[0015] Furthermore, the step 1 includes:

[0016] Step 1.1: Label the collected dataset. The original labels of the dataset are set to 1 for positive labels and 0 for negative labels. Then, the data is normalized.

[0017] Step 1.2: Divide the original signal in the dataset into samples of length 1000, with a total sample size of 500, and divide the training set and test set into a ratio of 4:1.

[0018] Furthermore, the step 2 includes:

[0019] Step 2.1: Construct the teacher model: The teacher model is a deep residual network structure with a wide first-layer convolution kernel, which consists of a first convolutional layer, three identical residual blocks, a second convolutional layer, a maximum pooling layer, and a fully connected layer.

[0020] The data is input into the teacher model, passes through the first convolutional layer, undergoes batch normalization and activation function, and then enters three identical residual blocks in sequence. Each residual block consists of two convolutional layers and a corresponding short-circuit mechanism. The data in the residual block is linearly added to itself after passing through two convolutional layers. After passing through three residual blocks, it enters the second convolutional layer and finally is classified through the maximum pooling layer and the fully connected layer to output the diagnosis result.

[0021] Step 2.2: Pre-train the teacher model: Input the training set obtained in step 1 into the teacher model, and take the cross entropy between the teacher model's predicted output and the sample's true label as the loss function:

[0022]

[0023] In the formula, score i is the i-th predicted output of the teacher model, target i is the true label of the original data i; the back-propagation algorithm is used to update the network weights, and the adaptive learning rate Adam algorithm and manually set learning rate decay are used to adjust the learning rate, which effectively improves the convergence speed, prevents falling into the local minimum, and alleviates the overfitting problem;

[0024] Step 2.3: Establish a student model: The network structure of the student model is similar to that of the teacher model, but the width of the convolution kernel of the first convolution layer of the student model is smaller than that of the teacher model; the width of the convolution kernel in the residual block remains unchanged, and the number of channels is reduced; the width of the convolution kernel and the number of channels of the second convolution layer are both smaller than those of the teacher model.

[0025] Furthermore, the step 3 is specifically as follows:

[0026] T ri =[t ri ,t ri ,...,t ri ] T

[0027] S ri =[s ri ,s ri ,...,s ri ] T

[0028] Where, t ri is the intermediate layer output of the teacher model under the i-th sample, s ri is the intermediate layer output of the student model under the i-th sample, T ri S represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model; ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model.

[0029] Furthermore, the step 4 is specifically as follows:

[0030] Step 4.1: Calculate the distillation loss between two points according to the following formula:

[0031]

[0032] Among them, L RKD-D Represents the distance distillation loss between two points, x i represents the i-th training sample, xj represents the jth training sample, χ 2 Represents a set of relations between two points, l δ () represents the SmoothL1 loss function, T ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model, T rj S represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the teacher model; ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model, and S rj It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the student model; D (T ri ,T rj ) indicates T ri and T rj The distance, Ψ D (S ri ,S rj ) indicates S ri and S rj distance;

[0033]

[0034] Where μ1 and μ2 are distance normalization parameters, which are used to solve the scale inconsistency between the outputs of the teacher model and the student model. Specifically:

[0035]

[0036] Where, χ 2 Represents a set of relationships between two points;

[0037] Step 4.2: Calculate the angular distillation loss between the three points according to the following formula:

[0038]

[0039] Among them, L RKD-A Represents the angle distillation loss between the three points, x k represents the kth training sample, χ 3 represents a set of three-point relationships, Ψ A (T ri ,T rj ,T rk ) indicates T ri ,T rj ,T rk The three-point angle relationship, Ψ A (S ri ,S rj ,Srk ) indicates S ri ,S rj ,S rk The three-point angle relationship, T rk S represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the k-th sample is input into the teacher model; rk It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the k-th sample is input into the student model;

[0040]

[0041] Step 4.3: Obtain the following loss function for relational knowledge distillation:

[0042] L RKD =ω1·L RKD-D +ω2·L RKD-A

[0043] Among them, ω1 and ω2 represent the weights of the distance distillation loss between two points and the angle distillation loss between three points, respectively.

[0044] Furthermore, the step 5 is specifically as follows:

[0045] L Relax-match =1-ρ p (u,v)

[0046]

[0047] Where, L Relax-match Represents the loss function of responsive knowledge distillation under relaxed matching, Cov(u,v) represents the covariance of u and v, and Std(u) represent the mean and standard deviation of u respectively. m and v m They represent the final responses of the teacher model and the student model after being calculated using the deformed Softmax formula, which is as follows:

[0048]

[0049]

[0050] Among them, z m represents the response of the network model under the mth category, z n represents the response of the network model under the nth category, C represents the total number of categories of the classification task, and T represents the temperature of knowledge distillation.

[0051] Furthermore, the step 6 is specifically as follows:

[0052] L all=α·(ω1·L RKD-D +ω2·L RKD-A )+(1-α)·L Relax-match

[0053] Where α is the weight of the loss function of relational knowledge distillation.

[0054] Furthermore, the step 7 is specifically as follows:

[0055] Step 7.1: Call the loaded teacher model and optimal parameters, input the training set data into the teacher model to obtain the intermediate layer features and upgrade the dimension, and obtain the final response;

[0056] Step 7.2: Input the training set data into the student model to obtain the intermediate layer features and upgrade the dimension, and obtain the final response;

[0057] Step 7.3: Use the final holistic knowledge distillation loss function to perform knowledge distillation on the obtained intermediate layer dimension-raising features and the final response to complete the training of the entire student model;

[0058] Step 7.4: Test the performance of the student model using the test set and save the best student model as the final deployed model.

[0059] The rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation of the present invention has the following beneficial effects:

[0060] (1) In existing bearing fault diagnosis methods based on knowledge distillation, the student model only learns the output of the teacher model's terminal softmax; or only considers the relationship between intermediate layer features, and forcibly uses cross entropy for the student model's prediction and actual target. In fact, there is an adversarial relationship between the two. Both of these solutions limit the student model's learning ability, resulting in low fault diagnosis accuracy. To address the above problems, the present invention proposes a new method that combines relational knowledge distillation with responsive knowledge distillation improved by relaxed matching. This method reduces the adversarial relationship between relational distilled knowledge and the student model's original knowledge, thereby achieving the purpose of holistic knowledge distillation and improving the model's fault diagnosis ability.

[0061] (2) In existing bearing fault diagnosis methods based on knowledge distillation, in order to improve the diagnostic ability of the student model, the original data is generally converted into a time-frequency graph as the model input. Although this objectively improves the diagnostic ability of the model, it causes the convolution layers of the student model to be two-dimensional, resulting in the inability to further compress the parameters of the student model. To address the above problems, the present invention uses a student network with a one-dimensional convolutional network structure and simultaneously increases the dimensionality of the intermediate layer features during the distillation process to reduce the performance loss caused by using a one-dimensional convolutional network, thereby further compressing the parameters of the model when applied. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation according to the present invention;

[0063] Figure 2 is a network structure diagram of the teacher model in an embodiment of the present invention;

[0064] Figure 3 4 is a network structure diagram of the student model in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] like Figure 1 As shown, the present invention provides an intelligent rolling bearing fault diagnosis method based on holistic knowledge distillation, comprising:

[0066] Step 1: Collect vibration signals of rolling bearings in different health states. These vibration signals represent the axial operation characteristics of the bearings. Preprocess them and divide them into training sets and test sets. Step 1 includes:

[0067] Step 1.1: Label the collected dataset. The original label of the dataset is 1 for positive labels and 0 for negative labels. Then the data is normalized.

[0068] Step 1.2: Divide the original signal in the dataset into samples of length 1000, with a total sample size of 500, and divide the training set and test set into a ratio of 4:1.

[0069] Step 2: Establish a teacher-student fault diagnosis model, use the training set to pre-train the teacher model, obtain the optimal parameters of the teacher model and save them. Step 2 includes:

[0070] Step 2.1: Construct the teacher model: Figure 2 As shown in the figure, the teacher model is a deep residual network structure with a wide convolution kernel in the first layer, which includes the first convolution layer, three identical residual blocks, the second convolution layer, the maximum pooling layer and the fully connected layer.

[0071] Data is input into the teacher model, passes through the first convolutional layer, undergoes batch normalization and an activation function, and then enters three identical residual blocks in sequence. Each residual block consists of two convolutional layers and a corresponding short-circuit mechanism. The data in the residual block is linearly added to itself after passing through the two convolutional layers, thus solving the degradation and gradient vanishing or exploding issues that conventional CNNs suffer from as the number of network layers increases. After passing through the three residual blocks, the data enters the second convolutional layer. The main purpose of the second convolutional layer is to align features with the student model and prepare them for entry into the corresponding fully connected layer classifier. Finally, classification is performed through the max pooling layer and the fully connected layer, and the diagnostic result is output.

[0072] In practice, the convolution kernel width of the first convolutional layer of the teacher model is 64 to allow the model to focus on the global characteristics of the data. The convolution kernel width of the residual block is 3 and the number of channels is 32. The convolution kernel width of the second convolutional layer is 10 and the number of channels is 32.

[0073] Step 2.2: Pre-train the teacher model: Input the training set obtained in step 1 into the teacher model, and take the cross entropy between the teacher model's predicted output and the sample's true label as the loss function:

[0074]

[0075] In the formula, score i is the i-th predicted output of the teacher model, target i is the true label of the original data i; the back-propagation algorithm is used to update the network weights, and the adaptive learning rate Adam algorithm and manually set learning rate decay are used to adjust the learning rate, which effectively improves the convergence speed, prevents falling into the local minimum, and alleviates the overfitting problem.

[0076] Step 2.3: Build the student model: Figure 3 As shown in the figure, the network structure of the student model is similar to that of the teacher model, but the width of the convolution kernel of the first convolution layer of the student model is smaller than that of the teacher model; the width of the convolution kernel in the residual block remains unchanged, and the number of channels is reduced; the width and number of channels of the convolution kernel of the second convolution layer are both reduced compared with the teacher model.

[0077] In specific implementation, the convolution kernel width of the first convolution layer of the student model is 8. The convolution kernel width of the residual block is 3 and the number of channels is 8. The convolution kernel width of the second convolution layer is also 8 and the number of channels is 8.

[0078] Step 3: Load the teacher model and optimal parameters saved in step 2, and perform dimension upgrade on the intermediate layer features of the teacher model and the student model. Specifically:

[0079] T ri =[t ri ,t ri ,...,t ri ] T

[0080] S ri =[s ri ,s ri ,...,s ri ] T

[0081] Where, t ri is the intermediate layer output of the teacher model under the i-th sample, s ri is the intermediate layer output of the student model under the i-th sample, Tri S represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model; ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model.

[0082] Step 4: Calculate the loss function of relational knowledge distillation, specifically:

[0083] Step 4.1: Calculate the distillation loss between two points according to the following formula:

[0084]

[0085] Among them, L RKD-D Represents the distance distillation loss between two points, x i represents the i-th training sample, x j represents the jth training sample, χ 2 Represents a set of relations between two points, l δ () represents the SmoothL1 loss function, T ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model, T rj S represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the teacher model; ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model, and S rj It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the student model; D (T ri ,T rj ) indicates T ri and T rj The distance, Ψ D (S ri ,S rj ) indicates S ri and S rj distance.

[0086]

[0087] Where μ1 and μ2 are distance normalization parameters, which are used to solve the scale inconsistency between the outputs of the teacher model and the student model. Specifically:

[0088]

[0089] Where, χ 2 Represents a set of relationships between two points.

[0090] Step 4.2: Calculate the angular distillation loss between the three points according to the following formula:

[0091]

[0092] Among them, L RKD-A Represents the angle distillation loss between the three points, x k represents the kth training sample, χ 3 represents a set of three-point relationships, Ψ A (T ri ,T rj ,T rk ) indicates T ri ,T rj ,T rk The three-point angle relationship, Ψ A (S ri ,S rj ,S rk ) indicates S ri ,S rj ,S rk The three-point angle relationship, T rk S represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the k-th sample is input into the teacher model; rk It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the k-th sample is input into the student model.

[0093]

[0094] Step 4.3: Obtain the following loss function for relational knowledge distillation:

[0095] L RKD =ω1·L RKD-D +ω2·L RKD-A

[0096] Among them, ω1 and ω2 represent the weights of the distance distillation loss between two points and the angle distillation loss between three points, respectively.

[0097] Step 5: Calculate the loss function of responsive knowledge distillation under relaxed matching, specifically:

[0098] L Relax-match =1-ρ p (u,v)

[0099]

[0100] Where, L Relax-match Represents the loss function of responsive knowledge distillation under relaxed matching, Cov(u,v) represents the covariance of u and v, and Std(u) represent the mean and standard deviation of u respectively.m and v m They represent the final responses of the teacher model and the student model after being calculated using the deformed Softmax formula, which is as follows:

[0101]

[0102] Among them, z m represents the response of the network model under the mth category, z n represents the response of the network model under the nth category, C represents the total number of categories of the classification task, and T represents the temperature of knowledge distillation.

[0103] Step 6: Calculate the final loss function of holistic knowledge distillation, specifically:

[0104] L all =α·(ω1·L RKD-D +ω2·L RKD-A )+(1-α)·L Relax-match

[0105] Where α is the weight of the loss function of relational knowledge distillation.

[0106] Step 7: Perform knowledge distillation to obtain the best student model and save the best student model as the final deployed model. Specifically:

[0107] Step 7.1: Call the loaded teacher model and optimal parameters, input the training set data into the teacher model to obtain the intermediate layer features and upgrade the dimension, and obtain the final response at the same time.

[0108] Step 7.2: Input the training set data into the student model to obtain the intermediate layer features and upgrade the dimension, and obtain the final response at the same time.

[0109] Step 7.3: Use the final holistic knowledge distillation loss function to perform knowledge distillation on the obtained intermediate layer dimensionality-enhanced features and the final response to complete the training of the entire student model.

[0110] Step 7.4: Test the performance of the student model using the test set and save the best student model as the final deployed model.

[0111] The specific effects of the present invention can be verified on the rolling bearing dataset published by the University of Parderborn in Germany. This dataset provides the vibration signal and current signal of the bearing. This verification uses the vibration signal of the 6203 bearing and selects 6 fault states, including inner ring damage and outer ring damage. The processing methods include electric engraving, drilling, and electric spark. The temperature is maintained between 45 and 50°C. Each bearing is measured 20 times, each time for 4 seconds, with a sampling frequency of 64kHz. The original vibration signal is divided into samples of length 1000, and the training set and data set are divided into 4:1. The initial learning rate of the teacher network and the student network is 0.01. The learning rate is manually set to decay by 0.99 each round. The Adam optimization algorithm is selected for iteration, the number of batches is 32, the hyperparameters are set to α=0.3, ω1=25, ω2=50, and the distillation temperature T=10. The effect of the present invention is verified by comparing the teacher model Teacher, the student model Student before knowledge distillation, the traditional response-based knowledge distillation student model KD-Student (using the loss function of response-type knowledge distillation), the relationship-based knowledge distillation student model RKD-Student (using the loss function of relationship-type knowledge distillation) and the knowledge distillation student model CKD-Student adopted by the present invention (using the loss function of holistic knowledge distillation). The experimental results are shown in Table 1. Compared with the teacher model Teacher, the CKD-Student model of the present invention reduces the number of parameters while ensuring that the student model still has a high diagnostic accuracy. Compared with the student model Student before knowledge distillation, the traditional response-based knowledge distillation student model KD-Student and the relationship-based knowledge distillation student model RKD-Student, the classification performance of the student model is effectively improved.

[0112] Table 1

[0113]

[0114]

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the concept of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation, characterized by: include: Step 1: Collect vibration signals of rolling bearings in different health states, preprocess them and divide them into training and test sets; Step 2: Establish a teacher-student fault diagnosis model, use the training set to pre-train the teacher model, obtain the optimal parameters of the teacher model and save them; Step 3: Load the teacher model and optimal parameters saved in step 2, and perform dimension upgrade on the intermediate layer features of the teacher model and the student model; Step 4: Calculate the loss function of relational knowledge distillation: Step 5: Calculate the loss function of responsive knowledge distillation under relaxed matching: Step 6: Calculate the loss function of the final holistic knowledge distillation: Step 7: Perform knowledge distillation to obtain the best student model and save the best student model as the final deployed model; The step 5 is specifically as follows: L Relax-match =1-r p (u,v) Where, L Relax-match Represents the loss function of responsive knowledge distillation under relaxed matching, Cov(u,v) represents the covariance of u and v, and Std(u) represent the mean and standard deviation of u respectively. m and v m They represent the final responses of the teacher model and the student model after being calculated using the deformed Softmax formula, which is as follows: Among them, z m represents the response of the network model under the mth category, z n represents the response of the network model under the nth category, C represents the total number of categories of the classification task, and T represents the temperature of knowledge distillation.

2. The rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation according to claim 1 is characterized in that: The step 1 comprises: Step 1.1: Label the collected dataset. The original labels of the dataset are set to 1 for positive labels and 0 for negative labels. Then, the data is normalized. Step 1.2: Divide the original signal in the dataset into samples of length 1000, with a total sample size of 500, and divide the training set and test set into a ratio of 4:

1.

3. The rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation according to claim 1 is characterized in that: The step 2 includes: Step 2.1: Construct the teacher model: The teacher model is a deep residual network structure with a wide first-layer convolution kernel, which consists of a first convolutional layer, three identical residual blocks, a second convolutional layer, a maximum pooling layer, and a fully connected layer. The data is input into the teacher model, passes through the first convolutional layer, undergoes batch normalization and activation function, and then enters three identical residual blocks in sequence. Each residual block consists of two convolutional layers and a corresponding short-circuit mechanism. The data in the residual block is linearly added to itself after passing through two convolutional layers. After passing through three residual blocks, it enters the second convolutional layer and finally is classified through the maximum pooling layer and the fully connected layer to output the diagnosis result. Step 2.2: Pre-train the teacher model: Input the training set obtained in step 1 into the teacher model, and take the cross entropy between the teacher model's predicted output and the sample's true label as the loss function: In the formula, score k is the k-th prediction output of the teacher model, target k is the true label of the kth original data; K is the total number of predicted outputs; the network weights are updated using the backpropagation algorithm, and the learning rate is adjusted using the adaptive learning rate Adam algorithm and manually set learning rate decay; Step 2.3: Establish a student model: The network structure of the student model is similar to that of the teacher model, but the width of the convolution kernel of the first convolution layer of the student model is smaller than that of the teacher model; the width of the convolution kernel in the residual block remains unchanged, and the number of channels is reduced; the width of the convolution kernel and the number of channels of the second convolution layer are both smaller than those of the teacher model.

4. The rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation according to claim 1 is characterized in that: The step 3 is specifically as follows: T ri =[t ri ,t ri ,...,t ri ] T S ri =[s ri ,s ri ,...,s ri ] T Where, t ri is the intermediate layer output of the teacher model under the i-th sample, s ri is the intermediate layer output of the student model under the i-th sample, T ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model; S ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model.

5. The rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation according to claim 4 is characterized in that: The step 4 is specifically as follows: Step 4.1: Calculate the distillation loss between two points according to the following formula: Among them, L RKD-D Represents the distance distillation loss between two points, x i represents the i-th training sample, x j represents the jth training sample, χ 2 Represents a set of relations between two points, l δ () represents the SmoothL1 loss function, T ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the teacher model, T rj S represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the teacher model; ri It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the i-th sample is input into the student model, and S rj It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the j-th sample is input into the student model; D (T ri ,T rj ) indicates T ri and T rj The distance, Ψ D (S ri ,S rj ) indicates S ri and S rj distance; Where μ1 and μ2 are distance normalization parameters, specifically: Where, χ 2 Represents a set of relationships between two points; Step 4.2: Calculate the angular distillation loss between the three points according to the following formula: Among them, L RKD-A Represents the angle distillation loss between the three points, x k represents the kth training sample, χ 3 represents a set of three-point relationships, Ψ A (T ri ,T rj ,T rk ) indicates T ri ,T rj ,T rk The three-point angle relationship, Ψ A (S ri ,S rj ,S rk ) indicates S ri ,S rj ,S rk The three-point angle relationship, T rk S represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the k-th sample is input into the teacher model; rk It represents the two-dimensional feature distribution of the one-dimensional feature output by the intermediate layer after the k-th sample is input into the student model; Step 4.3: Obtain the following loss function for relational knowledge distillation: L RKD =ω1·L RKD-D +ω2·L RKD-A Among them, ω1 and ω2 represent the weights of the distance distillation loss between two points and the angle distillation loss between three points, respectively.

6. The rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation according to claim 5 is characterized in that: The step 6 is specifically as follows: L all =α·(ω1·L RKD-D +ω2·L RKD-A )+(1-α)·L Relax-match Where α is the weight of the loss function of relational knowledge distillation.

7. The rolling bearing intelligent fault diagnosis method based on holistic knowledge distillation according to claim 6 is characterized in that: The step 7 is specifically as follows: Step 7.1: Call the loaded teacher model and optimal parameters, input the training set data into the teacher model to obtain the intermediate layer features and upgrade the dimension, and obtain the final response; Step 7.2: Input the training set data into the student model to obtain the intermediate layer features and upgrade the dimension, and obtain the final response; Step 7.3: Use the final holistic knowledge distillation loss function to perform knowledge distillation on the obtained intermediate layer dimension-raising features and the final response to complete the training of the entire student model; Step 7.4: Test the performance of the student model using the test set and save the best student model as the final deployed model.

Images