A lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation

Abstract

This invention discloses a lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation. Employing knowledge distillation and adversarial learning, the student model learns valuable information by studying soft labels provided by the teacher network. With the help of a well-learned teacher network, even a small student model can achieve diagnostic accuracy similar to deep networks. Simultaneously, combining uniform quantization with the distillation process significantly reduces the size of the student model. The method is computationally simple and effective, making lightweight networks more suitable for practical devices with limited computing and storage resources. This enables real-time bearing fault diagnosis on smaller devices such as mobile phones and embedded devices.

Description

Technical Field

[0001] This invention belongs to the field of fault diagnosis technology, and specifically relates to a lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation. Background Technology

[0002] Currently, for fault diagnosis of rolling bearings, researchers often use semi-supervised prototype enhancement networks for analysis and processing. For example, they use dual-pooling attention residual networks for feature extraction modules and algorithmic design to update prototypes with pseudo-labels. By accurately adjusting the prototype position of each class of finite labeled samples through unlabeled samples, the prototype is enhanced, thereby improving the network model's ability to distinguish fault categories.

[0003] This approach offers high accuracy in diagnosing rolling bearing faults, but its overly complex model structure and reliance on high-performance servers and expensive computing equipment significantly hinder its application in smaller devices such as mobile phones and embedded systems. Therefore, this invention aims to create lightweight, more efficient fault diagnosis models while maintaining performance levels for real-time applications.

[0004] Knowledge distillation can transfer knowledge from a larger deep network (teacher network) to a smaller network (student model). During training, the output of the teacher model is used as a soft label, and this label is provided to the student model for learning, thereby achieving the goal of transferring the knowledge learned by the teacher model to the student model.

[0005] Adversarial distillation is an improvement on knowledge distillation. It introduces an adversarial loss function, causing the student model not only to attempt to fit the soft labels of the teacher model but also to adversarially generate hard labels, making it difficult for the teacher model to distinguish the student model's output. This adversarial training helps the student model better capture the knowledge of the teacher model and improves performance. Adversarial distillation uses a GAN architecture: the teacher network uses real samples as the training set; the generator is the student model used to generate the output of the softmax layer; and the discriminator is used to determine the probability that the input sample is a real sample.

[0006] Parameter quantization refers to replacing high-precision floating-point values ​​with low-precision integer values, thereby significantly reducing network storage space and improving computation speed. This invention innovatively combines parameter quantization with knowledge distillation, supplemented by adversarial learning to enhance the model's generalization ability, minimizing precision loss during the sparsity process. This is of great significance for better utilizing the hardware of computing platforms. Summary of the Invention

[0007] The purpose of this invention is to address the high computational cost of deep networks with a large number of parameters in predictive maintenance scenarios, and to provide a lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation, comprising the following steps:

[0008] Step 1: Design a teacher model with multiple layers, a large number of convolutional kernels and neurons.

[0009] Step 2: Construct a student model.

[0010] The initial ResNet18 model is simplified based on the proposed residual blocks to obtain the lightweight network model Small ResNet, which is then used to build student models.

[0011] Step 3: Perform knowledge distillation.

[0012] A teacher model is used as a feature extractor and trained on different connection class classifiers or domain classifiers based on the dataset. After training, the teacher model uses the softmax layout in the network structure as soft labels to guide the student model.

[0013] Step 4: Use adversarial learning to help the student model achieve a higher accuracy.

[0014] Step 5: Perform uniform quantization on the student model using the improved quantization method.

[0015] Step 6: Perform quantitative distillation by combining anti-distillation and homogeneous quantization.

[0016] Step 7: Real-time diagnosis of lightweight bearing faults based on the trained student model.

[0017] As a preferred approach, the teacher model uses the ResNet18 residual network as its backbone. Depthily separable convolutional (DSC) and a focus module (CBAM) are introduced into the teacher model to improve the performance of ResNet18. A DSC layer is added after the first two blocks of ResNet18, and a CBAM layer is added after the third block. DSC can be used to reduce the computational complexity and parameters of the model.

[0018] CBAM consists of a channel attention module and a spatial attention module, combining the channel attention mechanism CBM and the spatial attention mechanism SBM. By learning the weights assigned to different channels and regions in an image, the network can better focus on task-relevant channels and regions. All steps can be simply represented as:

[0019]

[0020]

[0021] Where F is the input feature map, Q C (·) and Q S (·) represent the computational mechanisms for channel attention and spatial attention, respectively. This indicates element-wise multiplication, and F″ is the final output.

[0022] As a preferred option, step 2 is performed as follows:

[0023] Based on the residual structure of ResNet18, a "residual block" structure is proposed, consisting of a convolutional layer, a batch normalization (BN) layer, a ReLU function, another convolutional layer, and another BN layer from top to bottom.

[0024] The initial ResNet18 model is simplified based on the proposed residual blocks to obtain the lightweight network model Small ResNet, which is then used to build a student model. The entire student model consists of convolutional layers, two normalization layers, another convolutional layer, another normalization layer, another convolutional layer, and three residual blocks. The first convolutional layer uses a 1×9 kernel to extract initial features from the data. Then, two consecutive normalization layers adjust and stabilize the data distribution. The second convolutional layer uses a 1×5 kernel to further extract more complex features. The third normalization layer again adjusts the data distribution, providing stable input data for the next layer. The third convolutional layer uses a 1×3 kernel for final feature extraction. Then, through the operation of the three residual blocks, the data size is reduced to one-third of the original, and the number of channels is doubled. At the end of the network, a combination of global average pooling layers and fully connected layers is used, requiring no parameter computation. The fully connected layers map the features learned by the network to class labels, ultimately completing the pre-training of the student model.

[0025] As a preferred option, step 3 is performed as follows:

[0026] The teacher model is used as the feature extractor G. f (·;θ f ), and classify the different connection classes G in the dataset. y (·;θ y or domain classifier G d (·;θ d ), whose parameters are θ f θ y and θ d .

[0027] For the source dataset {x i}, its class tag is {y i}(i=1,...,n s ), where n s This refers to the amount of source data. The classifier is trained to minimize the source domain loss. That is, the negative log probability of the correct label within the source domain:

[0028]

[0029] The target dataset uses an unlabeled dataset {x} that is different from the source dataset. j}, its domain label is {y j}(j=1,...,n t ), where n t This refers to the quantity of the target data. If x... j If it comes from the source domain, then its domain label y j Set to 0, otherwise y j The value is 1. The domain classifier is trained to maximize the loss on both the source and target domains.

[0030]

[0031] The teacher model is first connected to a class classifier and trained using class labels. The goal is to minimize the source domain loss. Then, the teacher model is connected to the domain classifier and trained using domain labels to obtain the loss that maximizes both the source and target domains. Finally, the teacher model uses the softmax layout in the network structure as soft labels to guide the student model. These soft labels are the probability distributions of the class predictions made by the trained teacher model for each sample, reflecting the model's confidence in the class to which the sample belongs.

[0032] For the student model, it's necessary to fit not only the true labels of the dataset but also the output distribution of the teacher model. Therefore, the distillation loss function... Defined as hard loss function With soft loss function The sum. The teacher model is represented as f. T The student model is represented as f S In conclusion, The definition is as follows:

[0033]

[0034] Where λ∈[0,1], is a hyperparameter used to balance soft loss and hard loss. Soft loss The difference between the outputs of the student model and the teacher model is represented by the following definition, expressed using cross-entropy:

[0035]

[0036] For hard loss It represents the difference between the student model's output and the class label, expressed using cross-entropy:

[0037]

[0038] As a preferred option, step 4 is performed as follows:

[0039] A discriminator network D is used to determine whether the input comes from the student model or the teacher model. The loss function of the discriminator is:

[0040]

[0041] Where, p T (y i ) and p S (y i ) represent the probability distributions output by the teacher model and the student model, respectively. The loss function for the generator is as follows:

[0042]

[0043] The generator, or student model, aims to continuously optimize its output distribution to be as similar as possible to the teacher model's output distribution, to the point that the discriminator cannot distinguish between them. A smaller loss function indicates a better learning performance of the student model.

[0044] Finally, the overall loss function for counterdistillation can be expressed as:

[0045]

[0046] Where ∝∈[0,1] is a hyperparameter used to balance distillation loss and counteract loss.

[0047] As a preferred option, step 5 is performed as follows:

[0048] Uniform quantization first determines the quantization level *s* based on requirements, where *s* represents the number of integer bits after quantization. *s*-level uniform quantization selects *s* equally spaced points within the interval 0 to 1 as quantization points, mapping the weight values ​​of the weight vector to the nearest quantization point, thus reducing storage and computational overhead. The weight vector, in a fully connected neural network, is a matrix representing the connection weights between input and output neurons.

[0049] For the student model, the initial network weight parameters are v. i The quantized weight parameters v are obtained by sequentially applying a linear scaling function and a quantization function. s The linear scaling function normalizes a weight vector with values ​​from any range into a vector with values ​​in the range [0,1]. It is calculated as follows:

[0050]

[0051] The quantization function is expressed as follows:

[0052]

[0053] In the formula, s is the quantization series. Let ξ be the floor function. i ξ is the rounding function. i ~Bernoulli(k i ), where k i For the original value v i The normalized distance between the coordinates and the random rounding point. In the improved quantization method, each scaled vector coordinate v i Instead of assigning the nearest quantization point, the quantization point is rounded based on the probability of the unbiased estimator of vi with the smallest variance, to prevent the network from selecting a local optimum and avoid increasing quantization error.

[0054] As a preferred option, step 6 is performed as follows:

[0055] The student model constructed in step 2 stores the weights in 32-bit floating-point numbers, which is called the full-precision student model, and uniform quantization is performed on it.

[0056] The quantization process requires continuous iteration to balance efficiency gains and accuracy losses. Let v represent the initial network weights of the full-precision student model, then, according to the quantization function in step 5, calculate...

[0057] The new network weights can then be calculated as follows: in The overall loss from adversarial distillation in step 4 is represented by η, which is a hyperparameter. The new network weights v are then... new The gradient is used as the weight parameter in the student model for the next iteration, and this process is repeated until the gradient reaches zero, at which point the iteration ends. The distillation process is then complete, yielding the final output f of the student model. S That is, the predicted value of the student model.

[0058] As a preferred option, step 7 is performed as follows:

[0059] Raw vibration signal data is collected by an accelerometer mounted on the bearing base to obtain data on the entire life cycle of the bearing, and multiple vibration signal data segments are obtained by using a sliding window segmentation method.

[0060] By inputting the acquired vibration signal data into the student model, real-time fault diagnosis can be performed, and the output is the student model's own prediction output f. S .

[0061] The beneficial effects of this invention are as follows:

[0062] This invention proposes a lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation. Employing knowledge distillation and adversarial learning, the student model learns valuable information from soft labels provided by the teacher network. With the help of a well-learned teacher network, even a small student model can achieve diagnostic accuracy similar to deep networks. Simultaneously, combining uniform quantization with the distillation process significantly reduces the size of the student model. Various experiments and comparisons demonstrate the effectiveness of the proposed lightweight residual network. The method is computationally simple and yields effective results, making the lightweight network more suitable for practical devices with limited computing and storage resources. Furthermore, this invention has good reusability in similar application scenarios, demonstrating strong practical value. This invention provides a lightweight and more efficient fault diagnosis model that does not rely on high-performance servers and expensive computing equipment, enabling real-time bearing fault diagnosis on smaller devices such as mobile phones and embedded devices. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0064] Figure 1 A logical schematic diagram of a lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation provided by the present invention;

[0065] Figure 2 The network structure diagram of the main teacher model for knowledge distillation provided in this invention;

[0066] Figure 3 The network structure diagram of the main student model for knowledge distillation provided in this invention;

[0067] Figure 4 The main flowchart of uniform quantization provided for this invention;

[0068] Figure 5 The experimental results of the method provided by the present invention in the migration task between different bearing groups of the same dataset PU in the embodiment are shown in the figure.

[0069] Figure 6 Radar graph showing experimental results of the method provided by the present invention in a transfer task between different datasets PU and CWRU in an embodiment;

[0070] Figure 7 The confusion matrix diagram shows the experimental results of the method provided by the present invention in the migration task between different datasets PU and CWRU in an embodiment. Detailed Implementation

[0071] To enable those skilled in the art to better understand the technical content of this invention, the technical solution is described in detail below with reference to the accompanying drawings and specific embodiments. This invention discloses a lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation, which compresses the size of the student model as much as possible while maintaining a diagnostic accuracy similar to that of deep networks. The method of this invention is computationally simple, yields effective results, and demonstrates feasibility and superiority in practical bearing fault diagnosis and cross-domain scenarios. Furthermore, this invention has good reusability in similar application scenarios, exhibiting strong practical value.

[0072] like Figure 1 As shown, a lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation is implemented in an example, including the following steps:

[0073] Step 1: Design a teacher model with multiple layers, a large number of convolutional kernels, and neurons. The ResNet18 residual network is used as the backbone network. ResNet18 includes one convolutional layer, a max-pooling layer, four convolutional blocks, an average pooling layer, a fully connected layer, and a softmax output. See the attached diagram for the network structure. Figure 2 The input layer has 2048 nodes, and the output layer has 8 nodes.

[0074] like Figure 2 As shown, the teacher model incorporates Depthwise Separable Convolutional (DSC) and a Convolutional Attention Module (CBAM) to improve the performance of ResNet18. DSC layers are added after the first two blocks of ResNet18, and CBAM is added after the third block. DSC can reduce the computational complexity and parameters of the model. In a standard convolutional layer, each convolution kernel performs a convolution operation on each channel of the input data. Then, the results corresponding to each convolution kernel are summed as the output.

[0075] CBAM consists of a channel attention module and a spatial attention module, combining the channel attention mechanism CBM and the spatial attention mechanism SBM. By learning the weights assigned to different channels and regions in an image, the network can better focus on task-relevant channels and regions. All steps can be simply represented as:

[0076]

[0077]

[0078] Where F is the input feature map, Q C (·) and Q S (·) represent the computational mechanisms for channel attention and spatial attention, respectively. This indicates element-wise multiplication, and F″ is the final output.

[0079] Step 2: Construct a student model.

[0080] This invention designs a "residual block" structure, consisting of a convolutional layer, a batch normalization (BN) layer, a ReLU function, another convolutional layer, and another BN layer, from top to bottom. Based on the residual block, the initial ResNet18 model is simplified to obtain the lightweight network model SmallResNet, which is then used to construct a student model. Figure 3 As shown, the entire student model consists of a convolutional layer, two normalization layers, another convolutional layer, another normalization layer, another convolutional layer, and three residual blocks. The first convolutional layer uses a 1×9 kernel to extract initial features from the data. Then, two consecutive normalization layers adjust and stabilize the data distribution. The second convolutional layer uses a 1×5 kernel to further extract more complex features. The third normalization layer again adjusts the data distribution, providing stable input data for the next layer. The third convolutional layer uses a 1×3 kernel for final feature extraction. Then, through the operation of the three residual blocks, the data size is reduced to one-third of the original, and the number of channels is doubled. At the end of the network, a combination of a global average pooling layer and a fully connected layer is used, requiring no parameter calculation. The fully connected layer maps the features learned by the network to class labels, ultimately completing the pre-training of the student model.

[0081] Step 3: Use the teacher model as the feature extractor G f (·;θ f ), and classify the different connection classes G in the dataset. y (·;θ y or domain classifier G d (·;θ d ), whose parameters are θ f θ y and θ d The purpose of a classifier is to classify data samples into categories, that is, to determine which specific category a sample belongs to; while a domain classifier determines whether the data belongs to the source domain or the target domain.

[0082] For the source dataset {x i}, its class tag is {y i}(i=1,...,n s ), where n s This refers to the amount of source data. The classifier is trained to minimize the source domain loss. That is, the negative log probability of the correct label within the source domain:

[0083]

[0084] The target dataset uses an unlabeled dataset {x} that is different from the source dataset. j}, its domain label is {yj}(j=1,...,n t ), where n t This refers to the quantity of the target data. If x... j If it comes from the source domain, then its domain label y j Set to 0, otherwise y j The value is 1. The domain classifier is trained to maximize the loss on both the source and target domains.

[0085]

[0086] The teacher model is first connected to a class classifier and trained using class labels. The goal is to minimize the source domain loss. Then, the teacher model is connected to the domain classifier and trained using domain labels to obtain the loss that maximizes both the source and target domains. Finally, the teacher model uses the softmax layout in the network structure as soft labels to guide the student model. These soft labels are the probability distributions of the class predictions made by the trained teacher model for each sample, reflecting the model's confidence in the class to which the sample belongs.

[0087] For the student model, it's necessary to fit not only the true labels of the dataset but also the output distribution of the teacher model. Therefore, the distillation loss function... Defined as hard loss function With soft loss function The sum. The teacher model is represented as f. T The student model is represented as f S In conclusion, The definition is as follows:

[0088]

[0089] Where λ∈[0,1], is a hyperparameter used to balance soft loss and hard loss. Soft loss The difference between the outputs of the student model and the teacher model is represented by the following definition, expressed using cross-entropy:

[0090]

[0091] For hard loss It represents the difference between the student model's output and the class label, expressed using cross-entropy:

[0092]

[0093] Step 4: Use adversarial learning to help the student model achieve a higher accuracy.

[0094] A discriminator network D is used to determine whether the input comes from the student model or the teacher model. The loss function of the discriminator is:

[0095]

[0096] Where, p T (y i ) and p S (y i ) represent the probability distributions output by the teacher model and the student model, respectively. The loss function for the generator is as follows:

[0097]

[0098] The generator, or student model, aims to continuously optimize its output distribution to be as similar as possible to the teacher model's output distribution, to the point that the discriminator cannot distinguish between them. A smaller loss function indicates a better learning performance of the student model.

[0099] Finally, the overall loss function for counterdistillation can be expressed as:

[0100]

[0101] Where ∝∈[0,1] is a hyperparameter used to balance distillation loss and counteract loss.

[0102] Step 5, as attached Figure 4 As shown, the student model is uniformly quantized using the improved quantization method. The specific operation is as follows:

[0103] Quantization converts high-precision floating-point numbers into low-precision integers. First, the quantization level *s* is determined based on requirements; *s* represents the number of bits in the quantized integer. *s*-level uniform quantization selects *s* equally spaced points within the interval 0 to 1 as quantization points, mapping the weight values ​​of the weight vector to the nearest quantization point, thus reducing storage and computational overhead. The weight vector, in a fully connected neural network, is a matrix representing the connection weights between input and output neurons.

[0104] For the student model, the initial network weight parameters are v. i The quantized weight parameters v are obtained by sequentially applying a linear scaling function and a quantization function. s The linear scaling function normalizes a weight vector with values ​​from any range into a vector with values ​​in the range [0,1]. It is calculated as follows:

[0105]

[0106] The quantization function is expressed as follows:

[0107]

[0108] In the formula, s is the quantization series. Let ξ be the floor function. i ξ is the rounding function. i ~Bernoulli(k i ), where k i For the original value v i The normalized distance between the coordinates and the random rounding point. In the improved quantization method, each scaled vector coordinate v i Instead of assigning the nearest quantization point, the quantization point is rounded based on the probability of the unbiased estimator of vi with the smallest variance. This prevents the network from selecting a local optimum and avoids increasing quantization error.

[0109] Step 6: Perform quantitative distillation by combining anti-distillation and homogeneous quantization.

[0110] The student model constructed in step 2 stores the weights in 32-bit floating-point numbers, which can be called a full-precision student model, and uniform quantization is performed on it.

[0111] The quantization process requires continuous iteration to balance efficiency gains and accuracy losses. Let v represent the initial network weights of the full-precision student model, then, according to the quantization function in step 5, calculate...

[0112] The new network weights can then be calculated as follows: in The overall loss from adversarial distillation in step 4 is represented by η, which is a hyperparameter. The new network weights v are then... new The gradient is used as the weight parameter in the student model for the next iteration, and this process is repeated until the gradient reaches zero, at which point the iteration ends. The distillation process is then complete, yielding the final output f of the student model. S That is, the predicted value of the student model.

[0113] Therefore, the quantization and distillation of the student model are performed in parallel at the macro level and alternately at the micro level. Instead of distilling and then quantizing, the loss caused by quantization and the loss caused by distillation are combined, which simplifies the optimization of quantization and distillation and yields a more accurate student model.

[0114] Step 7: To verify the effectiveness and accuracy of the method provided by this invention, experiments were conducted on two open-source bearing assemblies, including:

[0115] (1) Dataset from the bearings at the University of Paderborn (PU). The sampling frequency was 64 kHz, and each record contained 4 seconds of vibration data. Table 1 lists more details.

[0116] (2) Dataset from the bearings at Case Western Reserve University (CWRU). The sampling frequencies for C1 and C2 are 12 kHz and 48 kHz, respectively, and other conditions are listed in Table 1.

[0117] For each record, data augmentation is used to generate more samples for signal analysis. Each signal is segmented into samples, each with 1024 data points. For each training set, 500 labeled samples in the source domain and 500 unlabeled samples in the target domain are selected as the training set and used as input to the teacher model. 500 unlabeled samples are selected as the test set and used as input to the student model.

[0118] Table 1 Vibration datasets and detailed settings for the two bearing assemblies

[0119]

[0120] To verify the diagnostic performance and robustness of the proposed network, this paper sets up two types of transfer tasks using two datasets. First, a transfer task is set up between different PU bearing groups, including different damage levels, occurrence methods, rotational speeds, and operating conditions. Then, a transfer task is designed between different datasets of PU and CWRU. Using the improved ResNet18 network as the teacher model, the difference in diagnostic accuracy between the Small ResNet (without the proposed method) and the student model (using the proposed method) is compared. The former does not use knowledge from the teacher but learns from the data itself. To compare compression performance, quantization parameters s=4 and s=2 are chosen respectively to construct student models quantized to 4-bit and 2-bit integer values, referred to as the 4-bit student model and the 2-bit student model. Their model capacity and diagnostic performance are compared with the unquantized, full-precision student model.

[0121] In addition, the classic convolutional neural network (CNN), the gradient matching-based domain generalization method Fish, and the state-of-the-art adversarial domain invariant generalization network ADIG were selected for comparison.

[0122] Comparative Experiment 1: Performance comparison of the fault diagnosis of the migration task between different bearing groups in the same dataset PU using the model of this invention.

[0123] Take different bearing groups P1 and P2 from the PU dataset. They have the same type of bearings and operating conditions, but different degrees of artificial damage. Design a fault diagnosis task that allows them to migrate between each other. P3 and P4 are set up similarly, but the damage occurs in different ways. Figure 5The figures show experimental results of the method provided by this invention on transfer tasks between different bearing groups of the same dataset PU in an embodiment. Using the same training and test sets, the deeper, modified network ResNet18 significantly outperforms Small ResNet in diagnostic accuracy across all four tasks and approaches the state-of-the-art ADIG method, demonstrating better learning capabilities. However, using the method provided by this invention, the gap between the student model and the teacher model narrows, and the model size is approximately half that of the latter. Further integration with quantization, while slightly reducing accuracy, significantly reduces model size; for example, the 4-bit student model achieves comparable accuracy to the full-precision student model, but its model size is only about one-eighth of the latter.

[0124] Comparative Experiment 2: Performance comparison of the fault diagnosis of the present invention model on the migration task between different datasets PU and CWRU bearings.

[0125] In this example, more migration tasks are set up in pairs between the two sets of open-source bearings. The only thing the two sets of bearings have in common is that the damage in C1, C2, and P4 is artificially generated, while the damage in P5 is spontaneously generated. Figure 6 Radar plot of experimental results for the transfer task between different datasets PU and CWRU; Figure 7 The confusion matrix diagram for the experimental results is shown. In large-domain transfer, the student model using the method provided in this invention significantly outperforms the teacher model in most tasks, demonstrating its superior performance. Although the performance of the 2-bit student model degrades significantly, the 4-bit student model achieves similar or even higher accuracy than the teacher model in most tasks, while being much smaller, approximately one-seventeenth the size of the teacher model.

[0126] It should be understood that any parts not described in detail in this specification belong to the prior art. Those skilled in the art should understand that the above embodiments are merely to help readers understand the principles and implementation methods of the present invention, and the scope of protection of the present invention is not limited to such embodiments. All equivalent substitutions made based on the present invention are within the scope of protection of the present invention.

Claims

1. A lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation, characterized in that, Includes the following steps: Step 1: Design a teacher model with multiple layers, a large number of convolutional kernels, and neurons; The teacher model uses ResNet18 as the backbone network. Depthwise separable convolutional (DSC) and attention module CBAM are introduced into the teacher model to improve the performance of ResNet18. DSC layers are added after the first two blocks of ResNet18, and CBAM is added after the third block. Step 2: Construct a student model; The initial ResNet18 model is simplified based on the proposed residual blocks to obtain the lightweight network model SmallResNet, which is then used to build a student model. The specific steps are as follows: Based on the residual structure of ResNet18, a "residual block" structure is proposed, which consists of a convolutional layer, a BN layer, a ReLU function, a convolutional layer, and a BN layer from top to bottom. The initial ResNet18 model is simplified based on the proposed residual blocks to obtain the lightweight network model SmallResNet, which is then used to construct a student model. The entire student model consists of a convolutional layer, two normalization layers, another convolutional layer, another normalization layer, another convolutional layer, and three residual blocks. The first convolutional layer uses a 1×9 kernel to extract initial features from the data. Then, two consecutive normalization layers adjust and stabilize the data distribution. The second convolutional layer uses a 1×5 kernel to further extract more complex features. The third normalization layer again adjusts the data distribution, providing stable input data for the next layer. The third convolutional layer uses a 1×3 kernel for final feature extraction. Then, through the operation of the three residual blocks, the data size is reduced to one-third of its original size, and the number of channels is doubled. At the end of the network, a combination of global average pooling and fully connected layers is used, requiring no parameter calculation. The fully connected layers map the features learned by the network to class labels, ultimately completing the pre-training of the student model. Step 3: Perform knowledge distillation; The teacher model is used as a feature extractor and trained according to different connection class classifiers or domain classifiers in the dataset. After training, the teacher model uses the softmax layout in the network structure as soft labels to guide the student model. Step 4: Use adversarial learning to help the student model achieve a higher accuracy; the specific method is as follows: A discriminator network D is used to determine whether the input comes from the student model or the teacher model; the loss function of the discriminator is: in, and Let represent the probability distributions output by the teacher model and the student model, respectively. The loss function for the generator is as follows: The generator, or student model, aims to continuously optimize its output distribution to be as similar as possible to the teacher model's output distribution, to the point that the discriminator cannot distinguish them. The smaller the loss function, the better the student model's learning performance. Finally, the overall loss function for counterdistillation can be expressed as: in ∈ [0, 1], which are hyperparameters used to balance distillation losses and counteracting losses; Step 5: Perform uniform quantization on the student model using the improved quantization method; the specific steps are as follows: Uniform quantization first determines the quantization level s based on requirements, where s is the number of integer bits after quantization. Uniform quantization at level s selects s ​​equally spaced points within the interval from 0 to 1 as quantization points, mapping the weight values ​​of the weight vector to the nearest quantization point, thus reducing storage and computation. The weight vector is a matrix representing the connection weights between each input and output neuron in a fully connected neural network. For the student model, the initial network weight parameters are: The quantized weight parameters are obtained by sequentially applying a linear scaling function and a quantization function. The linear scaling function normalizes a weight vector with values ​​from any range into a vector with values ​​in the range [0,1], and is calculated as follows: The quantization function is expressed as follows: In the formula, For quantization series, This is a rounding function. ~Bernoulli( ),in Original value The normalized distance between the coordinates and the random rounding point; in the improved quantization method, each scaled vector coordinate... Instead of assigning the nearest quantization point, the quantization point is rounded based on the probability of the unbiased estimator of vi with the smallest variance, to prevent the network from selecting a local optimum and avoid increasing quantization error. Step 6: Perform quantitative distillation by combining counter-distillation and homogeneous quantization; Step 7: Implement real-time diagnosis of lightweight bearing faults based on the trained student model. The specific method is as follows: Raw vibration signal data is collected by an accelerometer installed on the bearing base to obtain data on the entire life cycle of the bearing, and multiple vibration signal data are obtained by using a sliding window segmentation method. The acquired vibration signal data is input into the trained student model, enabling real-time fault diagnosis. The output is the student model's own prediction. .

2. The lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation according to claim 1, characterized in that, CBAM consists of a channel attention module and a spatial attention module, combining the channel attention mechanism CBM and the spatial attention mechanism SBM. By learning the weights assigned to different channels and regions in an image, the network can better focus on task-relevant channels and regions. All steps can be simply represented as: Where F is the input feature map. and These are the computational mechanisms for channel attention and spatial attention, respectively. This represents element-wise multiplication. This is the final output.

3. The lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation according to claim 2, characterized in that, The specific method for step 3 is as follows: Using the teacher model as a feature extractor And classify according to different connection classes of the dataset. or domain classifier Their parameters are respectively , and ; For the source dataset Its class tag is ,in It refers to the amount of source data; the classifier is trained to minimize the source domain loss. That is, the negative logarithmic probability of the correct label within the source domain: The target dataset uses a different unlabeled dataset than the source dataset. ,in It is the quantity of target data; if If it comes from the source domain, then its domain label Set to 0, otherwise The value is 1; the training of the domain classifier is to maximize the loss on both the source and target domains. : The teacher model is first connected to a class classifier and trained using class labels; this minimizes the source domain loss. Then, the teacher model is connected to the domain classifier and trained using domain labels to obtain the loss that maximizes both the source and target domains. Finally, the teacher model uses the softmax layout in the network structure as soft labels to guide the student model; the soft labels are the class probability distributions predicted by the trained teacher model for the samples, reflecting the model's confidence in the samples belonging to each class. For the student model, it is necessary to fit not only the true labels of the dataset but also the output distribution of the teacher model; therefore, the loss function is distilled. Defined as hard loss function With soft loss function sum; The student model is represented as In summary, The definition is as follows: in ∈ [0, 1], which are hyperparameters used to balance soft loss and hard loss; soft loss The difference between the outputs of the student model and the teacher model is represented by the following definition, expressed using cross-entropy: For hard loss This represents the difference between the student model's output and the class label, expressed using cross-entropy: )。 4. The lightweight bearing fault diagnosis method based on uniform quantization and counter-distillation according to claim 3, characterized in that, Step 6 is performed as follows: The student model constructed in step 2 stores the weights in 32-bit floating-point numbers, which is called the full-precision student model, and uniform quantization is performed on it. The quantization process requires continuous iteration to balance efficiency gains with accuracy losses; Let v represent the initial network weights of the full-precision student model, then calculate the quantization function in step 5. ; The new network weights can then be calculated as follows: ,in To counteract the overall loss during distillation in step 4, For hyperparameters; change the new network weights As the weight parameters of the student model, they are used for the next iteration, and this process is repeated until the gradient is 0, at which point the iteration ends. The distillation process is now complete, and the final output of the student model is obtained. That is, the predicted value of the student model.

Images