Lightweight bearing fault diagnosis method based on uniform quantization and anti-distillation

Abstract

The invention discloses a lightweight bearing fault diagnosis method based on uniform quantization and anti-distillation. A knowledge distillation and antagonistic learning method is adopted, a student model obtains valuable information by learning a soft label provided by a teacher network, and a smaller student model can also have diagnosis accuracy similar to that of a deep network under the help of the teacher network which is well learned. And meanwhile, uniform quantification is combined with the distillation process, so that the scale of the student model is remarkably compressed. According to the method, calculation is simple, the result is effective, the lightweight network is more suitable for actual equipment with limited calculation and storage resources, the method can be applied to some small equipment such as mobile phones and embedded equipment, and bearing fault real-time diagnosis is achieved.

Description

Technical Field

[0001] The invention belongs to the technical field of fault diagnosis, and in particular relates to a lightweight bearing fault diagnosis method based on uniform quantization and counter distillation. Background Art

[0002] Currently, for rolling bearing fault diagnosis, researchers mostly use semi-supervised prototype enhancement networks for analysis and processing, such as using double-pooled attention residual networks for feature extraction modules, using algorithms to design strategies for updating prototypes with pseudo-labels, and accurately adjusting the prototype positions of each type of limited labeled samples through unlabeled samples to achieve the effect of enhancing the prototype, thereby improving the network model's ability to discriminate fault categories.

[0003] This scheme has a high accuracy rate for rolling bearing fault diagnosis, but there are problems such as the model structure is too complex and the operation depends on high-performance servers and expensive computing devices, which greatly hinders the application of these models on some small devices, such as mobile phones and embedded devices. Therefore, the present invention is committed to creating a lightweight and more efficient fault diagnosis model while maintaining its performance level for real-time applications.

[0004] Knowledge distillation can transfer knowledge from a larger deep network (teacher network) to a smaller network (student model). During the training process, the output of the teacher model is used as a soft label, and this label is provided to the student model for learning, so as to achieve the purpose of transferring the knowledge learned by the teacher model to the student model.

[0005] Adversarial distillation is an improvement on knowledge distillation. It introduces an adversarial loss function so that the student model not only tries to fit the soft labels of the teacher model, but also adversarially generates hard labels, making it difficult for the teacher model to distinguish the output of the student model. This adversarial training can help the student model better capture the knowledge of the teacher model and improve performance. Adversarial distillation uses the GAN architecture. The teacher network uses real samples as the training set; the generator is the student model and is used to generate the output of the softmax layer; the discriminator is used to determine the probability that the input sample is a real sample.

[0006] Parameter quantization refers to replacing high-precision floating-point values ​​with low-precision integer values, thereby greatly reducing the storage space of the network and improving the computing speed. The present invention innovatively combines parameter quantization with knowledge distillation, and uses adversarial learning to enhance the generalization ability of the model, so that it can minimize the loss of precision in the sparse process. This is of great significance for better utilization of the hardware of the computing platform. Summary of the invention

[0007] The purpose of the present invention is to solve the problem of high computational cost generated by deep networks with a large number of parameters in predictive maintenance scenarios, and to provide a lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation, comprising the following steps:

[0008] Step 1. Design a teacher model with multiple layers, a large number of convolution kernels and neurons.

[0009] Step 2: Build a student model.

[0010] Based on the proposed residual block, the initial ResNet18 model is simplified to obtain a lightweight network model Small ResNet, which is used to construct the student model.

[0011] Step 3: Perform knowledge distillation.

[0012] The teacher model is used as a feature extractor and trained according to different connection class classifiers or domain classifiers of the dataset. After training, the teacher model uses the softmax layout in the network structure as a soft label to guide the student model.

[0013] Step 4: Use adversarial learning to help the student model achieve a higher accuracy.

[0014] Step 5: Uniformly quantize the student model using the improved quantization method.

[0015] Step 6: Comprehensively perform quantitative distillation on adversarial distillation and uniform quantization.

[0016] Step 7: Realize real-time diagnosis of lightweight bearing faults based on the trained student model.

[0017] As a preferred method, the teacher model uses the residual network ResNet18 as the backbone network. The teacher model introduces the deep separable convolution DSC and the attention module CBAM to improve the performance of ResNet18. The DSC layer is added after the first two blocks of ResNet18, and the CBAM is added after the third block. DSC can be used to reduce the computational complexity and parameters of the model.

[0018] CBAM consists of a channel attention module and a spatial attention module, which combines the channel attention mechanism CBM and the spatial attention mechanism SBM. By learning the weights assigned to different channels and regions in the image, the network can better focus on the channels and regions related to the task. All steps can be simply expressed as:

[0019]

[0020]

[0021] Where F is the input feature map, Q C (·) and Q S (·) are the operation mechanisms of channel attention and spatial attention, respectively. represents element-by-element multiplication, and F″ is the final output.

[0022] Preferably, the specific operation of step 2 is as follows:

[0023] Based on the residual structure of Resnet18, a "residual block" structure is proposed, which consists of convolutional layer, BN layer, ReLU function, convolutional layer, and BN layer from top to bottom.

[0024] Based on the proposed residual block, the initial ResNet18 model is simplified to obtain a lightweight network model Small ResNet, and the student model is constructed with it. The entire student model consists of a convolutional layer, two normalization layers, a convolutional layer, a normalization layer, a convolutional layer, and three residual blocks in sequence. In the first convolutional layer of the network, a convolution kernel of size 1×9 is used to extract the initial features of the data. Then, two consecutive normalization layers are used to adjust and stabilize the data distribution. In the second convolutional layer of the network, a convolution kernel of size 1×5 is used to further extract more complex features. The third normalization layer adjusts the data distribution again to provide stable input data for the next layer. The third convolutional layer uses a convolution kernel of size 1×3 for the final feature extraction. Then, through the operation of three residual blocks, the data size is reduced to 1 / 3 of the original size, and the number of channels is increased to twice the original size. At the end of the network, a combination of a global average pooling layer and a fully connected layer is used without any parameter calculation. The fully connected layer maps the features learned by the network to class labels, and finally completes the pre-training of the student model.

[0025] Preferably, the specific method of step 3 is as follows:

[0026] Use the teacher model as the feature extractor G f (·;θ f ), and connect the classifier G according to the different data sets y (·;θ y ) or domain classifier G d (·;θ d ), whose parameters are θ f ,θ y and θ d .

[0027] For the source dataset {x i}, whose class label is {y i}(i=1,...,n s ), where n s is the number of source data. The classifier is trained to minimize the source domain loss That is, the negative log probability of the correct label in the source domain:

[0028]

[0029] The target dataset uses an unlabeled dataset {x j}, whose domain label is {y j}(j=1,...,n t ), where n t is the number of target data. If x j From the source domain, its domain label y j Set to 0, otherwise y j is 1. The domain classifier is trained to maximize the loss on the source domain and the target domain

[0030]

[0031] The teacher model is first connected to the class classifier and trained with the class label. The source domain loss is minimized. Finally, the teacher model is connected to the domain classifier and trained with domain labels to maximize the loss on the source domain and the target domain. Finally, the teacher model uses the softmax layout in the network structure as a soft label to guide the student model. The soft label is the category probability distribution predicted by the trained teacher model for the sample, reflecting the model's confidence that the sample belongs to each category.

[0032] For the student model, it is necessary not only to fit the true label of the dataset, but also to fit the output distribution of the teacher model. Therefore, the distillation loss function Defined as a hard loss function With soft loss function The sum of the teacher model is represented as f T , the student model is represented by f S In summary, The definition is as follows:

[0033]

[0034] Where λ∈[0,1] is a hyperparameter used to balance soft loss and hard loss. It represents the gap between the output of the student model and the teacher model, which is defined as follows and expressed in cross entropy:

[0035]

[0036] For hard losses It represents the difference between the output of the student model and the class label, expressed as cross entropy:

[0037]

[0038] Preferably, the specific method of step 4 is as follows:

[0039] The discriminator network D is used to determine whether the input comes from the student model or the teacher model. The loss function of the discriminator is:

[0040]

[0041] Among them, p T (y i ) and p S (y i ) represent the probability distribution of the output of the teacher model and the student model respectively, Represents the loss function of the generator, as follows:

[0042]

[0043] The goal of the generator, or student model, is to make the output distribution as similar as possible to the output distribution of the teacher model through continuous optimization, to the extent that the discriminator cannot distinguish. The smaller the loss function, the better the learning effect of the student model.

[0044] Finally, the overall loss function of adversarial distillation can be expressed as:

[0045]

[0046] Where ∝∈[0,1] is a hyperparameter used to balance the distillation loss and the adversarial loss.

[0047] Preferably, step 5 is specifically performed as follows:

[0048] Uniform quantization first determines the quantization level s according to the requirements, where s is the integer number of bits after quantization. Uniform quantization of level s selects s ​​equally spaced points in the range of 0 to 1 as quantization points, and maps the weight value of the weight vector to the nearest quantization point to reduce the amount of storage calculation. The weight vector is a matrix representing the connection weights between each input and output neuron in a fully connected neural network.

[0049] For the student model, the initial network weight parameter is v i , use the linear scaling function and the quantization function in turn to get the quantized weight parameter v s . The linear scaling function can normalize the weight vector with values ​​from any range to a vector with values ​​in [0,1], calculated as:

[0050]

[0051] The quantization function is expressed as follows:

[0052]

[0053] In the formula, s is the quantization level, is the floor function, ξ i is the rounding function, ξ i ~Bernoulli(k i ), where k i is the original value v i The normalized distance between the random rounding point and the original point. In the improved quantization method, each scaling vector coordinate v i None of them are assigned to the nearest quantization point, but the quantization point is rounded according to the probability of the unbiased estimator of vi with the smallest variance, preventing the network from selecting the local optimal point and avoiding increasing the quantization error.

[0054] Preferably, step 6 is specifically performed as follows:

[0055] The student model constructed in step 2 stores weights in 32-bit floating point numbers, called the full-precision student model, on which uniform quantization is performed.

[0056] The quantization process needs to be iterated continuously to balance the efficiency improvement and accuracy loss. Let v represent the initial network weight of the full-precision student model, then according to the quantization function in step 5, calculate

[0057] Then the new network weight can be calculated as in is the overall loss of adversarial distillation in step 4, and η is a hyperparameter. new As the weight parameter of the student model, the next iteration is carried out, and the cycle repeats until the gradient is 0 and the iteration ends. At this point, the distillation process is completed and the final output f of the student model is obtained S , which is the predicted value of the student model.

[0058] Preferably, the specific method of step 7 is as follows:

[0059] The original vibration signal data is collected by installing an acceleration sensor on the bearing base to obtain data on the entire life cycle of the bearing, and a sliding window segmentation method is used to obtain multiple segments of vibration signal data.

[0060] The acquired vibration signal data is input into the student model to perform real-time fault diagnosis, and the output is the student model’s own predicted output f S .

[0061] The beneficial effects of the present invention are as follows:

[0062] The present invention proposes a lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation. The knowledge distillation and adversarial learning methods are adopted to allow the student model to obtain valuable information by learning the soft labels provided by the teacher network. With the help of a well-learned teacher network, a smaller student model can also have a diagnostic accuracy similar to that of a deep network. At the same time, the uniform quantization is combined with the distillation process to significantly compress the size of the student model. Various experiments and comparisons have proved the effectiveness of the proposed lightweight residual network. The method of the present invention is simple to calculate and the results are effective, making the lightweight network more suitable for practical devices with limited computing and storage resources. In addition, the present invention has good reusability in similar application scenarios, and the practical value of the invention is relatively strong. The present invention provides a lightweight and more efficient fault diagnosis model that does not rely on high-performance servers and expensive computing equipment, and can be used on some small devices, such as mobile phones and embedded devices, to achieve real-time diagnosis of bearing faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0064] Figure 1 A logic schematic diagram of a lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation provided by the present invention;

[0065] Figure 2 A network structure diagram of the main teacher model for knowledge distillation provided by the present invention;

[0066] Figure 3 A network structure diagram of the main student model of knowledge distillation provided by the present invention;

[0067] Figure 4 The main flow chart of uniform quantization provided by the present invention;

[0068] Figure 5 This is a graph showing the experimental results of the migration task between different bearing groups of the same data set PU in the embodiment of the method provided by the present invention;

[0069] Figure 6 A radar chart showing the experimental results of the migration task between different data sets PU and CWRU in the embodiment of the method provided by the present invention;

[0070] Figure 7 A confusion matrix diagram of experimental results of the migration task between different data sets PU and CWRU in an embodiment of the method provided by the present invention. DETAILED DESCRIPTION

[0071] In order to enable relevant personnel to more clearly understand the technical content of the present invention, the technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods. The present invention discloses a lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation, which compresses the scale of the student model as much as possible and maintains a diagnostic accuracy similar to that of a deep network. The method of the present invention is simple to calculate and the results are effective. It is feasible and superior in actual bearing fault diagnosis and cross-domain scenarios. In addition, the present invention has good reusability in similar application scenarios, and the practical value of the invention is relatively strong.

[0072] like Figure 1 As shown, a lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation, the implementation example includes the following steps:

[0073] Step 1. Design a teacher model with multiple layers, a large number of convolution kernels and neurons. The residual network ResNet18 is used as the backbone network. ResNet18 includes a convolution layer and a maximum pooling layer, four convolution blocks, an average pooling layer, a fully connected layer and a softmax output. The network structure diagram is attached. Figure 2 The number of input layer nodes is 2048, and the number of output layer nodes is 8.

[0074] like Figure 2 As shown in the figure, the teacher model introduces the depth-separable convolution DSC and the attention module CBAM to improve the performance of ResNet18. The DSC layer is added after the first two blocks of ResNet18, and the CBAM is added after the third block. DSC can be used to reduce the computational complexity and parameters of the model. Each convolution kernel of the standard convolution layer performs a convolution operation on each channel of the input data. Then, the result values ​​corresponding to each convolution kernel are accumulated as the output.

[0075] CBAM consists of a channel attention module and a spatial attention module, which combines the channel attention mechanism CBM and the spatial attention mechanism SBM. By learning the weights assigned to different channels and regions in the image, the network can better focus on the channels and regions related to the task. All steps can be simply expressed as:

[0076]

[0077]

[0078] Where F is the input feature map, Q C (·) and Q S (·) are the operation mechanisms of channel attention and spatial attention, respectively. represents element-by-element multiplication, and F″ is the final output.

[0079] Step 2: Build a student model.

[0080] The present invention designs a "residual block" structure, which is composed of convolution layer, BN layer, ReLU function, convolution layer, and BN layer from top to bottom. Based on the residual block, the initial ResNet18 model is simplified to obtain a lightweight network model SmallResNet, and the student model is constructed with it. Figure 3 As shown in the figure, the entire student model consists of a convolutional layer, two normalization layers, a convolutional layer, a normalization layer, a convolutional layer, and three residual blocks. In the first convolutional layer of the network, a convolution kernel of size 1×9 is used to extract the initial features of the data. Then, two consecutive normalization layers are used to adjust and stabilize the data distribution. In the second convolutional layer of the network, a convolution kernel of size 1×5 is used to further extract more complex features. The third normalization layer adjusts the data distribution again to provide stable input data for the next layer. The third convolutional layer uses a convolution kernel of size 1×3 for the final feature extraction. Then, through the operation of three residual blocks, the data size can be reduced to 1 / 3 of the original size, and the number of channels is increased to twice the original size. At the end of the network, a combination of a global average pooling layer and a fully connected layer is used without any parameter calculation. The fully connected layer maps the features learned by the network to class labels, and finally completes the pre-training of the student model.

[0081] Step 3: Use the teacher model as the feature extractor G f (·;θ f ), and connect the classifier G according to the different data sets y (·;θ y ) or domain classifier G d (·;θ d ), whose parameters are θ f ,θ y and θ d Among them, the purpose of the class classifier is to classify the data samples, that is, to determine which specific category the sample belongs to; the domain classifier is to determine whether the data belongs to the source domain or the target domain.

[0082] For the source dataset {x i}, whose class label is {y i}(i=1,...,n s ), where n s is the number of source data. The classifier is trained to minimize the source domain loss That is, the negative log probability of the correct label in the source domain:

[0083]

[0084] The target dataset uses an unlabeled dataset {x j}, whose domain label is {yj}(j=1,...,n t ), where n t is the number of target data. If x j From the source domain, its domain label y j Set to 0, otherwise y j is 1. The domain classifier is trained to maximize the loss on the source domain and the target domain

[0085]

[0086] The teacher model is first connected to the class classifier and trained with the class label. The source domain loss is minimized. Finally, the teacher model is connected to the domain classifier and trained with domain labels to maximize the loss on the source domain and the target domain. Finally, the teacher model uses the softmax layout in the network structure as a soft label to guide the student model. The soft label is the category probability distribution predicted by the trained teacher model for the sample, reflecting the model's confidence that the sample belongs to each category.

[0087] For the student model, it is necessary not only to fit the true label of the dataset, but also to fit the output distribution of the teacher model. Therefore, the distillation loss function Defined as a hard loss function With soft loss function The sum of the teacher model is represented as f T , the student model is represented by f S In summary, The definition is as follows:

[0088]

[0089] Where λ∈[0,1] is a hyperparameter used to balance soft loss and hard loss. It represents the gap between the output of the student model and the teacher model, which is defined as follows and expressed in cross entropy:

[0090]

[0091] For hard losses It represents the difference between the output of the student model and the class label, expressed as cross entropy:

[0092]

[0093] Step 4: Use adversarial learning to help the student model achieve a higher accuracy.

[0094] The discriminator network D is used to determine whether the input comes from the student model or the teacher model. The loss function of the discriminator is:

[0095]

[0096] Among them, p T (y i ) and p S (y i ) represent the probability distribution of the output of the teacher model and the student model respectively, Represents the loss function of the generator, as follows:

[0097]

[0098] The goal of the generator, or student model, is to make the output distribution as similar as possible to the output distribution of the teacher model through continuous optimization, to the extent that the discriminator cannot distinguish. The smaller the loss function, the better the learning effect of the student model.

[0099] Finally, the overall loss function of adversarial distillation can be expressed as:

[0100]

[0101] Where ∝∈[0,1] is a hyperparameter used to balance the distillation loss and the adversarial loss.

[0102] Step 5: Figure 4 As shown, the student model is uniformly quantized by the improved quantization method. The specific operations are as follows:

[0103] Quantization can convert high-precision floating-point numbers into low-precision integers. First, the quantization level s is determined according to the requirements, where s is the number of integer bits after quantization. Uniform quantization at level s selects s ​​equally spaced points in the range of 0 to 1 as quantization points, and maps the weight value of the weight vector to the nearest quantization point to reduce the amount of storage calculation. The weight vector is a matrix representing the connection weights between each input and output neuron in a fully connected neural network.

[0104] For the student model, the initial network weight parameter is v i , use the linear scaling function and the quantization function in turn to get the quantized weight parameter v s . The linear scaling function can normalize the weight vector with values ​​from any range to a vector with values ​​in [0,1], calculated as:

[0105]

[0106] The quantization function is expressed as follows:

[0107]

[0108] In the formula, s is the quantization level, is the floor function, ξ i is the rounding function, ξ i ~Bernoulli(k i ), where k i is the original value v i The normalized distance between the random rounding point and the original point. In the improved quantization method, each scaling vector coordinate v i None of them are assigned to the nearest quantization point, but the quantization point is rounded according to the probability of the unbiased estimator of vi with the smallest variance, which can prevent the network from selecting the local optimal point and avoid increasing the quantization error.

[0109] Step 6: Comprehensively perform quantitative distillation on adversarial distillation and uniform quantization.

[0110] The student model constructed in step 2 stores weights in 32-bit floating point numbers and can be called a full-precision student model, on which uniform quantization is performed.

[0111] The quantization process needs to be iterated continuously to balance the efficiency improvement and accuracy loss. Let v represent the initial network weight of the full-precision student model, then according to the quantization function in step 5, calculate

[0112] Then the new network weight can be calculated as in is the overall loss of adversarial distillation in step 4, and η is a hyperparameter. new As the weight parameter of the student model, the next iteration is carried out, and the cycle repeats until the gradient is 0 and the iteration ends. At this point, the distillation process is completed and the final output f of the student model is obtained S , which is the predicted value of the student model.

[0113] Therefore, the quantization and distillation of the student model are carried out in parallel on a macro level and alternately on a micro level. Instead of distillation followed by quantization, the loss caused by quantization and the loss caused by distillation are combined, which simplifies the optimization of quantization and distillation and obtains a more accurate student model.

[0114] Step 7: To verify the effectiveness and accuracy of the method provided by the present invention, experiments are conducted on two open source bearing groups, including:

[0115] (1) Dataset from Paderborn University (PU) bearings. The sampling frequency is 64 kHz and each record contains 4 s of vibration data. Table 1 lists more details.

[0116] (2) Dataset of bearings from Case Western Reserve University (CWRU). The sampling frequencies of C1 and C2 are 12 kHz and 48 kHz respectively, and other conditions are also listed in Table 1.

[0117] For each record, data augmentation is used to generate more samples for signal analysis. Each signal is split into samples, each with 1024 data points. 500 labeled samples in the source domain and 500 unlabeled samples in the target domain are selected as training sets for each training set and used as input for the teacher model. 500 unlabeled samples are selected as test sets and used as input for the student model.

[0118] Table 1 Vibration data sets of two bearing groups and their detailed settings

[0119]

[0120] In order to verify the diagnostic performance and robustness of the proposed network, two types of migration tasks were set up using two sets of data sets. First, the transfer task between different PU bearing groups was set, including different damage levels, generation methods, rotation speeds, and working conditions. Then, the migration task between different data sets PU and CWRU was designed. The improved ResNet18 network was used as the teacher model to compare the difference in diagnostic accuracy between the Small ResNet without the proposed method and the student model using the proposed method. The former does not use the knowledge from the teacher, but learns the data by itself. In order to compare the compression performance, the quantization parameters s=4 and s=2 were selected respectively, and the student models quantized to 4-bit integer values ​​and 2-bit integer values ​​were constructed, called 4-bit student model and 2-bit student model, and their model capacity and diagnostic performance were compared with the full-precision student model without uniform quantization.

[0121] In addition, the classic convolutional neural network CNN, the gradient matching based domain generalization method Fish and the most advanced adversarial domain invariant generalization network ADIG were selected for comparison.

[0122] Comparative experiment 1: Comparison of the fault diagnosis performance of the proposed model in migration tasks between different bearing groups in the same data set PU.

[0123] Take different bearing groups P1 and P2 in the PU dataset, which have the same type of bearings and working conditions, but different degrees of artificial damage, and design a fault diagnosis task that migrates between them. P3 and P4 use similar settings, but the damage is generated in different ways. Figure 5The figure shows the experimental results of the migration task between different bearing groups of the same data set PU of the embodiment of the method provided by the present invention. Using the same training set and test set, the deeper and modified network ResNet18 has a much higher diagnostic accuracy than Small ResNet on four tasks, and is close to the most advanced ADIG method, showing better learning ability. However, after using the method provided by the present invention, the gap between the student model and the teacher model is narrowed, and the model size is about half of the latter. Further combined with quantization, although the accuracy is slightly reduced, the model size is greatly reduced. For example, the accuracy of the 4-bit student model is comparable to that of the full-precision student model, but the model size is only about one-eighth of the latter.

[0124] Comparative experiment 2: Comparison of the fault diagnosis performance of the proposed model in migration tasks between different data sets PU and CWRU bearings.

[0125] In this case, more migration tasks are set up in pairs between two sets of open source bearings. The only thing in common between the two sets of bearings is that the damage in C1, C2, and P4 is artificially generated, while the damage in P5 is spontaneously generated. Figure 6 Radar chart showing the experimental results of the migration task between different datasets PU and CWRU; Figure 7 The confusion matrix diagram of the experimental results. In large domain migration, the student model using the method provided by the present invention performs much better than the teacher model in most tasks, reflecting its superior performance. Although the performance of the 2-bit student model is significantly reduced, in most tasks, the 4-bit student model has similar or even higher accuracy than the teacher model, while the scale is much smaller, about one seventeenth of the teacher model.

[0126] It should be known that the parts not elaborated in detail in this specification belong to the prior art. Relevant technical personnel should understand that the above embodiments are only to help readers understand the principles and implementation methods of the present invention, and the scope of protection of the present invention is not limited to such embodiments. All equivalent replacements made on the basis of the present invention are within the protection scope of the rights of the present invention.

Claims

1. A lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation, characterized in that: The steps include: Step 1: Design a teacher model with multiple layers, a large number of convolution kernels and neurons; Step 2: Construct a student model; Based on the proposed residual block, the initial ResNet18 model is simplified to obtain a lightweight network model SmallResNet, and the student model is constructed based on it; Step 3: Perform knowledge distillation; The teacher model is used as a feature extractor and trained according to different connection classifiers or domain classifiers of the data set; after the training is completed, the teacher model uses the softmax layout in the network structure as a soft label to guide the student model; Step 4: Use adversarial learning to help the student model achieve a higher accuracy; Step 5: Uniformly quantize the student model using the improved quantization method; Step 6: Comprehensively conduct quantitative distillation based on anti-distillation and uniform quantization; Step 7: Realize real-time diagnosis of lightweight bearing faults based on the trained student model.

2. A lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation according to claim 1, characterized in that: The teacher model uses the residual network ResNet18 as the backbone network; the teacher model introduces the depth-separable convolution DSC and the attention module CBAM to improve the performance of ResNet18. The DSC layer is added after the first two blocks of ResNet18, and the CBAM is added after the third block; DSC can be used to reduce the computational complexity and parameters of the model; CBAM consists of a channel attention module and a spatial attention module, which combines the channel attention mechanism CBM and the spatial attention mechanism SBM. By learning the weights assigned to different channels and regions in the image, the network can better focus on the channels and regions related to the task; all steps can be simply expressed as: Where F is the input feature map, Q C (·) and Q S (·) are the operation mechanisms of channel attention and spatial attention respectively; represents element-wise multiplication, F ″ is the final output.

3. A lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation according to claim 2, characterized in that: Step 2: Based on the residual structure of Resnet18, a "residual block" structure is proposed, which consists of convolution layer, BN layer, ReLU function, convolution layer, and BN layer from top to bottom. Based on the proposed residual block, the initial ResNet18 model is simplified to obtain a lightweight network model SmallResNet, and the student model is constructed with it; the entire student model consists of a convolutional layer, two normalization layers, a convolutional layer, a normalization layer, a convolutional layer and three residual blocks in sequence; a convolution kernel of size 1×9 is used in the first convolutional layer of the network to extract the initial features of the data; then two consecutive normalization layers are used to adjust and stabilize the data distribution; a convolution kernel of size 1×5 is used in the second convolutional layer of the network to further extract more complex features; the third normalization layer adjusts the data distribution again to provide stable input data for the next layer; the third convolutional layer uses a convolution kernel of size 1×3 for the final feature extraction; then through the operation of three residual blocks, the data size is reduced to 1 / 3 of the original size, and the number of channels is increased to twice the original size; a combination of a global average pooling layer and a fully connected layer is used at the end of the network without any parameter calculation; the fully connected layer maps the features learned by the network to class labels, and finally completes the pre-training of the student model.

4. A lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation according to claim 3, characterized in that: Step 3: Use the teacher model as the feature extractor G f (·;θ f ), and connect the classifier G according to the different data sets y (·;θ y ) or domain classifier G d (·;θ d ), whose parameters are θ f ,θ y and θ d ; For the source dataset {x i }, whose class label is {y i }(i=1,...,n s ), where n s is the amount of source data; the classifier is trained to minimize the source domain loss That is, the negative log probability of the correct label in the source domain: The target dataset uses an unlabeled dataset {x j }, whose domain label is {y j }(j=1,...,n t ), where n t is the number of target data; if x j From the source domain, its domain label y j Set to 0, otherwise y j is 1; the domain classifier is trained to maximize the loss on the source domain and the target domain The teacher model first connects the class classifier and trains it through the class label; the minimum source domain loss is obtained Finally, the teacher model is connected to the domain classifier and trained with domain labels to maximize the loss on the source domain and the target domain. Finally, the teacher model uses the softmax layout in the network structure as a soft label to guide the student model; the soft label is the category probability distribution predicted by the trained teacher model for the sample, reflecting the model's confidence that the sample belongs to each category; For the student model, it is necessary not only to fit the true label of the dataset, but also to fit the output distribution of the teacher model; therefore, the distillation loss function Defined as a hard loss function With soft loss function The sum of the teacher model is represented as f T , the student model is represented by f S In summary, The definition is as follows: Where λ∈[0,1] is a hyperparameter used to balance soft loss and hard loss; soft loss It represents the gap between the output of the student model and the teacher model, which is defined as follows and expressed in terms of cross entropy: For hard losses It represents the difference between the output of the student model and the class label, expressed as cross entropy:

5. A lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation according to claim 4, characterized in that: Step 4: The discriminator network D is used to determine whether the input comes from the student model or the teacher model; the loss function of the discriminator is: Among them, p T (y i ) and p S (y i ) represent the probability distribution of the output of the teacher model and the student model respectively, Represents the loss function of the generator, as follows: The goal of the generator, or student model, is to make the output distribution as similar as possible to that of the teacher model through continuous optimization, to the extent that the discriminator cannot distinguish them. The smaller the loss function, the better the learning effect of the student model. Finally, the overall loss function of adversarial distillation can be expressed as: Where ∝∈[0,1] is a hyperparameter used to balance the distillation loss and the adversarial loss.

6. A lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation according to claim 5, characterized in that: Step 5: Uniform quantization first determines the quantization level s according to the requirements, where s is the integer bit number after quantization. Uniform quantization of level s selects s ​​equally spaced points in the interval from 0 to 1 as quantization points, and maps the weight value of the weight vector to the nearest quantization point, reducing the amount of storage calculation. The weight vector is a matrix representing the connection weights between each input and output neuron in a fully connected neural network; For the student model, the initial network weight parameter is v i , use the linear scaling function and the quantization function in turn to get the quantized weight parameter v s ; The linear scaling function can normalize the weight vector with values ​​from any range to a vector with values ​​in [0,1], calculated as: The quantization function is expressed as follows: In the formula, s is the quantization level, is the floor function, ξ i is the rounding function, ξ i ~Bernoulli(k i ), where k i is the original value v i The normalized distance between the random rounding point and the original point; in the improved quantization method, each scaling vector coordinate v i None of them are assigned to the nearest quantization point, but the quantization point is rounded according to the probability of the unbiased estimator of vi with the smallest variance, preventing the network from selecting the local optimal point and avoiding increasing the quantization error.

7. A lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation according to claim 6, characterized in that: Step 6: The student model constructed in step 2 stores weights in 32-bit floating point numbers, called the full-precision student model, on which uniform quantization is performed; The quantization process needs to be continuously iterated to balance efficiency improvement and accuracy loss; Let v represent the initial network weight of the full-precision student model, then according to the quantization function in step 5, calculate Then the new network weight can be calculated as in is the overall loss of adversarial distillation in step 4, η is a hyperparameter; the new network weight v new As the weight parameter of the student model, the next iteration is carried out, and this cycle is repeated until the gradient is 0 and the iteration ends; at this time, the distillation process is completed and the final output f of the student model is obtained S , which is the predicted value of the student model.

8. The lightweight bearing fault diagnosis method based on uniform quantization and adversarial distillation according to claim 7 is characterized in that: Step 7: The original vibration signal data is collected by the acceleration sensor installed on the bearing base to obtain the data of the entire life cycle of the bearing, and the sliding window segmentation method is used to obtain multiple segments of vibration signal data; The acquired vibration signal data is input into the trained student model to perform real-time fault diagnosis, and the output is the predicted output f of the student model itself. S .

Images