A fault identification method, system and device based on feature-level cross-model knowledge distillation

Abstract

The present application relates to the technical field of simulation analysis, and in particular to a fault identification method, system and device based on feature-level cross-model knowledge distillation, which comprises the following steps: obtaining gearbox historical fault vibration data and corresponding true labels, and constructing a training set; constructing a teacher model, constructing a student model by using cross-channel pooling and coordinate attention joint optimization GhostNet, adjusting the student model structure so that the number of feature extraction modules of the student model and the teacher model is the same; aligning the multi-level features of the teacher model and the student model by using local maximum mean difference, and iteratively optimizing the student model by using classification loss and feature-level distillation loss multiple times, when the multi-level feature distribution difference between the teacher model and the student model is minimized, and the optimal student model is saved; inputting the data to be detected into the optimal student model to obtain the fault identification result; the present application improves the diagnostic precision on the basis of ensuring low calculation amount and low parameter amount of the identification model.

Description

Technical Field

[0001] This invention relates to the field of simulation analysis technology, and specifically to a fault identification method, system, and apparatus based on feature-level cross-model knowledge distillation. Background Technology

[0002] Gearboxes are common components for connecting and transmitting power, widely used in modern industrial equipment such as wind power, aviation, power systems, agricultural machinery, transportation machinery, and machine tools. However, due to the complexity of their structure and operating conditions, as well as the harsh working environment, critical parts of gearboxes are frequently damaged by corrosion and fracture. Untimely maintenance can endanger personal safety and cause significant economic losses. Therefore, fault diagnosis of gearboxes is of great importance in both academic and industrial fields.

[0003] Deep convolutional neural networks (CNNs) possess powerful fitting capabilities and excellent feature extraction performance, leading to their widespread application in fault diagnosis. Their general process involves feature extraction, feature classification, loss calculation, and network parameter optimization. However, high-precision network models typically involve high computational cost and numerous parameters, consuming significant storage space within nodes. Furthermore, the rapid development of edge computing, a next-generation information technology, has resulted in an increasing number of edge nodes with limited resources. Their computing power is insufficient to support the real-time operation of large-scale deep convolutional neural network models. This prevents the direct deployment of computationally intensive and parameter-rich deep convolutional neural network models on resource-constrained edge nodes. Therefore, ensuring model accuracy while reducing computational cost and parameter count to accommodate resource-limited edge nodes is a problem that needs to be addressed.

[0004] The gearbox fault diagnosis method based on feature-level knowledge distillation eliminates the need for complex model structure design. It guides a low-precision student model through a high-precision teacher model, forcing the student model's feature-level network response to approximate the corresponding feature-level network response of the teacher model. This aligns the feature-level outputs of the teacher and student models, thereby improving the student model's classification accuracy. The trained student model is not only small in size but also highly accurate, sometimes even surpassing the performance of the teacher model. Such a small-parameter student model can be easily applied to edge devices, such as mobile phones and mobile robots, maximizing efficiency with limited computing resources. Summary of the Invention

[0005] To ensure model accuracy and reduce computational and parameter counts to accommodate resource-constrained edge nodes, this invention proposes a fault identification method based on feature-level cross-model knowledge distillation, which specifically includes the following steps:

[0006] S1. Obtain historical fault vibration data of the gearbox and corresponding real labels, and construct a training set;

[0007] S2. A residual network is used to construct a teacher model. The teacher model is pre-trained and the optimal teacher model parameters are fixed. A GhostNet model is constructed using cross-channel pooling and coordinate attention joint optimization. The structure of the student model is adjusted so that the number of feature extraction modules in the student model and the teacher model are the same.

[0008] S3. Input the training set samples into the teacher model and student model respectively for feature-level distillation training. Use the local maximum mean difference to align the multi-level features of the teacher model and student model. Iterate multiple times to jointly optimize the student model through classification loss and feature-level distillation loss. When the difference in the multi-level feature distribution between the teacher model and student model is minimized, save the optimal student model.

[0009] S4. Input the data to be detected into the optimal student model to obtain the fault identification result.

[0010] Furthermore, a student model is constructed using GhostNet with joint optimization of cross-channel pooling and coordinate attention. Specifically, cross-channel pooling is used to improve the bottleneck module with a stride of 1 in GhostNet, and cross-channel pooling and coordinate attention are combined to improve the bottleneck module with a stride of 2 in GhostNet. The specific steps include:

[0011] For bottleneck modules with a stride of 1 in GhostNet, add a cross-channel pooling between two Ghost modules;

[0012] For the bottleneck module with stride 2 in GhostNet, cross-channel pooling is added between the first Ghost module and the separable convolutional layer, and a coordinate attention layer is added between the separable convolutional layer and the second Ghost module.

[0013] Furthermore, the cross-channel pooling process includes:

[0014] The feature information of the same position in different channels of the input feature map is transformed into the same plane to form a new feature map;

[0015] Perform max pooling on the new feature map;

[0016] After pooling, inverse pooling is performed, which restores the information of each channel to the channel dimension corresponding to the input feature map, resulting in a cross-channel pooled feature map.

[0017] Furthermore, the coordinate attention layer's operations on the input feature map include:

[0018] Two 1D pooling kernels, (H, 1) and (1, W), are used to perform average pooling along the horizontal and vertical directions respectively on the input feature map.

[0019] The results of the two average pooling operations are concatenated in the spatial dimension and a 1×1 convolution is used to compress the channels.

[0020] Spatial information in the vertical and horizontal directions is encoded through batch normalization and nonlinear transformation;

[0021] The segmentation process involves splitting the feature map along the spatial dimension into two tensors. Each tensor is then processed by a 1×1 convolution to obtain the result with respect to the input feature map. Figure 1 A feature map of the number of channels;

[0022] Normalize and weight the feature maps obtained from the two tensors;

[0023] Where H represents the height of the feature map and W represents the width of the feature map.

[0024] Furthermore, the local maximum mean difference is used to align the multi-layer features of the teacher and student models. Alignment is completed when the local maximum mean difference between the teacher and student models no longer decreases. The local maximum mean difference between the teacher and student models is expressed as:

[0025]

[0026] in, For the local maximum mean difference loss, p represents the high-dimensional feature distribution of the samples in the teacher model, q represents the high-dimensional feature distribution of the samples in the student model, and D... c This represents the sample set belonging to the c-th type of fault. Let φ(·) represent the i-th sample belonging to the c-th fault type. teacher Let φ(·) represent the high-dimensional feature mapping of the teacher model. student This represents the high-dimensional feature mapping of the student model, where {1, 2, ..., C} represents the set of class labels, and C represents the total number of class labels; ||·|| H This indicates that the parameters are mapped to the Hilbert space for computation.

[0027] Furthermore, the student model is jointly optimized through multiple iterations using classification loss and feature-level distillation loss, including:

[0028]

[0029]

[0030] Among them, Loss 蒸馏 The loss function used when training the student model using distillation. The loss represents the local maximum mean difference; N is the total number of samples; y (i) The true label is the one corresponding to the i-th sample. Let be the soft label corresponding to the i-th sample.

[0031] This invention also provides a fault identification system based on feature-level cross-model knowledge distillation. This system is used to implement a fault identification method based on feature-level cross-model knowledge distillation. The system includes a teacher model and a student model deployed on resource-limited edge nodes. The teacher model is a fault identification network based on a residual network, and the student model is a GhostNet jointly optimized by cross-channel pooling and coordinate attention. After aligning the student model with the teacher model, the student model is jointly optimized through multiple iterations of classification loss and feature-level distillation loss. When the difference in multi-level feature distribution between the teacher model and the student model is minimized, the optimal student model is saved, and the optimal student model is called for fault identification on resource-limited edge nodes.

[0032] Furthermore, GhostNet employs a joint optimization of cross-channel pooling and coordinate attention. Specifically, for the bottleneck module with a stride of 1 in GhostNet, a cross-channel pooling layer is added between two Ghost modules; for the bottleneck module with a stride of 2 in GhostNet, a cross-channel pooling layer is added between the first Ghost module and the separable convolutional layer, and a coordinate attention layer is added between the separable convolutional layer and the second Ghost module.

[0033] The present invention also provides a fault identification device based on feature-level cross-model knowledge distillation, which is deployed on resource-limited edge nodes to implement a fault identification method based on feature-level cross-model knowledge distillation.

[0034] Compared with existing fault diagnosis methods, the gearbox fault diagnosis method based on feature-level knowledge distillation in this invention does not require changes to the internal structure of the student model, reducing the time spent designing the model. At the same time, it ensures low computational load and low parameter count for the student model. By using a high-precision teacher model to guide the low-precision student model in learning, the network response at the feature level of the student model is forced to approximate the network response at the feature level of the teacher model, thereby aligning the feature-level outputs of the teacher model and the student model and minimizing the difference between the student model and the true label to improve the fault diagnosis accuracy of the student model. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating a specific implementation of a gearbox fault diagnosis method based on feature-level knowledge distillation disclosed in this invention.

[0036] Figure 2 This is a structural diagram of the GhostNetCACP model of the present invention;

[0037] Figure 3 This is a schematic diagram of the feature-level knowledge distillation training process of the present invention;

[0038] Figure 4 This is a flowchart of the feature-level knowledge distillation training process of the present invention;

[0039] Figure 5 The graph shows the simulation results of this invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] This invention proposes a fault identification method based on feature-level cross-model knowledge distillation, such as... Figure 1 Specifically, it includes the following steps:

[0042] S1. Obtain historical fault vibration data of the gearbox and corresponding real labels, and construct a training set;

[0043] S2. A residual network is used to construct a teacher model. The teacher model is pre-trained and the optimal teacher model parameters are fixed. A GhostNet model is constructed using cross-channel pooling and coordinate attention joint optimization. The structure of the student model is adjusted so that the number of feature extraction modules in the student model and the teacher model are the same.

[0044] S3. Input the training set samples into the teacher model and student model respectively for feature-level distillation training. Use the local maximum mean difference to align the multi-level features of the teacher model and student model. Iterate multiple times to jointly optimize the student model through classification loss and feature-level distillation loss. When the difference in the multi-level feature distribution between the teacher model and student model is minimized, save the optimal student model.

[0045] S4. Input the data to be detected into the optimal student model to obtain the fault identification result.

[0046] Example 1

[0047] In the existing technology, the GhostNet network includes two types of GhostBlock: GhostBlock with stride of 1 and GhostBlock with stride of 2. GhostBlock with stride of 1 consists of two Ghost modules cascaded together, while GhostBlock with stride of 2 is based on GhostBlock with stride of 1, with an added depthwise separable convolutional layer between the two Ghost modules.

[0048] In this embodiment, a cross-channel pooling (CP) operation is introduced into the existing GhostNet network. Specifically, in the bottleneck module with a stride of 1, cross-channel pooling is set between two Ghost modules; in the bottleneck module with a stride of 2, cross-channel pooling is set between the first Ghost module and the separable convolutional layer.

[0049] The GhostNet network with added cross-channel pooling, abbreviated as GhostNetCP, performs batch normalization (BN) and ReLU operations on the feature map output by the first module before performing cross-channel pooling. The feature map input to the cross-channel pooling layer is processed as follows:

[0050] The feature information of the same position in different channels of the input feature map is transformed into the same plane to form a new feature map;

[0051] Perform max pooling on the new feature map;

[0052] After pooling, inverse pooling is performed, which restores the information of each channel to the channel dimension corresponding to the input feature map, resulting in a cross-channel pooled feature map.

[0053] Example 2

[0054] This embodiment introduces Coord_ATT (CA) into the existing GhostNet network. The GhostNet network with Coord_ATT is referred to as GhostNetCA. In this network, Coord_ATT is placed in the bottleneck module with a stride of 2. Specifically, it is placed between the separable convolutional layer and the second Ghost module. The feature map after batch normalization of the output of the separable convolutional layer is input into the Coord_ATT layer. The processing of the feature map by the Coord_ATT layer includes the following steps:

[0055] Two 1D pooling kernels, (H, 1) and (1, W), are used to perform average pooling along the horizontal and vertical directions respectively on the input feature map.

[0056] The results of the two average pooling operations are concatenated in the spatial dimension and a 1×1 convolution is used to compress the channels.

[0057] Spatial information in the vertical and horizontal directions is encoded through batch normalization and nonlinear transformation;

[0058] The segmentation process involves splitting the feature map along the spatial dimension into two tensors. Each tensor is then processed by a 1×1 convolution to obtain the result with respect to the input feature map. Figure 1 A feature map of the number of channels;

[0059] The feature maps obtained from the two tensors are normalized and weighted.

[0060] Example 3

[0061] This embodiment improves the existing GhostNet network by combining Embodiments 1 and 2. The improved network is referred to as GhostNetCACP. Figure 2 The improvements include:

[0062] For bottleneck modules with a stride of 1 in GhostNet, add a cross-channel pooling between two Ghost modules;

[0063] For the bottleneck module with stride 2 in GhostNet, cross-channel pooling is added between the first Ghost module and the separable convolutional layer, and a coordinate attention layer is added between the separable convolutional layer and the second Ghost module.

[0064] The training process of the gearbox fault diagnosis method based on feature-level knowledge distillation involves using a high-precision teacher model to guide a low-precision student model in learning. This forces the student model's feature-level network response to approximate the teacher model's corresponding feature-level network response. The loss is calculated and backpropagated using the differences between the feature-level outputs of the teacher and student models, as well as the differences between the soft labels and true labels output by the student model. The parameters of the student model are then optimized and updated to minimize the distance between the sub-feature-level outputs of the two models. Training stops when the loss no longer decreases, at which point the teacher and student models are considered to have achieved feature alignment, meaning the output of the student model is identical to that of the teacher model.

[0065] The performance of the lightweight student model used in this invention before and after the improvement is shown in Table 1.

[0066] Table 1 Performance of the Lightweight Student Model Before and After Improvement

[0067]

[0068]

[0069] As can be seen, the two improved methods disclosed in this invention, cross-channel pooling and coordinate attention mechanism, have significant lightweight effects on the computational and parameter quantities of the model, and the accuracy has also increased from the original 90.82% to 94.08%.

[0070] The network structure and feature alignment of the teacher and student models are shown in Table 2. In this embodiment, the input layers of the teacher and student models include Conv2d, BN, and Maxp (Max pooling), the feature extraction modules include four modules, and the output layers include fully connected, BN, and ReLU.

[0071] Table 2. Network structure and feature alignment of teacher and student models.

[0072]

[0073] Example 4

[0074] In this embodiment, the teacher model is a residual network. It should be noted that the teacher model can also be any network structure used in existing fault diagnosis. This invention does not substantially improve the network used as the teacher model; therefore, the network structure of the teacher model will not be described in detail. The loss function used to train the teacher model in this embodiment is:

[0075]

[0076] Where L is the cross-entropy loss and y is the true label. This is the classification result.

[0077] Those skilled in the art can also use other loss functions to train the teacher model according to actual needs. In this invention, the teacher model is defined as a fault identification network with a large number of parameters, relatively accurate fault identification, and which, due to its large number of parameters, cannot be deployed on resource-limited edge nodes.

[0078] In this invention, those skilled in the art can divide the recognition network of the teacher model into an input layer, an output layer, and multiple feature extraction layers, requiring that the number of feature extraction layers in the student model correspond to the number of feature extraction layers in the teacher model.

[0079] In this embodiment, the feature extraction layers of the teacher model and the student model are aligned. During alignment, the local maximum mean difference between the teacher model and the student model is minimized. In practical applications, alignment is completed when the local maximum mean difference between the teacher model and the student model is less than a set threshold. The formula for the local maximum mean difference loss function used for aligning the multi-layer feature extraction layers of the teacher model and the student model in the gearbox fault diagnosis method based on feature-level knowledge distillation is as follows:

[0080]

[0081] in, For the local maximum mean difference loss, p represents the high-dimensional feature distribution of the samples in the teacher model, q represents the high-dimensional feature distribution of the samples in the student model, and D... c This represents the sample set belonging to the c-th type of fault. Let φ(·)teacher represent the i-th sample belonging to the c-th fault type, φ(·)student represent the high-dimensional feature mapping of the teacher model, φ(·)student represent the high-dimensional feature mapping of the student model, {1, 2, ..., C} represent the set of class labels, and C represent the total number of class labels; ||·|| H This indicates that the parameters are mapped to the Hilbert space for computation.

[0082] In this embodiment, after alignment is completed, the student model needs to be trained. The loss function used for training the student model is:

[0083]

[0084] The above adjustment process for the student model is as follows: Figures 3-4 , Figure 3 The paper proposes aligning each feature extraction layer of the student and teacher models using Local Maximum Mean Difference (LMMD). Figure 4 The process for fine-tuning the student model is given, including the following steps:

[0085] Fine-tune the student model, divide the feature extraction modules into student and teacher models, and align the last layer of each module;

[0086] Import the training set data into the student model;

[0087] The hyperparameters of the student model are set during the training process;

[0088] The loss function is used to determine whether the subdomain feature layer outputs of the teacher model and the student model are aligned, and whether the classification results of the student model are aligned with the true labels.

[0089] If all are aligned, then the optimal student network is obtained;

[0090] Otherwise, optimize the network based on the loss function and determine whether the maximum number of optimization iterations has been reached. If it has, output the result; otherwise, continue importing training data into the student model.

[0091] Example 5

[0092] This embodiment proposes a system for implementing the fault identification method based on feature-level cross-model knowledge distillation in Embodiment 3. The system includes a teacher model and a student model deployed on resource-limited edge nodes. The teacher model is a fault identification network based on a residual network, and the student model is a GhostNet jointly optimized by cross-channel pooling and coordinate attention. After aligning the student model with the teacher model, the student model is jointly optimized through multiple iterations of classification loss and feature-level distillation loss. When the difference in multi-level feature distribution between the teacher model and the student model is minimized, and the optimal student model is saved, the optimal student model is called for fault identification on resource-limited edge nodes.

[0093] Similarly, this embodiment also proposes a device deployed on resource-limited edge nodes, which is used to implement a fault identification method based on feature-level cross-model knowledge distillation.

[0094] To verify the effectiveness of the method disclosed in this invention, vibration signals of planetary gears under different health conditions were collected using an accelerometer. Simultaneously, to simulate actual equipment failures, this invention designed eight different failures of the planetary gearbox: gear surface wear failure, gear tooth root crack failure, gear tooth breakage failure, gear eccentricity failure, gear missing tooth failure, bearing rolling element failure, bearing inner ring failure, and bearing outer ring failure. All data were collected under constant speed and constant load. A data point was taken for every 1024 points of the original vibration, resulting in a total of 6400 vibration data points of 1024 points each, with 800 data points per class. After randomly shuffling the data, 10 samples were selected from each class (80 samples for the training set) and 750 samples from each class were selected for the test set, for a total of 6000 samples. The number of iterations was 100. Five random trials were conducted, and the average value was taken. The accuracy curves of the teacher model, the student model before distillation, and the model of the method disclosed in this invention are shown in the figure below. Figure 5 As shown in Table 3, the curve Teacher represents the classification accuracy curve of the teacher model, Student represents the classification curve of the student model before distillation, and Student_with_KD represents the classification curve of this invention. The specific parameter values ​​are shown in Table 3.

[0095] Table 3 Comparison of fault diagnosis effects between the teacher model, the student model before distillation, and the present invention.

[0096]

[0097] As can be seen, the gearbox fault diagnosis method based on feature-level knowledge distillation in this invention improves accuracy by 2.33% compared to the student model before distillation, and even exceeds the teacher model by 0.95%; in terms of parameter quantity and computational cost, it shows a significant decrease compared to the teacher model, with the parameter quantity decreasing by 0.93M and the computational cost decreasing by 280.54MMac.

[0098] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A fault identification method based on feature-level cross-model knowledge distillation, characterized in that, Specifically, the following steps are included: S1. Obtain historical fault vibration data of the gearbox and corresponding real labels, and construct a training set; S2. Construct and train the teacher model, and construct the student model using GhostNet with joint optimization of cross-channel pooling and coordinate attention. Adjust the structure of the student model so that the number of feature extraction modules in the student model and the teacher model are the same. Construct the student model using GhostNet with joint optimization of cross-channel pooling and coordinate attention, that is, use cross-channel pooling to improve the bottleneck module with a stride of 1 in GhostNet, and combine cross-channel pooling and coordinate attention to improve the bottleneck module with a stride of 2 in GhostNet. Specifically, this includes the following steps: For bottleneck modules with a stride of 1 in GhostNet, add a cross-channel pooling between two Ghost modules; For the bottleneck module with stride 2 in GhostNet, cross-channel pooling is added between the first Ghost module and the separable convolutional layer, and a coordinate attention layer is added between the separable convolutional layer and the second Ghost module. S3. Input the training set samples into the teacher model and student model respectively for feature-level distillation training, including: using local maximum mean difference to align the multi-level features of the teacher model and student model, iterating and jointly optimizing the student model through classification loss and feature-level distillation loss multiple times, and saving the optimal student model when the difference in the multi-level feature distribution between the teacher model and student model is minimized. S4. Input the data to be detected into the optimal student model to obtain the fault identification result.

2. The fault identification method based on feature-level cross-model knowledge distillation according to claim 1, characterized in that, The cross-channel pooling process includes: The feature information of the same position in different channels of the input feature map is transformed into the same plane to form a new feature map; Perform max pooling on the new feature map; After pooling, inverse pooling is performed, which restores the information of each channel to the channel dimension corresponding to the input feature map, resulting in a cross-channel pooled feature map.

3. The fault identification method based on feature-level cross-model knowledge distillation according to claim 1, characterized in that, The coordinate attention layer operates on the input feature map including: Two 1D pooling kernels, (H, 1) and (1, W), are used to perform average pooling along the horizontal and vertical directions respectively on the input feature map. The results of the two average pooling operations are concatenated in the spatial dimension and a 1×1 convolution is used to compress the channels. Spatial information in the vertical and horizontal directions is encoded through batch normalization and nonlinear activation. The feature map is split into two tensors along the spatial dimension, and each tensor is convolved with a 1×1 convolution to obtain a feature map with the same number of channels as the input feature map. Normalize and weight the feature maps obtained from the two tensors; Where H represents the height of the feature map and W represents the width of the feature map.

4. The fault identification method based on feature-level cross-model knowledge distillation according to claim 1, characterized in that, The local maximum mean difference is used to align the multi-layer features of the teacher and student models. Alignment is complete when the local maximum mean difference between the teacher and student models no longer decreases. The local maximum mean difference between the teacher and student models is expressed as: in, For the local maximum mean difference loss, p represents the high-dimensional feature distribution of the samples in the teacher model, q represents the high-dimensional feature distribution of the samples in the student model, and D... c This represents the sample set belonging to the c-th type of fault. This represents the i-th sample belonging to the c-th fault type. This represents the high-dimensional feature mapping of the teacher model. Let {1, 2, ..., C} represent the high-dimensional feature mapping of the student model, where {1, 2, ..., C} represent the set of class labels, and C represents the total number of class labels. This indicates that the parameters are mapped to the Hilbert space for computation.

5. The fault identification method based on feature-level cross-model knowledge distillation according to claim 1, characterized in that, The student model is jointly optimized through multiple iterations using classification loss and feature-level distillation loss, including: in, The loss function used when training the student model using distillation. This represents the loss due to the local maximum mean difference; N is the total number of samples. Let i be the true label corresponding to the i-th sample; Let be the soft label corresponding to the i-th sample.

6. A fault identification system based on feature-level cross-model knowledge distillation, characterized in that, This system is used to implement the fault identification method based on feature-level cross-model knowledge distillation as described in claim 1. The system includes a teacher model and a student model deployed on resource-limited edge nodes. The teacher model is a fault identification network based on a residual network, and the student model is a GhostNet jointly optimized by cross-channel pooling and coordinate attention. After aligning the student model with the teacher model, the student model is jointly optimized through multiple iterations of classification loss and feature-level distillation loss. When the difference in multi-level feature distribution between the teacher model and the student model is minimized, the optimal student model is saved, and the optimal student model is called on the resource-limited edge nodes for fault identification.

7. A fault identification system based on feature-level cross-model knowledge distillation according to claim 6, characterized in that, GhostNet employs a joint optimization of cross-channel pooling and coordinate attention. Specifically, for bottleneck modules with a stride of 1 in GhostNet, a cross-channel pooling layer is added between two Ghost modules; for bottleneck modules with a stride of 2 in GhostNet, a cross-channel pooling layer is added between the first Ghost module and the separable convolutional layer, and a coordinate attention layer is added between the separable convolutional layer and the second Ghost module.

8. A fault identification device based on feature-level cross-model knowledge distillation, characterized in that, The device is deployed on resource-constrained edge nodes to implement the fault identification method based on feature-level cross-model knowledge distillation as described in claim 1.

Images