Insulator defect detection method, system and equipment based on cross-layer knowledge distillation

Abstract

The invention belongs to the technical field of insulator defect detection, and particularly relates to an insulator defect detection method, system and equipment based on cross-layer knowledge distillation, a teacher detection model and a to-be-trained student detection model are constructed and pre-trained, multi-scale features of the teacher detection model and the to-be-trained student detection model for the same image are extracted, space and channel alignment is carried out on the student features, and the defect detection result is obtained. Performing weighted fusion on the aligned high and low layer features of the student, dividing a local region, extracting local features, generating a channel weight through an attention mechanism in combination with global features, and obtaining an enhanced feature map after weighting; according to the method, a student model is optimized in combination with detection loss of students, first distillation loss based on feature alignment and second distillation loss based on classification / regression feature cross-correlation statistic alignment, and multi-scale feature extraction and prediction are performed on an input image by using the optimized student model, so that efficient insulator defect detection is realized.

Description

Technical Field

[0001] This invention belongs to the field of insulator defect detection technology based on knowledge distillation, specifically relating to insulator defect detection methods, systems, and equipment based on cross-layer knowledge distillation. Background Technology

[0002] In China's extensive power transmission network, insulators serve as critical components for current isolation and electrical safety. However, long-term exposure to extreme weather conditions can cause corrosion, leading to defects such as cracks, aging, or complete detachment. Global statistics show that more than 75% of power grid accidents each year originate from insulator defects.

[0003] Existing methods lack cross-scale collaboration and fail to establish effective connections between multiple scales. For insulator detection that exhibits scale changes, high-level semantic information should guide the extraction of low-level details, but current methods lack this top-down path. They also lack effective mechanisms to ensure feature consistency, and the models often exhibit contradictions between confidence and localization bias. Furthermore, they lack joint modeling of spatial location and channel importance. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and device for detecting insulator defects based on cross-layer knowledge distillation.

[0005] Insulator defect detection methods based on cross-layer knowledge distillation include: S1. Construct a teacher detection model and pre-train the teacher detection model using centralized images from the insulator defect dataset; S2. Construct a student detection model; S3. The teacher detection model and the student detection model extract multi-scale feature maps of the same image in the insulator defect dataset to obtain high-level features of the teacher, low-level features of the teacher, high-level features of the student, and low-level features of the student. The low-level features of the student are matched with the low-level features of the teacher in terms of space and channel to obtain aligned low-level features of the student. The high-level features of the student are matched with the high-level features of the teacher in terms of space and channel to obtain aligned high-level features of the student. S4. The aligned low-level features of students and the aligned high-level features of students are weighted and fused to obtain fused features. The fused features and the low-level features of students are skipped to obtain connected features. Global features are obtained based on connected features. The connected features are divided into multiple non-overlapping local regions. The local features of each region are concatenated with the global features and attention processing is performed to obtain the channel attention weight of the region. The channel attention weight of each region is used to perform weighted summation of the corresponding local features to generate the enhanced feature map of students. S5. Obtain the internal detection loss of the student detection model; calculate the first distillation loss based on the aligned low-level features of the student and the enhanced feature map of the student; extract classification features and regression features from the enhanced feature map of the student and the multi-scale feature map of the teacher detection model training image, respectively, calculate the cross-correlation statistic, and calculate the second distillation loss based on the cross-correlation statistic; obtain the total distillation loss based on the internal detection loss, the first distillation loss and the second distillation loss. S6. Update the student detection model parameters, repeat S3 to S5 a preset number of times, and select the student detection model with the student detection model parameters that have the minimum total distillation loss as the final student detection model. S7. The final student detection model acquires the image to be detected, extracts multi-scale features, and extracts classification features and regression features on each feature map layer respectively. It then predicts the defect category and bounding box coordinates, obtains the defect category label and confidence score, and completes the insulator defect detection.

[0006] The cross-correlation statistics in S5 include: the mean and variance of the classification features of the enhanced feature maps of students and the multi-scale feature maps of the teacher detection model training images, the mean and variance of the regression features, and the channel cross-correlation between the classification features and regression features of the feature maps.

[0007] The specific steps for calculating the channel cross-correlation between the categorical and regression features of the feature map are as follows: , in, This represents the cross-correlation of channels, where N is the total number of spatial locations. , These represent the mean of the categorical feature and the mean of the regression feature, respectively. , These are categorical features and regression features, respectively. Let C be the set of all real numbers in dimension C.

[0008] The variance of the classification features is calculated using a gradient-based weighting strategy. Specifically, the weight of each spatial location is determined based on the gradient norm of the classification feature on the internal detection loss, and then the variance of the classification features is calculated.

[0009] In S5, the total distillation loss is obtained based on the internal detection loss, the first distillation loss, and the second distillation loss. The specific operation is as follows: The distillation loss is obtained by weighted summation of the first and second distillation losses: ; in, For distillation loss, , This is an empirical coefficient. , These are the first distillation loss and the second distillation loss, respectively. The total distillation loss is obtained by summing the distillation loss and the internal detection loss: , in, This is an empirical coefficient. This is due to internal detection losses.

[0010] The pre-training described in S1 specifically involves training the teacher detection model using a detection loss function until convergence, so that the teacher detection model can output the defect classification and location regression results of the insulator.

[0011] In S4, after concatenating the local features and global features of each region, attention processing is performed to obtain the channel attention weights of that region. Specifically, after concatenating the local features and global features of each region along the channel dimension, the first fully connected processing, ReLU activation processing, second fully connected processing, and Sigmoid activation processing are performed sequentially to obtain the channel attention weights of that region.

[0012] S7 also includes performing filtering and non-maximum suppression processing based on confidence thresholds on the obtained defect category labels and confidence scores to obtain insulator defect detection results and complete the insulator defect detection.

[0013] An insulator defect detection system based on cross-layer knowledge distillation is used to implement the aforementioned insulator defect detection method based on cross-layer knowledge distillation, including: The model building module constructs a teacher detection model and pre-trains it using images from a dataset of insulator defects; it also constructs a student detection model. In the alignment module, the teacher detection model and the student detection model extract multi-scale feature maps of the same image in the insulator defect dataset to obtain high-level features of the teacher, low-level features of the teacher, high-level features of the student, and low-level features of the student. The low-level features of the student are matched with the low-level features of the teacher in terms of space and channel to obtain aligned low-level features of the student. The high-level features of the student are matched with the high-level features of the teacher in terms of space and channel to obtain aligned high-level features of the student. The image enhancement module performs weighted fusion of aligned low-level student features and aligned high-level student features to obtain fused features. It then skips connections between the fused features and low-level student features to obtain connected features. Based on the connected features, it obtains global features and divides the connected features into multiple non-overlapping local regions. After concatenating the local features of each region with the global features, it performs attention processing to obtain the channel attention weights of that region. Finally, it uses the channel attention weights of each region to perform weighted summation of the corresponding local features to generate the enhanced feature map of the student. The loss calculation module obtains the internal detection loss of the student detection model; calculates the first distillation loss based on the aligned low-level features of the student and the enhanced feature map of the student; extracts classification features and regression features from the enhanced feature map of the student and the multi-scale feature map of the training image of the teacher detection model, respectively, calculates the cross-correlation statistic, and calculates the second distillation loss based on the cross-correlation statistic; and obtains the total distillation loss based on the internal detection loss, the first distillation loss, and the second distillation loss. The update module updates the parameters of the student detection model and selects the student detection model with the parameters that minimize the total distillation loss as the final student detection model. The detection module uses the final student detection model to acquire the image to be detected, extract multi-scale features, and extract classification features and regression features on each feature map layer, respectively, to predict the defect category and bounding box coordinates, obtain the defect category label and confidence score, and complete the insulator defect detection.

[0014] An insulator defect detection device based on cross-layer knowledge distillation includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement an insulator defect detection method based on cross-layer knowledge distillation.

[0015] The beneficial effects of this application are as follows: (1) This application transforms the problem of insulator defect detection into quantitative constraints for feature enhancement and loss calculation, and establishes a knowledge distillation method specifically for insulator defect detection rather than general target detection.

[0016] (2) This application fully utilizes the complementarity of multi-scale features by establishing a direct connection path between high-level and low-level features, enabling the student detection model to capture semantic context and spatial details simultaneously.

[0017] (3) By calculating the mean, variance and cross-correlation statistics, this application ensures that it can accurately handle complex scenarios with blurred boundaries and partial occlusion. Attached Figure Description

[0018] Figure 1 This is a schematic diagram showing the results of insulator defect detection on the image to be inspected. Figure 1 (a) is the image to be detected, (b) is a schematic diagram of the defect detection results obtained by missing S4, S5 and S6, (c) is a schematic diagram of the defect detection results obtained by missing S5 and S6, (d) is a schematic diagram of the defect detection results obtained by missing S4, and (e) is a schematic diagram of the defect detection results obtained in this application. Detailed Implementation

[0019] Example 1 To further understand the content of this invention, the invention will be described in detail with reference to the embodiments.

[0020] Knowledge distillation offers a promising paradigm for coordinating lightweight deployment and high accuracy. Existing methods lack cross-scale collaboration and fail to establish effective connections between different image layers. For insulator detection, which exhibits 3-5x scale variations, high-level semantic information should guide low-level detail extraction, but current methods lack this top-down path. Furthermore, the lack of effective mechanisms to ensure feature consistency leads to inconsistent behavior in student models.

[0021] This invention relates to a method, system, and device for insulator defect detection based on cross-layer knowledge distillation, comprising: Insulator defect detection methods based on cross-layer knowledge distillation include: S1. Construct a teacher detection model and pre-train the teacher detection model using images from a dataset of insulator defects.

[0022] The teacher detection model can be constructed using a ResNet-101 backbone network. The pre-training process involves training the teacher detection model using a detection loss function until convergence. The teacher detection model can then output the defect classification and location regression results for the insulator.

[0023] S2. Construct a student detection model.

[0024] The student detection model can be constructed using a lightweight ResNet-18 / 50 backbone network.

[0025] S3. The teacher detection model and the student detection model extract multi-scale feature maps of the same image in the insulator defect dataset to obtain high-level features of the teacher, low-level features of the teacher, high-level features of the student, and low-level features of the student. The low-level features of the student are matched with the low-level features of the teacher in terms of space and channel to obtain aligned low-level features of the student. The high-level features of the student are matched with the high-level features of the teacher in terms of space and channel to obtain aligned high-level features of the student.

[0026] Given the same image in an insulator defect dataset, both the teacher detection model and the student detection model extract multi-scale features, obtaining high-level teacher features, low-level teacher features, high-level student features, and low-level student features, resulting in two multi-scale feature maps. Due to the difference in depth between the two models, the two multi-scale feature maps have inconsistent spatial scales and channel dimensions. It is necessary to use adaptive pooling processing through bilinear interpolation to unify the spatial resolution, upsample the high-level features to the low-level resolution, and use convolution processing to align the channel dimensions. Therefore, the student low-level features are matched with the teacher low-level features in terms of space and channel to obtain aligned student low-level features, and the student high-level features are matched with the teacher high-level features in terms of space and channel to obtain aligned student high-level features.

[0027] S4. The aligned low-level features of students are weighted and fused with the aligned high-level features of students to obtain fused features. The fused features and the low-level features of students are skipped to obtain connected features. Global features are obtained based on connected features. The connected features are divided into multiple non-overlapping local regions. Local features of each local region are extracted. The local features of each region are concatenated with the global features and attention processing is performed to obtain the channel attention weights of the region. The channel attention weights of each region are used to perform weighted summation of the corresponding local features to generate the enhanced feature map of students.

[0028] Specifically, the aligned low-level student features and the aligned high-level student features are weighted and fused to obtain the fused features. for: in, For students with low-level characteristics, For students with high-level characteristics, , All are empirical coefficients, with α=0.7 and β=0.3 determined empirically to balance low-level detail preservation and high-level semantic guidance.

[0029] By skipping the connection fusion feature and the student's low-level feature, the connection feature is obtained. Specifically: , The connectivity features are divided into N non-overlapping local regions {Ri}N, each with a size of p×p pixels. i =1 (p = 11 is derived from the median insulator size at typical UAV altitude). For the i-th local region, convolution processing is used to extract local features. Simultaneously, global features are obtained by processing the connection features through global average pooling. .

[0030] By establishing cross-layer connections from high to low levels and integrating global semantics with local details, this application fully utilizes the complementarity of multi-scale features by establishing direct connection paths between high-level and low-level features, enabling the student detection model to capture both semantic context and spatial details simultaneously.

[0031] Local and global features are concatenated along the channel dimension, and attention processing is performed to obtain the channel attention weights for that region. Specifically, the local features of each region are concatenated with the global features along the channel dimension, and then sequentially processed by a first fully connected layer, ReLU activation, a second fully connected layer, and a Sigmoid activation to obtain the channel attention weights for that region. .

[0032] By using the channel attention weights of each region to perform a weighted summation of their corresponding local features, an enhanced feature map of the student is generated. Obtained through element-wise weighted aggregation: .

[0033] S5. Obtain the internal detection loss of the student detection model; calculate the first distillation loss based on the aligned low-level features of the student and the enhanced feature map of the student; extract classification features and regression features from the enhanced feature map of the student and the multi-scale feature map of the teacher detection model training image, respectively, calculate the cross-correlation statistic, and calculate the second distillation loss based on the cross-correlation statistic; obtain the total distillation loss based on the internal detection loss, the first distillation loss and the second distillation loss.

[0034] Obtain the internal detection loss of the student detection model Based on the aligned low-level features of the students and the enhanced feature maps of the students, the first distillation loss is calculated. : , in, For students' enhanced feature maps, This is a transpose of the low-level features of students.

[0035] Classification features and regression features are extracted from the enhanced feature maps of students and the multi-scale feature maps of the teacher detection model training images, respectively. Cross-correlation statistics are calculated, including the mean and variance of the classification features, the mean and variance of the regression features, and the channel cross-correlation between the classification features and regression features of the multi-scale feature maps of students and the teacher detection model training images. By calculating the mean, variance, and cross-correlation statistics, this application ensures that it can accurately handle complex scenes with blurred boundaries and partial occlusion.

[0036] The specific steps for calculating the mean of categorical features are as follows: Flatten the feature map to be calculated into a vector. The mean of the categorical features is defined as the average of the categorical features at all spatial locations. Obtain the total number of spatial locations. Based on the total number of spatial locations, calculate the mean of the categorical features at all spatial locations: , in, Let N be the mean of the categorical features, and N be the total number of spatial locations. For classification features, Let C be the set of all real numbers in dimension C.

[0037] The variance of the classification features is calculated using a gradient-based weighting strategy. Specifically, the weight of each spatial location is determined based on the gradient norm of the classification feature on the internal detection loss, and then the variance of the classification features is calculated.

[0038] Weight of this position for: , in, Let be the set of all real numbers.

[0039] Classification feature variance : .

[0040] Mean of regression feature With variance The calculation is similar to the one above, but without weighting, because the positioning quality is more evenly distributed in space.

[0041] The specific steps for calculating the channel cross-correlation between the categorical and regression features of the feature map are as follows: , in, This represents the cross-correlation of channels, where N is the total number of spatial locations. , These represent the mean of the categorical feature and the mean of the regression feature, respectively. , These are categorical features and regression features, respectively. Let C be the set of all real numbers in dimension C.

[0042] The synergistic relationship between classification and regression tasks is calculated using cross-correlation statistics.

[0043] Calculation of second distillation loss based on cross-correlation statistics : in, , , , The mean and variance of the classification features and the mean and variance of the regression features of the enhanced feature map for students; , , , The mean and variance of classification features and the mean and variance of regression features of multi-scale feature maps used to train images for teacher detection models; Enhance the channel cross-correlation between categorical and regression features in the student's feature map; The cross-correlation between classification and regression features in multi-scale feature maps of training images for teacher detection models is studied.

[0044] The total distillation loss is obtained based on the internal detection loss, the first distillation loss, and the second distillation loss. The specific operation is as follows: The distillation loss is obtained by weighted summation of the first and second distillation losses: ; in, For distillation loss, , These are empirical coefficients, taken as 0.5 and 0.5 respectively. , These are the first distillation loss and the second distillation loss, respectively. The total distillation loss is obtained by summing the distillation loss and the internal detection loss: , in, This is an empirical coefficient, set to 1.0. This is due to internal detection losses.

[0045] S6. Update the student detection model parameters, repeat S3 to S5 a preset number of times, and select the student detection model with the minimum total distillation loss as the final student detection model.

[0046] S7. The final student detection model acquires the image to be detected, extracts multi-scale features, and extracts classification features and regression features on each feature map layer respectively. It then predicts the defect category and bounding box coordinates, obtains the defect category label and confidence score, and completes the insulator defect detection.

[0047] This application transforms the insulator defect detection problem into a quantitative constraint for feature enhancement and loss calculation, and establishes a knowledge distillation method specifically for insulator defect detection rather than general target detection.

[0048] S7 also includes performing filtering and non-maximum suppression processing based on confidence thresholds on the obtained defect category labels and confidence scores to obtain insulator defect detection results and complete the insulator defect detection.

[0049] An insulator defect detection system based on cross-layer knowledge distillation is used to implement the aforementioned insulator defect detection method based on cross-layer knowledge distillation, including: The model building module constructs a teacher detection model and pre-trains it using images from a dataset of insulator defects; it also constructs a student detection model. In the alignment module, the teacher detection model and the student detection model extract multi-scale feature maps of the same image in the insulator defect dataset to obtain high-level features of the teacher, low-level features of the teacher, high-level features of the student, and low-level features of the student. The low-level features of the student are matched with the low-level features of the teacher in terms of space and channel to obtain aligned low-level features of the student. The high-level features of the student are matched with the high-level features of the teacher in terms of space and channel to obtain aligned high-level features of the student. The image enhancement module performs weighted fusion of aligned low-level student features and aligned high-level student features to obtain fused features. It then skips connections between the fused features and low-level student features to obtain connected features. Based on the connected features, it obtains global features and divides the connected features into multiple non-overlapping local regions. After concatenating the local features of each region with the global features, it performs attention processing to obtain the channel attention weights of that region. Finally, it uses the channel attention weights of each region to perform weighted summation of the corresponding local features to generate the enhanced feature map of the student. The loss calculation module obtains the internal detection loss of the student detection model; calculates the first distillation loss based on the aligned low-level features of the student and the enhanced feature map of the student; extracts classification features and regression features from the enhanced feature map of the student and the multi-scale feature map of the training image of the teacher detection model, respectively, calculates the cross-correlation statistic, and calculates the second distillation loss based on the cross-correlation statistic; and obtains the total distillation loss based on the internal detection loss, the first distillation loss, and the second distillation loss. The update module updates the parameters of the student detection model and selects the student detection model with the parameters that minimize the total distillation loss as the final student detection model. The detection module uses the final student detection model to acquire the image to be detected, extract multi-scale features, and extract classification features and regression features on each feature map layer, respectively, to predict the defect category and bounding box coordinates, obtain the defect category label and confidence score, and complete the insulator defect detection.

[0050] An insulator defect detection device based on cross-layer knowledge distillation includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement an insulator defect detection method based on cross-layer knowledge distillation.

[0051] See Figure 1 This application provides a schematic diagram of the insulator defect detection results for the image to be inspected. Figure 1 (a) is the image to be detected, (b) is a schematic diagram of the defect detection results obtained by missing S4, S5 and S6, (c) is a schematic diagram of the defect detection results obtained by missing S5 and S6, (d) is a schematic diagram of the defect detection results obtained by missing S4, and (e) is a schematic diagram of the defect detection results obtained in this application.

[0052] Furthermore, (b) the student detection model lacks S4, S5, and S6, completely skipping the knowledge distillation process. That is, after executing S3, S4, S5, and S6 are not executed, relying solely on the internal detection loss of the student detection model itself. The final result is a detection model trained only on the dataset without any knowledge guidance from the teacher detection model; (c) the complete S4 is executed, but the distillation loss calculation based on the teacher detection model in S5 and the update based on the total distillation loss in S6 are skipped. S4 functions as an internal learnable component, but its learning objective is not to imitate the teacher, but to directly serve the final detection task. A detection model that integrates cross-layer and local-global attention mechanisms, but does not receive any knowledge injection from the teacher detection model; (d) Skip the entire S4, do not generate enhanced feature maps, and jump directly from the features aligned by S3 to S5. In S5, it only calculates the internal detection loss and the second distillation loss. Since there is no enhanced feature map from S4, when calculating the second distillation loss, the step of extracting features from the enhanced feature map of the student is changed to extracting classification and regression features from the original features aligned by the student detection model. It is a knowledge distillation that accepts the classification and regression task relationship of the teacher detection model, but it does not have cross-layer feature enhancement capabilities.

[0053] Figure 1 The results clearly show (b) producing a scattered, unfocused effect on the graph. (c) effectively focusing attention on the insulator region (red / orange area in the heatmap) while suppressing background interference from the power line structure. (d) further refining the spatial focus and enhancing the boundary response. (e) achieving the most concentrated and localized attention distribution with strong activation and minimal background noise around the insulator target.

[0054] This comparison demonstrates how the proposed method progressively refines feature representations to enhance target discrimination, validating the collaborative design of the cross-level knowledge transfer mechanism.

[0055] In addition, this application conducted ablation experiments based on the CPLID dataset. The experimental results show that the student detection model using this application achieves an mAP of 51.6%, while the mAP of the student detection model missing S4, S5 and S6 is 44.8%, the mAP of the student detection model missing S5 and S6 is 50.9%, and the mAP of the student detection model missing S4 is 50.2%. Compared with the student detection model missing S4, S5 and S6, this represents an improvement of 6.8% in mAP.

Claims

1. A method for detecting insulator defects based on cross-layer knowledge distillation, characterized in that, include: S1. Construct a teacher detection model and pre-train the teacher detection model using images from a dataset of insulator defects. S2. Construct a student detection model; S3. The teacher detection model and the student detection model extract multi-scale feature maps of the same image in the insulator defect dataset to obtain high-level features of the teacher, low-level features of the teacher, high-level features of the student, and low-level features of the student. The low-level features of the student are matched with the low-level features of the teacher in terms of space and channel to obtain aligned low-level features of the student. The high-level features of the student are matched with the high-level features of the teacher in terms of space and channel to obtain aligned high-level features of the student. S4. The aligned low-level features of students and the aligned high-level features of students are weighted and fused to obtain fused features. The fused features and the low-level features of students are skipped to obtain connected features. Global features are obtained based on connected features. The connected features are divided into multiple non-overlapping local regions. The local features of each region are concatenated with the global features and attention processing is performed to obtain the channel attention weight of the region. The channel attention weight of each region is used to perform weighted summation of the corresponding local features to generate the enhanced feature map of students. S5. Obtain the internal detection loss of the student detection model; calculate the first distillation loss based on the aligned low-level features of the student and the enhanced feature map of the student; extract classification features and regression features from the enhanced feature map of the student and the multi-scale feature map of the teacher detection model training image, respectively, calculate the cross-correlation statistic, and calculate the second distillation loss based on the cross-correlation statistic; obtain the total distillation loss based on the internal detection loss, the first distillation loss and the second distillation loss. S6. Update the student detection model parameters, repeat S3 to S5 a preset number of times, and select the student detection model with the student detection model parameters that have the minimum total distillation loss as the final student detection model. S7. The final student detection model acquires the image to be detected, extracts multi-scale features, and extracts classification features and regression features on each feature map layer respectively. It then predicts the defect category and bounding box coordinates, obtains the defect category label and confidence score, and completes the insulator defect detection.

2. The insulator defect detection method based on cross-layer knowledge distillation according to claim 1, characterized in that, The cross-correlation statistics in S5 include: the mean and variance of the classification features of the enhanced feature maps of students and the multi-scale feature maps of the teacher detection model training images, the mean and variance of the regression features, and the channel cross-correlation between the classification features and the regression features of the feature maps.

3. The insulator defect detection method based on cross-layer knowledge distillation according to claim 2, characterized in that, The specific steps for calculating the channel cross-correlation between the categorical and regression features of the feature map are as follows: ， in, This represents the cross-correlation of channels, where N is the total number of spatial locations. , These represent the mean of the categorical feature and the mean of the regression feature, respectively. , These are categorical features and regression features, respectively. Let C be the set of all real numbers in dimension C.

4. The insulator defect detection method based on cross-layer knowledge distillation according to claim 2, characterized in that, The variance of the classification features adopts a gradient-based weighting strategy, specifically: the weight of each spatial location is determined based on the gradient norm of the classification feature on the internal detection loss, and then the variance of the classification features is calculated.

5. The insulator defect detection method based on cross-layer knowledge distillation according to claim 1, characterized in that, In S5, the total distillation loss is obtained based on the internal detection loss, the first distillation loss, and the second distillation loss. The specific operation is as follows: The distillation loss is obtained by weighted summation of the first and second distillation losses: ； in, For distillation loss, , This is an empirical coefficient. , These are the first distillation loss and the second distillation loss, respectively. The total distillation loss is obtained by summing the distillation loss and the internal detection loss: ， in, This is an empirical coefficient. This is due to internal detection losses.

6. The insulator defect detection method based on cross-layer knowledge distillation according to claim 1, characterized in that, The pre-training described in S1 specifically involves training the teacher detection model using a detection loss function until convergence, at which point the teacher detection model can output the defect classification and location regression results of the insulator.

7. The insulator defect detection method based on cross-layer knowledge distillation according to claim 1, characterized in that, In S4, after concatenating the local features and global features of each region, attention processing is performed to obtain the channel attention weights of that region. Specifically, after concatenating the local features and global features of each region along the channel dimension, the first fully connected processing, ReLU activation processing, second fully connected processing, and Sigmoid activation processing are performed sequentially to obtain the channel attention weights of that region.

8. The insulator defect detection method based on cross-layer knowledge distillation according to claim 1, characterized in that, S7 also includes performing filtering and non-maximum suppression processing based on confidence thresholds on the obtained defect category labels and confidence scores to obtain insulator defect detection results and complete the insulator defect detection.

9. An insulator defect detection system based on cross-layer knowledge distillation, used to implement the insulator defect detection method based on cross-layer knowledge distillation as described in any one of claims 1-8, characterized in that, include: The model building module constructs a teacher detection model and pre-trains it using images from a dataset of insulator defects; it also constructs a student detection model. In the alignment module, the teacher detection model and the student detection model extract multi-scale feature maps of the same image in the insulator defect dataset to obtain high-level features of the teacher, low-level features of the teacher, high-level features of the student, and low-level features of the student. The low-level features of the student are matched with the low-level features of the teacher in terms of space and channel to obtain aligned low-level features of the student. The high-level features of the student are matched with the high-level features of the teacher in terms of space and channel to obtain aligned high-level features of the student. The image enhancement module performs weighted fusion of aligned low-level student features and aligned high-level student features to obtain fused features. It then skips connections between the fused features and low-level student features to obtain connected features. Based on the connected features, it obtains global features and divides the connected features into multiple non-overlapping local regions. After concatenating the local features of each region with the global features, it performs attention processing to obtain the channel attention weights of that region. Finally, it uses the channel attention weights of each region to perform weighted summation of the corresponding local features to generate the enhanced feature map of the student. The loss calculation module obtains the internal detection loss of the student detection model; calculates the first distillation loss based on the aligned low-level features of the student and the enhanced feature map of the student; extracts classification features and regression features from the enhanced feature map of the student and the multi-scale feature map of the training image of the teacher detection model, respectively, calculates the cross-correlation statistic, and calculates the second distillation loss based on the cross-correlation statistic; and obtains the total distillation loss based on the internal detection loss, the first distillation loss, and the second distillation loss. The update module updates the parameters of the student detection model and selects the student detection model with the parameters that minimize the total distillation loss as the final student detection model. The detection module uses the final student detection model to acquire the image to be detected, extract multi-scale features, and extract classification features and regression features on each feature map layer, respectively, to predict the defect category and bounding box coordinates, obtain the defect category label and confidence score, and complete the insulator defect detection.

10. An insulator defect detection device based on cross-layer knowledge distillation, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the insulator defect detection method based on cross-layer knowledge distillation as described in any one of claims 1-8.

Images