A method for insulator defect detection based on saliency-aware data augmentation and semantic prior transfer

Abstract

This invention discloses an insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer, comprising: constructing a data augmentation model to augment the acquired original insulator image to generate an augmented insulator image; the data augmentation model includes a visual backbone module, an insulator geometric perception enhancement module, a saliency-aware module, and a random destructive augmentation module; constructing a defect diagnosis model and training it using the augmented insulator image; the defect diagnosis model includes a structural prior injection module, a semantic information extraction module, a defect diagnosis module, and a semantic alignment module; during training, the semantic information extraction module is used as the teacher model and the defect diagnosis module is used as the student model for offline distillation training to achieve collaborative transfer; and completing insulator defect detection based on the trained defect diagnosis model. This invention can significantly improve the model's robustness, generalization performance, and efficiency in practical engineering applications.

Description

Technical Field

[0001] This invention belongs to the field of power transmission line testing technology, specifically relating to an insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer. Background Technology

[0002] With the rapid development of computer vision technology, image processing and object recognition have been widely applied in various fields, especially in industrial inspection, security monitoring, and intelligent transportation. Insulator defect detection, as a critical task in power systems, has received increasing attention in recent years. Insulators, as important components in power equipment, directly affect the stability and safety of the power system. Traditional insulator defect detection methods mainly rely on manual inspection and traditional image processing techniques; however, these methods are not only inefficient but also involve high labor costs and significant subjectivity. Therefore, automated defect detection methods based on deep learning have gradually become a research hotspot.

[0003] Deep learning methods have demonstrated tremendous potential in visual recognition tasks, especially convolutional neural networks (CNNs), which have achieved remarkable results in tasks such as image classification and object detection. For example, in existing technology, CN117455923A discloses an insulator defect detection method and system based on a YOLO detector. The method involves collecting and processing aerial photography datasets to obtain processed aerial photography datasets; constructing an initial defect detection model consisting of a backbone feature extraction network, a path aggregation parameterless attention feature pyramid network, and a deformable decoupled detection head; training the initial defect detection model with a target loss function to obtain an optimized defect detection model; and inputting the processed aerial photography dataset into the optimized defect detection model to obtain the defect detection results. However, training deep neural networks typically requires a large amount of labeled data, which is a significant challenge in practical applications. To effectively alleviate the problem of insufficient data, data augmentation techniques have emerged. Data augmentation techniques generate more diverse training data by transforming and perturbing existing training samples, thereby improving the model's generalization ability and robustness.

[0004] Currently, most data augmentation methods do not fully utilize prior information in images (such as target regions and semantic masks), potentially destroying key features related to the target. For insulator defect detection, ensuring the preservation of insulator regions and defect features during augmentation is particularly crucial, as these key regions directly affect detection accuracy. Excessive destruction of these regions during augmentation may prevent the trained model from accurately identifying defects, thus impacting detection performance in practical applications.

[0005] To effectively address this issue, traditional augmentation methods often rely on manually labeled prior knowledge, such as bounding boxes or semantic masks, to mark key regions. However, this approach not only increases annotation costs but also often fails to meet accuracy requirements in practical applications. Therefore, how to introduce reasonable prior knowledge into the data augmentation process to ensure that the augmentation process does not destroy key features, while improving the robustness and accuracy of the model for defect detection, has become a key research focus.

[0006] Therefore, existing technologies urgently need a method that can effectively increase the diversity of data while ensuring that key feature information is preserved during the augmentation process, thereby improving the robustness and generalization ability of the model. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer.

[0008] The objective of this invention can be achieved through the following technical solutions: This invention provides an insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer, comprising the following steps: A data augmentation model is constructed to obtain the original insulator image, and the original insulator image is augmented by the data augmentation model to generate an augmented insulator image; wherein, the data augmentation model includes a visual backbone module, an insulator geometry perception enhancement module, a saliency perception module, and a random destructive augmentation module; A defect diagnosis model is constructed and trained using the augmented insulator image; wherein, the defect diagnosis model includes a structural prior injection module, a semantic information extraction module, a defect diagnosis module, and a semantic alignment module. Defect detection of insulators is achieved based on the trained defect diagnosis model.

[0009] Furthermore, the visual backbone module is an image feature extraction network for feature extraction and classification of insulator images, comprising multiple sequentially connected convolutional processing units; wherein, the visual backbone module is any one of a convolutional neural network, a feature pyramid network, or a graph convolutional network; The processing steps of the visual backbone module include: The input original insulator image is subjected to multi-layer convolution operation and feature mapping to extract the low-level features and high-level semantic features of the original insulator image step by step. The extracted features are downsampled to obtain multi-scale feature representations; Output backbone feature information for fusing with the edge features of the insulator geometry perception enhancement module.

[0010] Furthermore, the insulator geometry perception enhancement module includes an edge feature extraction submodule and a feature fusion submodule; the edge feature extraction submodule is used to extract edge feature information representing the geometric structure of the insulator from the insulator image, and the feature fusion submodule is used to fuse the edge feature information with the backbone feature information output by the visual backbone module; The processing steps of the insulator geometry sensing enhancement module include: Convert the input insulator image from an RGB image to a grayscale image; The grayscale image is processed using the contrast-limited adaptive histogram equalization algorithm CLAHE. Gaussian blur processing is applied to the grayscale image after CLAHE processing, where the Gaussian blur kernel size is 5×5; Edge detection is performed on the grayscale image after Gaussian blurring to obtain the edge feature map. The formula is as follows: in, , These represent the horizontal gradient and the vertical gradient, respectively. Represents a grayscale image. Indicates the convolution operation; Represents the edge feature map; The edge feature map is input into the edge feature extraction submodule, which consists of multiple convolutional layers, for feature extraction. The multiple convolutional layers include two convolutional layers. Each convolutional layer performs a downsampling operation and is connected to a batch normalization layer and an activation function in sequence. The feature fusion submodule splices or fuses the edge features output by the edge feature extraction submodule with the corresponding scale features in the visual backbone module to obtain enhanced backbone feature information.

[0011] Furthermore, the saliency perception module includes a gradient map generation submodule and a protection region generation submodule. The gradient map generation submodule is used to acquire the response gradient map of the insulator image and the defect category. The gradient map generation submodule includes a convolutional layer for feature mapping and gradient calculation. The protection region generation submodule is used to generate an insulator protection region based on the response gradient map. The protection region generation submodule includes a binarization processing layer and a morphological dilation processing layer. The binarization processing layer calculates a binarized map based on the grayscale gradient map, and the morphological dilation processing layer performs a dilation operation on the binarized map to generate the protection region. The processing procedure of the saliency perception module includes: The enhanced backbone feature information output by the visual backbone module is input into the gradient map generation submodule, and the feature gradient is calculated through the convolutional layer to obtain the gradient map. The gradient map is normalized to obtain a grayscale gradient map; The grayscale gradient image is input into the binarization layer, and the grayscale gradient is processed based on the median pixel value of the grayscale image. Generate binarized image The formula is: in, This is a binary image; This is the median value of all pixels in the grayscale gradient image; For the grayscale gradient image at the th Liede The pixel value of the row; , These represent the width and height of the grayscale gradient image, respectively. Binarization image The input morphological dilation processing layer uses a 5×5 square structure as the dilation kernel. Two consecutive morphological dilation operations are performed on the binarized image to generate the insulator protection region. The formula is as follows: in, Indicates the protected area of ​​the insulator; It is a 5×5 square structure; The morphological dilation operation is represented as: in, Indicates pixel position; This indicates moving the center of the structural element to... Place; This represents the empty set.

[0012] Furthermore, the random destructive augmentation module includes a mask generation submodule and a mask filtering submodule; The mask generation submodule is used to generate a random destruction mask based on the Cutout and / or GridMask methods. The mask generation submodule includes: The Cutout submodule is used to generate continuous square destruction regions, and includes two parameters: This represents the proportion of the pixel area of ​​the continuous square region to be randomly destroyed to the total pixel area of ​​the image. This represents the aspect ratio of a continuous rectangular region, and the position of the rectangular region is randomly generated. The GridMask submodule is used to generate mesh destruction areas and includes two parameters: d This represents the pixel side length of the square grid. This indicates the percentage of the grid to be retained; the grid starting point is randomly generated. The mask filtering submodule is used to determine the overlap between the generated mask and the insulator protection area generated by the saliency sensing module, and to remove overlapping masks. The mask filtering submodule includes a five-point overlap determination layer, and its processing formula is as follows: in, Represents a single mask; Indicates the protected area of ​​the insulator; It is a counting function; , , , , These represent the coordinates of the top left, bottom right, and center point of the mask, respectively. The filtered mask set is The formula is: in, Represents the final mask set; This represents the original mask set generated by the mask generation submodule; The random destructive augmentation module processing procedure includes: Step A1: Input the original insulator image into the mask generation submodule, and randomly select either the Cutout submodule or the GridMask submodule to generate the initial mask set; Step A2: Input the generated initial mask set into the mask filtering submodule. Based on the five-point overlap judgment, remove the masks that overlap with the protected area to obtain the final mask set. ; Step A3: For the final mask set The corresponding image region is subjected to random destruction operations, filling the region with random noise or a black background; Step A4: Repeat steps A1 to A3 until the final mask set is reached. If not empty, obtain the augmented insulator image after random destructive augmentation.

[0013] Furthermore, the structural prior injection module includes: Two convolutional layers are used to achieve four-fold downsampling. Each convolutional layer includes a convolution operation, a BatchNorm layer, and a GELU activation function. Three convolutional modules are used to extract downsampled features at 8x, 16x, and 32x respectively. Each convolutional module includes a downsampled convolutional layer and a regular convolutional layer. The structure prior injection module fuses the feature maps extracted by the convolutional layer and the convolutional module with the corresponding scale feature maps of the semantic information extraction module. The feature fusion process adopts a pixel-by-pixel stitching or weighted fusion method. The processing steps of the structure prior injection module include: The input insulator image is downsampled fourfold through a convolutional layer to obtain the bottom-level feature map; The downsampled feature map is normalized through a BatchNorm layer and then non-linearly activated using the GELU activation function. Three convolutional modules are used to extract downsampled features at 8x, 16x, and 32x respectively. The output feature map of each convolutional module is then fused with the feature map at the corresponding scale position of the semantic information extraction module to obtain a fused multi-scale feature map.

[0014] Furthermore, the semantic information extraction module is one of a semantic segmentation network, a residual network, or a generative adversarial network, used to extract high-level semantic information from the insulator image and generate a semantic feature map for defect diagnosis. The processing steps of the semantic information extraction module include: Input an insulator image and extract features from the insulator image using a semantic segmentation network, residual network, or generative adversarial network; The extracted features are subjected to multi-layer convolution operations to gradually obtain the deep semantic information of the image; The extracted deep semantic information is predicted pixel by pixel to generate a high-dimensional semantic feature map that reflects the defect region and semantic region in the image; The high-dimensional semantic feature map is output to the defect diagnosis module and the semantic alignment module for subsequent defect classification and semantic alignment optimization.

[0015] Furthermore, the defect diagnosis module is one of a convolutional neural network, a feature pyramid network, or a graph convolutional network, used to process and classify the high-dimensional semantic feature map extracted by the semantic information extraction module, and to determine the defect type in the insulator image. The processing steps of the defect diagnosis module include: The high-dimensional semantic feature map output by the input semantic information extraction module is used to extract deep features through multiple convolutional layers, and high-level feature maps are obtained step by step. Apply downsampling operations to the extracted high-level feature maps to obtain multi-scale feature representations; Fully connected layers are used to classify defects using multi-scale feature representations, outputting the probability distribution of defect categories, where each category represents a defect type; Output the defect classification results and perform post-processing to generate the final defect detection results.

[0016] Furthermore, the semantic alignment module includes a local detail feature alignment module and a global semantic alignment module, which are used to align and optimize semantic information during the defect diagnosis process.

[0017] Furthermore, the step of training the defect diagnosis model using the augmented insulator image specifically includes: The augmented insulator image is input into the structural prior injection module. The module extracts the low-level feature map through convolution operation and performs non-linear activation through BatchNorm layer normalization and GELU activation function to obtain the feature map after four times downsampling. The feature map after four-fold downsampling is input into the semantic information extraction module, which performs deep feature extraction on the input features to generate a high-dimensional semantic feature map. The high-dimensional semantic feature map output by the semantic information extraction module is input into the defect diagnosis module. The defect diagnosis module performs deep feature extraction and downsampling on the high-dimensional semantic feature map to obtain multi-scale feature representation. In multi-scale feature representation, defect classification is performed through a fully connected layer, outputting the probability distribution of defect categories and generating the final defect detection results; During training, an offline distillation method is used. Knowledge transfer is achieved through a semantic information extraction module as the teacher model and a defect diagnosis module as the student model. The total loss function is calculated based on [reasoning missing], and the formula is: in, For the total loss function, Basic classification loss; This is the global semantic alignment loss; For local detail feature alignment loss; The basic classification loss is formulated as follows: in, The weights are determined by cross-entropy loss. The cross-entropy loss function; The prediction results for the student model; This is a real label; The global semantic alignment loss is calculated as follows: in, , These are the weights for the KL divergence loss and the global feature alignment loss, respectively. Temperature coefficient; Let KL divergence be a metric. and The student model and the teacher model were respectively in the first... i The predicted probability of a class; This is the last layer of feature maps in the teacher model; This is the last layer feature map of the student model; For global semantic extractors; , and These represent three different 1×1 convolutions; This represents Layer Normalization; This represents the GELU activation function; This represents the total number of spatial locations in the feature map; The local detail feature alignment loss is calculated as follows: in, , These are the weights for local feature alignment loss and attention alignment loss, respectively. For local feature alignment loss; This is for attention alignment loss; , and Input the length, width, and number of channels of the feature map respectively; , Teacher model and student model in the channel respectively k ,Location eigenvalues; For teacher models in the channel k Channel attention weights; For the teacher model in spatial location Spatial attention weights; This is for attention alignment loss; , These are spatial attention maps for the teacher model and the student model, respectively. These are the channel attention vectors for the teacher model and the student model, respectively; , These are spatial attention alignment loss and channel attention loss, respectively. , The spatial attention maps of the teacher model and the student model are respectively represented at position. eigenvalues; , These represent the channel attention vectors of the teacher model and the student model, respectively, in the channel. The value; By minimizing the total loss function, the parameters of the defect diagnosis model are updated, enabling knowledge transfer between the teacher model and the student model.

[0018] Compared with the prior art, the present invention has the following advantages: (1) In the prior art, although traditional image augmentation methods can improve the robustness and generalization ability of the model by performing random destructive augmentation (such as occlusion, noise injection, etc.) on the image, these methods do not fully consider how to preserve the key target region features during the augmentation process. Without prior knowledge, the augmentation process may destroy important features in the image, making it difficult for the model to identify the target region. Especially in the insulator defect detection task, the loss of key features will directly affect the accuracy of the detection results. To avoid this problem, this invention constructs a saliency-aware data augmentation model and combines it with semantic prior transfer technology, which can reasonably preserve the feature information of key regions during the augmentation process, while enriching the diversity of data and significantly improving the generalization ability of the model.

[0019] (2) Existing data augmentation methods often rely on a large amount of manually annotated prior information (such as bounding boxes or semantic masks) to ensure that the target region is not destroyed during the augmentation process. However, manual annotation is not only time-consuming and costly, but the annotation accuracy is often limited by human experience, making it difficult to achieve perfect consistency, which further affects the augmentation effect and model performance. To solve this problem, this invention proposes an augmentation scheme based on a saliency-aware module and a structural prior injection module. By automatically generating protected regions and guiding the augmentation process, it avoids the destruction of key regions in traditional methods. The saliency-aware module achieves accurate protection of key regions through gradient map generation and protected region generation sub-modules, thereby reducing the need for manual annotation, lowering annotation costs, and avoiding the impact of insufficient annotation accuracy, thus ensuring the quality and diversity of augmented data.

[0020] (3) Traditional defect diagnosis models typically rely on manually designed feature extraction methods or global feature learning using traditional convolutional neural networks (CNNs). However, these methods have a weak understanding of the semantic information and structural priors of images, and easily overlook important local features. To overcome this problem, this invention designs a semantic information extraction module that utilizes powerful shape perception capabilities to accurately extract high-level semantic information (such as defect regions and insulator morphology) from images and input it into the defect diagnosis model. Guided by semantic priors, the defect diagnosis model can more accurately locate and classify defects, avoiding the limitations of traditional methods lacking semantic guidance, thereby improving the accuracy and precision of defect detection.

[0021] (4) Existing deep learning models often face problems such as complex model architecture, large computational load, and long training time during the training process. Especially when using traditional knowledge transfer methods, it is necessary to deal with the cumbersome process of obtaining and injecting structural priors, which not only increases the computational burden but also reduces the efficiency of model training. In order to simplify this process, this invention introduces a distillation training method based on the semantic information extraction module and the defect diagnosis module, thus eliminating the complex structural prior injection link in the traditional method. By using an offline distillation method for knowledge transfer, efficient feature alignment and knowledge transfer are achieved between the teacher model (semantic information extraction module) and the student model (defect diagnosis module), thereby simplifying the defect diagnosis process and improving training efficiency and model operability.

[0022] (5) Existing defect detection methods typically only consider the global features of an image, neglecting the importance of local details. For insulator defect detection, the local features of defects are often more complex and highly variable, making it difficult for traditional methods to effectively capture these detailed features. To address this issue, this invention designs a local detail feature alignment module to optimize feature extraction in local regions. By introducing a local feature alignment loss function based on an attention mechanism, the model can focus on the detailed information of key local regions, effectively improving the defect recognition capability. Furthermore, by optimizing spatial attention and channel attention, the model's detail recognition capability is further enhanced, enabling it to more accurately identify insulator defects in practical applications. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the semantic prior-based random destructive augmentation method for insulators according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the edge extraction effect of the edge detection algorithm in an embodiment of the present invention; Figure 3 These are schematic diagrams illustrating three different network structure schemes according to embodiments of the present invention; Figure 4 This is a schematic diagram of a defect diagnosis model based on semantic priors according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the global semantic alignment module in an embodiment of the present invention; Figure 6 A schematic diagram illustrating the impact of the number of generative defect samples on the performance of the defect diagnosis model in an embodiment of the present invention; Figure 7 This is a visual comparison diagram of the methods proposed in the embodiments of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] Example 1: like Figures 1-7 As shown, this embodiment provides an insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer, including: A data augmentation model is constructed to augment insulator data images. The data augmentation model includes a visual backbone module, an insulator geometry perception enhancement module, a saliency perception module, and a random destructive augmentation module. The visual backbone module is used to classify insulator images. The insulator geometry perception enhancement module uses an edge detection algorithm to extract edge feature maps for input. The saliency perception module is used to determine the protected area in the insulator image. The random destructive augmentation module removes parts that overlap with the protected area when generating the mask, generating an augmented dataset. A defect diagnosis model is constructed and trained using augmented data to detect defects in insulators in insulator images. The defect diagnosis model includes a structure prior injection module, a semantic information extraction module, a defect diagnosis module, and a semantic alignment module. The structure prior injection module provides the shape features of the insulator, the semantic information extraction module extracts the semantics from the insulator image, the defect diagnosis module identifies the defects in the insulator image, and the semantic alignment module aligns the weighted features of the channel and spatial attention. The structure of the semantic information extraction module and the defect diagnosis module are then fused to output the defect features of the insulator.

[0026] In this invention, new insulator data images are generated by constructing a data augmentation model, effectively expanding the dataset size and increasing data diversity. Richer data helps improve the model's adaptability to different scenarios and insulator morphologies, avoiding overfitting due to limited data and thus improving the model's generalization performance. The defect diagnosis model of this invention transfers the powerful shape-aware capabilities of the semantic information extraction module to the defect diagnosis model. Under effective semantic guidance, the defect diagnosis model can improve the utilization rate of generative defect augmentation samples, promoting improved defect diagnosis performance. Simultaneously, it eliminates the cumbersome structure prior acquisition and injection steps, simplifying the defect diagnosis process and enhancing the efficiency and operability of the defect diagnosis model in practical applications.

[0027] Data augmentation models such as Figure 1 As shown, the entire framework consists of three parts: a visual backbone module, an insulator geometry perception enhancement module, a saliency perception module, and a random destructive augmentation module.

[0028] The visual backbone module uses standard image classification networks, such as convolutional neural networks, feature pyramid networks, and graph convolutional networks.

[0029] The present invention relates to an insulator geometry perception enhancement module, which utilizes an edge detection algorithm to extract edge feature maps for input to the geometry perception enhancement module.

[0030] The edge extraction algorithm is shown in Algorithm 1. Figure 2 As shown, the first row is the original image; the second row is the Sobel grayscale edge feature map. First, the CLAHE algorithm (Contrast-Limited Adaptive Histogram Equalization) is used to adaptively enhance the contrast of the grayscale image by calculating the local histogram. Then, Gaussian blur is used to suppress environmental noise. Finally, an edge detection algorithm is applied to extract edge features. The edge extraction algorithm is represented as follows: in, , These represent the horizontal gradient and the vertical gradient, respectively. Represents a grayscale image. Indicates the convolution operation; It represents the edge feature map; the edge extraction algorithm provides detailed characterization. It can extract richer edge feature information from the insulator edges.

[0031] This invention presents an insulator geometry perception enhancement module. Instead of directly concatenating the grayscale image of the insulator edge with an RGB image and inputting it into the model, this module only accepts the edge grayscale image as input and extracts edge features. The extracted edge features are then concatenated with the corresponding backbone features to continuously enhance the edge information at different locations within the visual backbone module.

[0032] like Figure 3 As shown, from left to right, these are schemes 1, 2, and 3, illustrating three different network structure schemes for the insulator geometry perception enhancement module. The main difference lies in the connection position and method between edge features and features from the visual backbone module. Scheme 1 concatenates features from different positions in the edge feature branch with features from corresponding positions in the visual backbone module; Scheme 2 concatenates the output features of the edge feature branch with the output features of the penultimate convolutional block of the visual backbone module; Scheme 3 directly concatenates the edge features with the input RGB features.

[0033] The multi-layer convolution in the insulator geometry perception enhancement module consists of two convolutional layers. Each convolutional layer performs a downsampling task and is followed by a BatchNorm layer and the ReLU activation function. The convolutional blocks are set in different vision backbone modules. For example, ResNet has four convolutional blocks, and ResNet18 has four convolutional blocks, each containing a combination of two convolutional layers, a BatchNorm layer, and a ReLU function.

[0034] The saliency perception module can obtain the model's response gradient map to the insulator image and defect category. After normalization, a grayscale gradient map can be formed. However, at this point, the grayscale map cannot be directly used to identify the area that needs protection. The saliency perception module obtains the gradient of the feature map of the last convolutional layer to form a grayscale map, but because deep neural networks compress spatial resolution through layer-by-layer downsampling (such as pooling), the gradient map size is much smaller than the input image (e.g., the input resolution of common mainstream visual backbones such as ResNet is 224×224, and after 32 times downsampling, the resolution of the last layer feature map is compressed to 7×7). Therefore, this grayscale map is difficult to accurately locate fine-grained areas. At the same time, when upsampling the low-resolution grayscale image and interpolating it back to the original image size, there are geometric distortions and blurring effects, especially in the target edge area, which will produce a large deviation. At this time, the grayscale image needs to be processed to obtain a more accurate protection area and augmentable area. Therefore, appropriate transformations, such as morphological dilation, are needed to convert the grayscale image into a binary image, as shown in the following formula: in, This is a binary image; This is the median value of all pixels in the grayscale gradient image; For the grayscale gradient image at the th Liede The pixel value of the row; , These represent the width and height of the grayscale gradient image, respectively. To avoid the influence of complex parameter adjustments on the experimental results, the median value of all pixels in the grayscale image was selected as the median. Subsequently, morphological dilation is applied to the binarized image to obtain the final protected region. Morphological dilation can be defined by the following formula: in, Indicates pixel position; This indicates moving the center of the structural element to... Place; This represents the empty set.

[0035] The dilation operation involves continuously sliding a structuring element across the image. When the structuring element overlaps with the foreground in the binarized image, the area between the contact center and the foreground is designated as the foreground. The structuring element is typically a continuous square structure; here, a 5×5 square structure is chosen as the dilation kernel. This allows for a wider protected area, thus reducing the risk of damage to critical regions. The protected area is obtained as shown in the following formula: in, Indicates the protected area of ​​the insulator; It is a 5×5 square structure.

[0036] The random destructive augmentation module uses Cutout and / or GridMask as the random destructive augmentation method, and selects them randomly during the augmentation process. Its specific operation is as follows: (1) Cutout requires two parameters to be set. This represents the proportion of the pixel area of ​​the continuous square region to be randomly destroyed to the total pixel area of ​​the image. This represents the aspect ratio of a continuous rectangular region. The positions of the continuous rectangles are randomly generated and discarded when they exceed the boundaries.

[0037] (2) GridMask requires two parameters to be set. d This represents the pixel side length of the square grid. Indicates the percentage of mesh retention, i.e. This represents the ratio of the side length of the square region to be randomly destroyed to the side length of the square grid. The grid starting point is randomly generated.

[0038] Each augmentation of the above parameters is randomly generated from a uniform distribution, and the upper and lower bounds of different parameters can be obtained from [the above].

[0039] Table 1 Hyperparameters used for Cutout and GridMask augmentations After Cutout or GridMask, you will get a mask or a list of masks. Each mask represents a square area of ​​the mask with its top-left and bottom-right coordinates. Therefore, the four boundary points of the mask can be obtained. , , , and its center point Therefore, the overlap between the mask and the protected area is determined by these five points. If at least three of the five points are within the protected area, it is considered an overlap, as expressed by the following formula: in, Represents a single mask; Indicates the protected area of ​​the insulator; It is a counting function; , , , , These represent the coordinates of the top-left, bottom-right, and center points of the mask, respectively; then, masks that overlap with the protected area are removed according to the following formula: in, Represents the final mask set; This represents the original mask set generated by the mask generation submodule; repeat the above process until... If the value is not empty, then random noise or other content is filled into the mask area of ​​the original image, as shown in Algorithm 2, as shown in the black background. The proposed semantic prior-based random destructive augmentation technique is reusable. Since this method relies on a pre-trained model to provide a semantic prior (i.e., the model's response map or saliency map to the defect region) to generate a protected region and guide the semantic preservation of the augmentation process, the accuracy of the semantic prior needs to be guaranteed. By iterating this process, the semantic prior provided by a new model trained on the augmented image becomes more accurate, thereby improving the accuracy of the protected region and promoting the training of subsequent models. Furthermore, the semantic prior can be provided not only by pre-training for insulator defect diagnosis tasks but also by pre-trained models for general scenarios.

[0040] This invention also proposes a defect diagnosis model based on semantic prior driving, such as... Figure 4As shown, a distillation technique is used to transfer the powerful shape perception capability acquired from structural priors in the semantic information extraction module to the defect diagnosis model, and generative defect augmentation samples are combined to improve the overall performance of the defect diagnosis model. This defect diagnosis model includes a structural prior injection module, a semantic information extraction module, a defect diagnosis module, and a semantic alignment module. The semantic alignment module constructs a semantic prior transfer channel, which can be divided into two parts: a local detail feature alignment module and a global semantic alignment module. The local detail feature alignment module uses an attention mechanism to provide channel importance and spatial importance, and achieves key semantic feature selection and alignment by assigning weights to local detail features, enabling the defect diagnosis model to better learn the powerful shape perception capability derived from structural priors in the semantic information extraction module. The global semantic alignment module achieves fine-grained global semantic alignment through the alignment of global semantic information, improving the overall alignment effect. The introduction of the semantic alignment module not only realizes the transfer of perception capabilities from the semantic information extraction module, but also effectively suppresses the label flipping problem of generative defect samples, promoting the utilization rate of generative defect samples.

[0041] The structural prior injection module primarily utilizes low-level visual cues to provide insulator shape features for the defect diagnosis model, reducing noise from complex backgrounds and interference from diverse insulator textures, thus enabling efficient detection of morphological differences and defect diagnosis. Therefore, using structural priors is a reasonable choice to address the complex high-frequency perturbations that may exist in the generated images.

[0042] The structure prior injection module achieves 4x downsampling through two convolutional layers. Each convolutional layer contains a convolution operation, a BatchNorm layer, and a GELU activation function. Subsequently, three convolutional modules extract downsampled features at 8x, 16x, and 32x respectively, and fuse them with the corresponding scale feature maps from the semantic information extraction module. Each convolutional module contains one downsampled convolutional layer and one regular convolutional layer.

[0043] The semantic information extraction module can be a semantic segmentation network, residual network, or generative adversarial network, etc. The defect diagnosis module can be a convolutional neural network, feature pyramid network, or graph convolutional network, etc. Depending on how the teacher model (semantic information extraction module) is acquired, it can be offline distillation, online distillation, or self-distillation. Offline distillation requires an additional pre-trained model as the teacher model to provide knowledge. Online distillation focuses on simultaneously completing the learning and knowledge transfer between the teacher model and the student model (defect diagnosis module) in end-to-end optimization. This invention adopts offline distillation. This simplifies the operation process, avoids complex model design, and makes the entire framework more concise. The standard image classification model is not only simple in structure and easy to operate, but it can also be better integrated with existing diagnostic frameworks, providing a highly alternative defect diagnosis performance improvement solution.

[0044] Knowledge distillation methods can be categorized into three types based on their alignment methods: response-level, feature-level, and relation-level. Response-level distillation often achieves knowledge transfer by aligning the output probabilities of the teacher and student models. Its core lies in extracting class probability labels through soft labels, thereby achieving semantically informative label smoothing regularization. Feature-level distillation focuses on exploring intermediate-layer features and providing comprehensive supervision. It calculates the loss by computing the difference between teacher and student features to achieve intermediate-layer feature alignment. Both methods have their advantages. In insulator defect diagnosis tasks, local detail features play a crucial role. Designing an alignment module that better matches local detail features is essential for transferring the powerful shape-aware capabilities of the structure-prior-based semantic information extraction module to the defect diagnosis model. The core is to address the loss of local detail features in global alignment methods, achieving accurate alignment at the region level or even pixel level by focusing on the spatial correlation of local features. Feature-level distillation can be modeled as follows: in, , and Representing the input features respectively The number of channels, feature map width, and length. and These represent teacher characteristics and student characteristics, respectively. This section only demonstrates the loss calculation method based on Mean Square Error (MSE), but Mean Absolute Error (MAE) can also be used as the loss.

[0045] This simple error loss method directly constrains the pixel-by-pixel differences between student and teacher features to achieve knowledge transfer. It has the advantages of simple computation and stable gradients, but all features are treated equally, lacking attention to key features. Furthermore, it easily overlooks the relationships between local structures; key edge texture features and other high-frequency signals are smoothed due to the averaging tendency of MSE or MAE, resulting in information loss. Simultaneously, the student model mechanically replicates the local patterns of the teacher features, losing the generalization flexibility to cope with input perturbations (such as occlusion and lighting changes). Especially in generated defective images, there are many invisible high-frequency perturbations. If the teacher model overfits to these high-frequency signals, the student may inherit this noise sensitivity, leading to performance degradation.

[0046] Therefore, this invention proposes a local detail feature alignment module. First, in the channel dimension, alignment through channel attention enhances the guiding role of the teacher network's key semantic channels on the student network. Second, in the spatial dimension, a dynamic region focusing mechanism is adopted, utilizing spatial attention to learn discriminative regions in the feature map, providing weighted spatial information for pixel-level fine-grained feature alignment.

[0047] First, spatial dimension global average pooling (GAP) is used to pool the input features. Convert to Then, channel normalization is achieved using Softmax, and finally, channel attention is generated by scaling based on the number of channels. It can be expressed as the following formula: in, Indicates global average pooling; Indicates channel attention; Indicates the activation function; Represents the input feature map Middle position The first step is to extract the feature values ​​at the specified location; secondly, global max pooling along the channel dimension is used to extract the input features. Convert to Then, spatial normalization is achieved using Softmax (the spatial dimensions need to be flattened first, normalized, and then unfolded again). Finally, spatial attention is generated by scaling using length and width. It can be expressed as the following formula: in, Represents input features; Indicates input features in channels eigenvectors; This indicates that a global average pooling operation is performed along the channel dimension; Indicates spatial attention; After obtaining channel attention and spatial attention, they are used as weights to calculate local detail feature alignment, which can be expressed by the following formula: in, and These represent teacher characteristics and student characteristics, respectively. , Teacher model and student model in the channel respectively k ,Location eigenvalues; For teacher models in the channel k Channel attention weights; For the teacher model in spatial location Spatial attention weights; Since there are significant differences between student and teacher characteristics during the learning process, the spatial attention and channel attention acquired by the two also differ. Here, the teacher's spatial attention and channel attention are weighted and calculated to implicitly encourage the student model's spatial attention and channel attention to approach those of the teacher model.

[0048] Simultaneously, explicit supervision can be used to narrow the spatial and channel attention of the teacher and student models by calculating the mean variance error of their attention separately. This allows us to obtain the difference between the two models, thereby explicitly promoting the convergence of their learning patterns. The calculation method is expressed by the following formula: in, This is for attention alignment loss; , These are spatial attention maps for the teacher model and the student model, respectively. These are the channel attention vectors for the teacher model and the student model, respectively; , These are spatial attention alignment loss and channel attention loss, respectively. , The spatial attention maps of the teacher model and the student model are respectively represented at position. eigenvalues; , These represent the channel attention vectors of the teacher model and the student model, respectively, in the channel. The value; In summary, the overall training loss of the local detail feature alignment module This can be expressed by the following formula, which includes attention-based local feature alignment loss. Loss of attention alignment , and These represent the weights of the summation of their losses: The local detail feature alignment module can effectively improve the detail extraction performance of the intermediate layers of the student model. However, it should be noted that global features are also important for insulator defect diagnosis tasks, and the local detail feature alignment module severs the connection between different blocks, weakening the global features.

[0049] Therefore, this invention designs a distillation module based on global semantic alignment. The global semantic alignment module aims to capture the differences in global semantic information between the student model and the teacher model, reducing the impact of high-frequency perturbations in local detail features, thereby achieving more efficient knowledge transfer. The global semantic alignment module integrates feature-level alignment and response-level alignment, thus enabling rich knowledge transfer.

[0050] The structure of the global semantic alignment module is as follows: Figure 5 As shown. It contains a shared classifier and a global feature extractor, and uses the last layer of features from the teacher model. and the last layer features of the student model As input.

[0051] First is the classifier, which maps features to probabilities of different classification classes. The classifier uses a shared classifier and simultaneously accepts... and A shared classifier forces student features to be compatible with teacher features, reducing distribution shift caused by classifier training bias and ensuring that student and teacher feature extractors receive the same gradient signals, thus aligning the learning direction. In this case, the shared classifier implicitly guides the student feature extractor to the teacher feature space. Independent classifiers are used for gradient updates of the teacher and student models, respectively. Let... Indicates the prediction result. Representing the truth labels allows us to obtain the cross-entropy loss. , The base loss represents the weight of the loss. It can be expressed as the following formula: in, The weights are determined by cross-entropy loss. The cross-entropy loss function; The prediction results for the student model; This is a real label; The global feature extractor significantly enhances feature representation by aggregating global semantic features from feature maps, thereby optimizing the global semantic-level feature distillation effect. Its core lies in extracting global contextual information from the teacher and student models and reducing their differences in the global semantic space through feature alignment loss. Specifically, this module first processes the input teacher features... and student characteristics Global semantic enhancement is performed by dynamically modeling spatial and channel correlations within feature maps using spatial and channel attention mechanisms, further aggregating global semantic information across regions and channels. This not only preserves high-order semantics about categories but also effectively compensates for the lack of deep semantic connections in the student model due to capacity limitations by implicitly encoding spatial contextual relationships. Finally, by calculating the feature-level alignment loss (MSE can be used), the global feature distribution of the student model is forced to converge towards the teacher's feature space.

[0052] Loss function of global feature alignment module It can be composed of the following formula, including the response-level distillation loss. Alignment loss with global features , and These represent the weights of the sum of their losses. Calculated from KL divergence, let Denotes KL divergence, This indicates the prediction results of the defect diagnosis model. This indicates the prediction result from the semantic information extraction module. Indicates the temperature coefficient: in, Indicates input features, Represents the global semantic extractor. , and These represent three different 1×1 convolutions. This represents Layer Normalization. This represents the GELU activation function.

[0053] In summary, after combining the local detail feature alignment module and the global semantic alignment module, the training loss function of the entire distillation framework can be expressed as follows: in, This is the total loss function.

[0054] Example 2: This embodiment designs numerical experiments to verify the performance of the defect diagnosis model based on semantic priors.

[0055] Datasets: The original dataset contains 1209 images of defect-free insulators and 383 images of spontaneously exploding insulators. The image ratio of the training set to the test set is set to 1:1, meaning 604 images of defect-free insulators and 191 images of spontaneously exploding insulators are used for training, and the remaining images are used for testing. The hybrid dataset includes the original dataset and 200 generative defect samples; all generative samples are added to the training set.

[0056] Experimental Setup: ResNet-18 was selected as the experimental platform. Random horizontal and vertical flip augmentation with a probability of 0.5 was used, followed by ColorJitter with a probability of 0.5 (brightness, contrast, saturation, and hue parameters were all set to 0.1). All images were resized to 256×256 resolution. The learning rate was multiplied by 0.95 per epoch. The initial learning rate was set to 0.1, SGD was selected as the optimizer with a momentum of 0.9 and a weight decay factor of 1e-4. The batch size was set to 64, with a total of 100 training epochs. The WarmUp batch size was set to 10.

[0057] Loss function hyperparameter settings: Set to 1.0, Set to 2.0. Set to 1.0, Set to 1.0, Set to 1.0, temperature coefficient t Set to 5.0.

[0058] The overall experimental results are shown in Table 2. As can be seen from the table, the semantic information extraction module achieves a significant performance improvement with the help of structural priors and the feature fusion module based on global-local modeling. With the help of the proposed local detail feature alignment module and global semantic alignment module, the baseline model can learn the perceptual capabilities of the semantic information extraction module, thus achieving a substantial performance improvement even without structural priors and with only the baseline backbone network, successfully reaching performance similar to the semantic information extraction module. Furthermore, with the help of generated defect images, the baseline model significantly outperforms the semantic information extraction module. This fully demonstrates that generated defect images, guided by the semantic information extraction module, can provide additional information for the defect diagnosis model. Although these generated defect images are derived from real defect images, the inherent high-frequency noise perturbations in the generation pattern can help the model perform regularization, thereby achieving the effect of data augmentation.

[0059] Table 2 Overall Experimental Results To investigate the impact of generated defective samples, this invention designed an erosion experiment based on these samples. Fifty, 100, 150, and 200 generated defective images were used as the research objects. The experimental results are as follows: Figure 6 As shown in Table 3.

[0060] As shown in the table, generative defect samples can improve the performance of different methods to a certain extent. The graph shows that, due to the semantic information extraction module, generative augmentation of defects of varying numbers achieves stable performance improvements, without exhibiting the initial rise followed by a decline curve seen in the baseline model. The baseline model's curve shows a clear performance inflection point, caused by the inherent, invisible high-frequency perturbations in the generative pattern. These perturbations disrupt the feature extraction pattern of convolutional neural networks, which are sensitive to high-frequency signals, leading to a shift in the latent distribution of the training set. When the number of generative defect augmentations is small, the distribution shift remains manageable and can even improve model performance due to the regularization effect brought about by the high-frequency perturbations. However, as the number of generative defect augmentations increases, exceeding a certain critical point, this distribution shift becomes uncontrollable. The model overfits to this unconventional distribution, reducing its focus on the true latent distribution, resulting in a decline in performance on the real defect validation set. The semantic information extraction module introduces semantic information into the generative defect samples, providing semantic reference for the training process of the defect diagnosis model. In this process, the effective semantic information contained in the generative defect samples is correctly captured by the semantic information extraction module and fed back to the defect diagnosis model. This preserves the defect semantic information with perturbations contained in the generative defects while avoiding interference from high-frequency perturbations in the background, thus keeping the impact of such high-frequency perturbations under control. Due to the good performance of the semantic information extraction module, the improvement brought by this method exhibits a significant marginal effect. Nevertheless, as the number of generative defects increases, the defect diagnosis model no longer shows a performance decline, i.e., no significant inflection point appears. This indicates that, with the help of the semantic information extraction module, the defect diagnosis model possesses a level of resistance to high-frequency perturbations in generative defects that the baseline model lacks.

[0061] Table 3. Erosion experiment results augmented with 200 generative defect samples. To explore the impact of different components, this invention designed an erosion experiment, where component A represents the global semantic feature alignment module, component B represents the local detail feature alignment module, and component C represents the shared classifier. To avoid the influence of the generated image distribution, the erosion experiment was conducted only on the original dataset. The results of the erosion experiment are shown in Table 4. It is clear from Table 4 that different components can provide varying degrees of performance improvement to the model. The local detail feature alignment module contributes the most to the overall performance, exceeding the global semantic alignment module. This may be because the local detail feature alignment module plays a greater role in the transfer of shape perception ability from the semantic information extraction module to the defect diagnosis model. Effective supervision of intermediate features makes this shape perception ability transfer smoother. In contrast, the global semantic alignment module provides an alignment method based on global information, which may not be able to capture fine-grained information, thus performing worse than the local detail feature alignment module in practical applications. The shared classifier provides a slight improvement to the baseline model, increasing the F1 score of the defect class by 1.82. This performance improvement is attributed to the classifier from the semantic information extraction module implicitly promoting the features of the defect diagnosis model to be closer to the features of the semantic information extraction module.

[0062] Table 4. Erosion test results for each component To more vividly demonstrate the superiority of the proposed generative defect augmentation framework based on semantic prior guidance, this invention presents a comparative visualization of model interest regions based on Grad-CAM (Visual Model). ResNet-18 is selected as the test object, and the output of its second residual block is chosen as the visualization object to showcase finer-grained feature differences. Figure 7 As shown.

[0063] from Figure 7 As can be seen, the proposed method significantly improves the model's ability to focus on the overall structural shape of the insulator. This shape perception capability originates from the semantic information extraction module, which enhances its shape perception capability by introducing a saliency map prior. This prior-based perception capability is successfully transferred to the prior-free defect diagnosis model using the method of this invention. Although the generated defect images do not show significant differences in insulator morphology, their transformation of the background and imperceptible noise provide the model with additional regularization information. Combined with the shape perception capability based on structural priors, this enhances the model's ability to capture defects and reduce background interference. The figures show that the baseline model's area of ​​interest is more concentrated, distributed in a point-like pattern, and also exhibits some response to the areas of towers and connecting triangles. The semantic information extraction module and the proposed method, to a certain extent, focus more on the shape features of the insulator, with feature extraction basically covering the entire insulator and removing as much background interference as possible.

[0064] This invention studies a defect diagnosis model based on semantic prior guidance. By distilling local detailed features and global semantic features, the powerful shape-aware capabilities of the semantic information extraction module are transferred to the defect diagnosis model. Under effective semantic guidance, the defect diagnosis model can improve the utilization rate of generative defect augmentation samples, thereby enhancing defect diagnosis performance. Simultaneously, this framework allows the defect diagnosis model to eliminate the cumbersome structural prior acquisition and injection steps, simplifying the defect diagnosis process and enhancing the efficiency and operability of the defect diagnosis model in practical applications.

[0065] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting insulator defects based on saliency-aware data augmentation and semantic prior transfer, characterized in that, Includes the following steps: A data augmentation model is constructed to obtain the original insulator image, and the original insulator image is augmented by the data augmentation model to generate an augmented insulator image; wherein, the data augmentation model includes a visual backbone module, an insulator geometry perception enhancement module, a saliency perception module, and a random destructive augmentation module; A defect diagnosis model is constructed and trained using the augmented insulator image; wherein, the defect diagnosis model includes a structural prior injection module, a semantic information extraction module, a defect diagnosis module, and a semantic alignment module. Defect detection of insulators is achieved based on the trained defect diagnosis model.

2. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The visual backbone module is an image feature extraction network used for feature extraction and classification of insulator images, including multiple sequentially connected convolutional processing units; wherein, the visual backbone module is any one of a convolutional neural network, a feature pyramid network, or a graph convolutional network; The processing steps of the visual backbone module include: The input original insulator image is subjected to multi-layer convolution operation and feature mapping to extract the low-level features and high-level semantic features of the original insulator image step by step. The extracted features are downsampled to obtain multi-scale feature representations; Output backbone feature information for fusing with the edge features of the insulator geometry perception enhancement module.

3. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The insulator geometry perception enhancement module includes an edge feature extraction submodule and a feature fusion submodule; the edge feature extraction submodule is used to extract edge feature information representing the geometric structure of the insulator from the insulator image, and the feature fusion submodule is used to fuse the edge feature information with the backbone feature information output by the visual backbone module; The processing steps of the insulator geometry sensing enhancement module include: Convert the input insulator image from an RGB image to a grayscale image; The grayscale image is processed using the contrast-limited adaptive histogram equalization algorithm CLAHE. Gaussian blur processing is applied to the grayscale image after CLAHE processing, where the Gaussian blur kernel size is 5×5; Edge detection is performed on the grayscale image after Gaussian blurring to obtain the edge feature map. The formula is as follows: in, , These represent the horizontal gradient and the vertical gradient, respectively. Represents a grayscale image. Indicates the convolution operation; Represents the edge feature map; The edge feature map is input into the edge feature extraction submodule, which consists of multiple convolutional layers, for feature extraction. The multiple convolutional layers include two convolutional layers. Each convolutional layer performs a downsampling operation and is connected to a batch normalization layer and an activation function in sequence. The feature fusion submodule splices or fuses the edge features output by the edge feature extraction submodule with the corresponding scale features in the visual backbone module to obtain enhanced backbone feature information.

4. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The saliency perception module includes a gradient map generation submodule and a protection region generation submodule. The gradient map generation submodule is used to obtain the response gradient map of the insulator image and the defect category. The gradient map generation submodule includes a convolutional layer for feature mapping and gradient calculation. The protection region generation submodule is used to generate the insulator protection region based on the response gradient map. The protection region generation submodule includes a binarization processing layer and a morphological dilation processing layer. The binarization processing layer calculates the binarization map based on the grayscale gradient map, and the morphological dilation processing layer performs a dilation operation on the binarization map to generate the protection region. The processing procedure of the saliency perception module includes: The enhanced backbone feature information output by the visual backbone module is input into the gradient map generation submodule, and the feature gradient is calculated through the convolutional layer to obtain the gradient map. The gradient map is normalized to obtain a grayscale gradient map; The grayscale gradient image is input into the binarization layer, and the grayscale gradient is processed based on the median pixel value of the grayscale image. Generate binarized image The formula is: in, This is a binary image; This is the median value of all pixels in the grayscale gradient image; For the grayscale gradient image at the th Liede The pixel value of the row; , These represent the width and height of the grayscale gradient image, respectively. Binarization image The input morphological dilation processing layer uses a 5×5 square structure as the dilation kernel. Two consecutive morphological dilation operations are performed on the binarized image to generate the insulator protection region. The formula is as follows: in, Indicates the protected area of ​​the insulator; It is a 5×5 square structure; The morphological dilation operation is represented as: in, Indicates pixel position; This indicates moving the center of the structural element to... Place; This represents the empty set.

5. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The random destructive augmentation module includes a mask generation submodule and a mask filtering submodule; The mask generation submodule is used to generate a random destruction mask based on the Cutout and / or GridMask methods. The mask generation submodule includes: The Cutout submodule is used to generate continuous square destruction regions, and includes two parameters: This represents the proportion of the pixel area of ​​the continuous square region to be randomly destroyed to the total pixel area of ​​the image. This represents the aspect ratio of a continuous rectangular region, and the position of the rectangular region is randomly generated. The GridMask submodule is used to generate mesh destruction areas and includes two parameters: d This represents the pixel side length of the square grid. This indicates the percentage of the grid to be retained; the grid starting point is randomly generated. The mask filtering submodule is used to determine the overlap between the generated mask and the insulator protection area generated by the saliency sensing module, and to remove overlapping masks. The mask filtering submodule includes a five-point overlap determination layer, and its processing formula is as follows: in, Represents a single mask; Indicates the protected area of ​​the insulator; It is a counting function; , , , , These represent the coordinates of the top left, bottom right, and center point of the mask, respectively. The filtered mask set is The formula is: in, Represents the final mask set; This represents the original mask set generated by the mask generation submodule; The random destructive augmentation module processing procedure includes: Step A1: Input the original insulator image into the mask generation submodule, and randomly select either the Cutout submodule or the GridMask submodule to generate the initial mask set; Step A2: Input the generated initial mask set into the mask filtering submodule. Based on the five-point overlap judgment, remove the masks that overlap with the protected area to obtain the final mask set. ; Step A3: For the final mask set The corresponding image region is subjected to random destruction operations, filling the region with random noise or a black background; Step A4: Repeat steps A1 to A3 until the final mask set is reached. If not empty, obtain the augmented insulator image after random destructive augmentation.

6. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The structure prior injection module includes: Two convolutional layers are used to achieve four-fold downsampling. Each convolutional layer includes a convolution operation, a BatchNorm layer, and a GELU activation function. Three convolutional modules are used to extract downsampled features at 8x, 16x, and 32x respectively. Each convolutional module includes a downsampled convolutional layer and a regular convolutional layer. The structure prior injection module fuses the feature maps extracted by the convolutional layer and the convolutional module with the corresponding scale feature maps of the semantic information extraction module. The feature fusion process adopts a pixel-by-pixel stitching or weighted fusion method. The processing steps of the structure prior injection module include: The input insulator image is downsampled fourfold through a convolutional layer to obtain the bottom-level feature map; The downsampled feature map is normalized through a BatchNorm layer and then non-linearly activated using the GELU activation function. Three convolutional modules are used to extract downsampled features at 8x, 16x, and 32x respectively. The output feature map of each convolutional module is then fused with the feature map at the corresponding scale position of the semantic information extraction module to obtain a fused multi-scale feature map.

7. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The semantic information extraction module is one of a semantic segmentation network, a residual network, or a generative adversarial network, used to extract high-level semantic information from insulator images and generate semantic feature maps for defect diagnosis. The processing steps of the semantic information extraction module include: Input an insulator image and extract features from the insulator image using a semantic segmentation network, residual network, or generative adversarial network; The extracted features are subjected to multi-layer convolution operations to gradually obtain the deep semantic information of the image; The extracted deep semantic information is predicted pixel by pixel to generate a high-dimensional semantic feature map that reflects the defect region and semantic region in the image; The high-dimensional semantic feature map is output to the defect diagnosis module and the semantic alignment module for subsequent defect classification and semantic alignment optimization.

8. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The defect diagnosis module is one of a convolutional neural network, a feature pyramid network, or a graph convolutional network, used to process and classify the high-dimensional semantic feature map extracted by the semantic information extraction module, and to determine the defect type in the insulator image. The processing steps of the defect diagnosis module include: The high-dimensional semantic feature map output by the input semantic information extraction module is used to extract deep features through multiple convolutional layers, and high-level feature maps are obtained step by step. Apply downsampling operations to the extracted high-level feature maps to obtain multi-scale feature representations; Fully connected layers are used to classify defects using multi-scale feature representations, outputting the probability distribution of defect categories, where each category represents a defect type; Output the defect classification results and perform post-processing to generate the final defect detection results.

9. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The semantic alignment module includes a local detail feature alignment module and a global semantic alignment module, which are used to align and optimize semantic information during the defect diagnosis process.

10. The insulator defect detection method based on saliency-aware data augmentation and semantic prior transfer according to claim 1, characterized in that, The step of training the defect diagnosis model using the augmented insulator image specifically includes: The augmented insulator image is input into the structural prior injection module. The module extracts the low-level feature map through convolution operation and performs non-linear activation through BatchNorm layer normalization and GELU activation function to obtain the feature map after four times downsampling. The feature map after four-fold downsampling is input into the semantic information extraction module, which performs deep feature extraction on the input features to generate a high-dimensional semantic feature map. The high-dimensional semantic feature map output by the semantic information extraction module is input into the defect diagnosis module. The defect diagnosis module performs deep feature extraction and downsampling on the high-dimensional semantic feature map to obtain multi-scale feature representation. In multi-scale feature representation, defect classification is performed through a fully connected layer, outputting the probability distribution of defect categories and generating the final defect detection results; During training, an offline distillation method is used. Knowledge transfer is achieved through a semantic information extraction module as the teacher model and a defect diagnosis module as the student model. The total loss function is calculated based on [reasoning missing], and the formula is: in, For the total loss function, Basic classification loss; This is the global semantic alignment loss; For local detail feature alignment loss; The basic classification loss is formulated as follows: in, The weights are determined by cross-entropy loss. The cross-entropy loss function; The prediction results for the student model; This is a real label; The global semantic alignment loss is calculated as follows: in, , These are the weights for the KL divergence loss and the global feature alignment loss, respectively. Temperature coefficient; Let KL divergence be a metric. and The student model and the teacher model were respectively in the first... i The predicted probability of a class; This is the last layer of feature maps in the teacher model; This is the last layer feature map of the student model; For global semantic extractors; , and These represent three different 1×1 convolutions; This represents Layer Normalization; This represents the GELU activation function; This represents the total number of spatial locations in the feature map; The local detail feature alignment loss is calculated as follows: in, , These are the weights for local feature alignment loss and attention alignment loss, respectively. For local feature alignment loss; This is for attention alignment loss; , and Input the length, width, and number of channels of the feature map respectively; , Teacher model and student model in the channel respectively k ,Location eigenvalues; For teacher models in the channel k Channel attention weights; For the teacher model in spatial location Spatial attention weights; This is for attention alignment loss; , These are spatial attention maps for the teacher model and the student model, respectively. These are the channel attention vectors for the teacher model and the student model, respectively; , These are spatial attention alignment loss and channel attention loss, respectively. , The spatial attention maps of the teacher model and the student model are respectively represented at position. eigenvalues; , These represent the channel attention vectors of the teacher model and the student model, respectively, in the channel. The value; By minimizing the total loss function, the parameters of the defect diagnosis model are updated, enabling knowledge transfer between the teacher model and the student model.

Images