A method for testing and detecting defects in multi-task insulator components

Abstract

This invention relates to the field of power equipment condition perception and computer vision inspection technology, and discloses a multi-task insulator component detection and defect detection method, comprising: acquiring a basic insulator image dataset; manually annotating and data augmenting the basic insulator image dataset to obtain a training set, a validation set, and a test set; improving the detection head of the target detection network based on a cross-task attention guidance mechanism; and using a multi-task loss function and the training set to perform multi-stage training on the improved target detection network to obtain an insulator detection model; inputting the test set into the insulator detection model, and using a non-maximum suppression algorithm, a preset cross-interval ratio (CIP) threshold, and a confidence threshold to perform post-processing on the detection results to obtain the final detection list. This invention, through network structure improvement and dual-task collaboration, completes component detection and defect detection in a single forward inference, balancing real-time performance and detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of power equipment condition sensing and computer vision inspection technology, and more specifically, to a method for multi-task insulator component inspection and defect detection. Background Technology

[0002] Insulators are critical components in power transmission lines, serving to support conductors and provide insulation protection. Their operational status directly affects the safety and stability of the power grid. During long-term service or in harsh environments, insulators may develop defects such as cracks, damage, flashover marks, and missing sheds. Failure to detect these defects in a timely manner can lead to a decline in insulation performance and, in severe cases, tripping accidents. Traditionally, power transmission line inspections rely primarily on manual visual inspection or the use of telescopes / infrared equipment to check each insulator at high altitudes, which suffers from low efficiency, high risk, and a high rate of missed detections. With the rise of drone inspections, a large number of close-range images of insulators provide a data foundation for intelligent detection. However, manually analyzing massive amounts of images is still time-consuming and labor-intensive. Therefore, there is an urgent need to utilize computer vision algorithms to achieve automatic defect identification.

[0003] In recent years, deep learning-based object detection has been applied in power line inspection. Typical methods fall into two categories: cascaded / sequential detection, which first detects the location of insulators and then performs defect detection on their local areas; and multi-task parallel detection, which outputs insulator components and their defects simultaneously in a single model. Sequential detection offers relatively higher accuracy due to effectively reducing background interference, but it requires two stages of inference, resulting in higher computational complexity and latency. Parallel multi-task methods, such as improved R-CNN and YOLO series algorithms, can complete localization and defect judgment simultaneously in a single forward pass, offering strong real-time performance. However, directly using single-stage detection models in complex outdoor scenarios still faces the following challenges and existing problems: (1) Missed detection of small target defects: Cracks, chipping, and other defects on the surface of insulators are small targets that are often difficult to identify in standard target detection networks, and are prone to being missed. In particular, when using deep features with a large number of downsampled features, information about small defects may be lost. Existing algorithms such as the standard YOLOv8n have a minimum feature layer resolution of only 1 / 8 of the original image under default configuration, which is insufficient for the detection accuracy of extremely small defects.

[0004] (2) Background interference and false alarms: Insulators are often in complex backgrounds, such as the sky, trees, and hardware. Ordinary algorithms may misidentify background textures as defects, resulting in false alarms. A single detection model learns both component and defect tasks simultaneously. If there is a lack of coordination, inconsistent detection results may occur. For example, background noise may be misdetected as a defect while component detection fails to detect the corresponding component. This cross-task inconsistency reduces the reliability of the results.

[0005] (3) Class Imbalance: In defect detection scenarios, normal parts far outnumber defective parts, and the proportion of defective samples is extremely low. The imbalance between positive and negative samples and different defect categories during deep model training can cause the model to be biased towards outputting "normal" and have low confidence in defect classification. Traditional cross-entropy loss is difficult to take into account the minority classes, and loss weight balancing or hard sample mining mechanisms need to be introduced, otherwise it will affect the detection of low-probability defects.

[0006] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0007] In view of the problems in related technologies, this invention proposes a method for multi-task insulator component inspection and defect detection, so as to overcome the above-mentioned technical problems existing in the existing related technologies.

[0008] Therefore, the specific technical solution adopted by the present invention is as follows: According to one aspect of the present invention, a method for detecting and diagnosing defects in multi-task insulator components is provided, the method comprising: S1. Obtain close-up images or video frames containing transmission line insulators to obtain a basic image dataset of insulators; S2. Manually annotate the insulator basic image dataset and perform data augmentation on the manually annotated insulator basic image dataset to obtain the insulator detection sample dataset. Divide the insulator detection sample dataset into training set, validation set and test set. S3. Based on the cross-task attention guidance mechanism, the detection head of the target detection network is decoupled and improved; and using the multi-task loss function and training set, the improved target detection network is trained in multiple stages to obtain the insulator detection model. S4. Input the test set into the insulator detection model, and use the non-maximum suppression algorithm, preset cross-parallel ratio threshold and confidence threshold to post-process the detection results of the insulator detection model to obtain the final detection list.

[0009] Furthermore, the insulator basic image dataset is manually labeled, and data augmentation is performed on the manually labeled insulator basic image dataset to obtain the insulator detection sample dataset. The insulator detection sample dataset is divided into training set, validation set and test set, including the following steps: S21. Manually annotate insulator components and defects in the basic image dataset of insulators; S22. Perform data augmentation processing on the insulator basic image dataset after manual annotation; S23. Divide the data augmentation-processed insulator basic image dataset into a training set, a validation set, and a test set to form an insulator detection sample dataset.

[0010] Furthermore, manual annotation of insulator components and defects on the insulator basic image dataset includes: The location boundary boxes and component categories of insulator components are marked, and the component categories include top, insulator body, flange, and base; The location bounding box and defect category of the insulator defect are marked. The defect categories include cracks, chipped edges, flashover marks, missing umbrellas, and foreign matter attachment. For missing umbrella defects, a virtual bounding box or marker symbol is drawn at the location of the missing umbrella skirt for annotation.

[0011] Furthermore, the improved target detection network includes a backbone network, a feature pyramid neck layer, and a dual-branch detection head; The dual-branch detection head includes a component detection branch and a defect detection branch, which are connected by a cross-task feature guidance module. The cross-task feature guidance module is used to guide the detection area of ​​the component detection branch to the defect detection branch.

[0012] Furthermore, guiding the detection area of ​​the defect detection branch from the component detection branch includes the following steps: Extract the feature map of the corresponding level from the component detection branch as the guiding feature, and upsample the guiding feature to the same spatial resolution as the feature map of the target level of the defect detection branch; The upsampled guiding features are then subjected to global average pooling and global max pooling in sequence to obtain several single-channel spatial feature maps. Several single-channel spatial feature maps are concatenated along the channel dimension to obtain a concatenated feature map. The concatenated feature map is then input into a convolutional layer for feature fusion and normalized using the Sigmoid activation function to generate a spatial attention map. The spatial attention map is expanded to the same number of channels as the feature map of the target level of the defect detection branch to obtain the expanded spatial weights. The expanded spatial weights are then multiplied element-wise with the feature map of the target level of the defect detection branch to obtain the modulated defect feature map. The defect detection branch performs defect detection based on the modulated defect feature map to guide the detection area.

[0013] Furthermore, using a multi-task loss function and training set, the improved target detection network is trained in multiple stages to obtain the insulator detection model, which includes the following steps: A multi-task loss function is constructed based on component inspection branch loss, defect inspection branch loss, and structural consistency constraint loss. Initialize the model parameters of the improved object detection network and input the training set into the improved object detection network; The improved target detection network is trained in multiple stages. Once the improved target detection network converges to a preset accuracy or completes a preset total number of training rounds, the training is stopped and the parameters of the improved target detection network are saved to obtain the insulator detection model.

[0014] Furthermore, based on the component inspection branch loss, defect detection branch loss, and structural consistency constraint loss, a multi-task loss function is constructed, including: Based on the component bounding box regression loss and component classification loss, a component detection branch loss is constructed; Based on the defect bounding box regression loss and the defect classification loss, a defect detection branch loss is constructed; The structural consistency constraint loss is calculated based on the maximum intersection-union ratio between the defect detection box and the component detection box. The component detection branch loss, defect detection branch loss, and structural consistency constraint loss are weighted and summed to obtain the total multi-task loss function.

[0015] Furthermore, multi-stage training includes first-stage training, second-stage training, and third-stage training; The training process involves three stages: the first stage requires freezing the defect detection branch and training only the component detection branch; the second stage requires unfreezing the defect detection branch, enabling the cross-task feature guidance module, and jointly training both branches; and the third stage requires combining structural consistency constraint loss to jointly optimize the parameters of the entire improved target detection network.

[0016] Furthermore, the test set is input into the insulator detection model, and the detection results of the insulator detection model are post-processed using a non-maximum suppression algorithm, a preset crossover ratio threshold, and a confidence threshold to obtain the final detection list, including the following steps: S41. Input the test set into the insulator detection model to obtain the component detection results and defect detection results; S42. Perform non-maximum suppression processing on the component detection results and defect detection results respectively to obtain the component detection box and the defect detection box; S43. Based on the preset intersection-union ratio threshold, filter out component detection boxes and defect detection boxes with high overlap; combined with the preset confidence threshold, remove component detection boxes and defect detection boxes with confidence levels below the threshold. S44. Integrate the filtered and rejected component detection boxes and defect detection boxes, associate the defect detection boxes with the corresponding component detection boxes, and generate the final detection list.

[0017] According to another aspect of the present invention, a multi-task insulator component inspection and defect detection system is provided, the system comprising: an insulator data acquisition module, an insulator inspection sample dataset construction module, an insulator inspection model training module, and a post-processing module; The insulator data acquisition module is used to acquire close-up images or video frames containing transmission line insulators to obtain a basic insulator image dataset. The insulator detection sample dataset construction module is used to manually annotate the basic insulator image dataset and perform data augmentation on the manually annotated basic insulator image dataset to obtain the insulator detection sample dataset. The insulator detection sample dataset is divided into training set, validation set and test set. The insulator detection model training module is used to decouple and improve the detection head of the target detection network based on a cross-task attention guidance mechanism; and to perform multi-stage training on the improved target detection network using a multi-task loss function and training set to obtain the insulator detection model. The post-processing module is used to input the test set into the insulator detection model and use the non-maximum suppression algorithm, preset cross-parallel ratio threshold and confidence threshold to post-process the detection results of the insulator detection model to obtain the final detection list.

[0018] The beneficial effects of this invention are as follows: 1. This invention optimizes the multi-scale feature fusion strategy in the target detection network, enhances the model's ability to perceive fine-grained defects, and improves the recall rate of small target defects such as cracks and edge breakage by introducing higher resolution P2 layer feature fusion and corresponding small-scale detection head.

[0019] 2. This invention employs a dual-branch decoupled detection head, outputting insulator components and defects separately. A cross-task guidance module is introduced between the two branches, using the heatmap of interest generated by the component detection branch to modulate the features of the defect branch. This ensures that defect detection only occurs within the predicted component region, guiding the model to achieve a similar focusing effect to a two-stage model within a single-stage architecture, avoiding inconsistencies between defect detection results and component detection results. Simultaneously, a structural consistency constraint loss is added during training to further ensure that the outputs of the two branches match each other spatially, reducing the false detection rate.

[0020] 3. This invention uses Tversky loss instead of ordinary cross-entropy for the defect detection branch, reducing the weight of easily classified samples and focusing on difficult samples, thereby improving the detection accuracy of minority class defects. Furthermore, an uncertainty-weighted multi-task loss balancing strategy is introduced, automatically adjusting the weights of the loss terms for component and defect sub-tasks, which is superior to fixed hyperparameter weighting and alleviates the problem of inconsistent gradient scales across tasks during training. At the data level, data augmentation, such as defect region copy-paste, simulated rain / fog / glare, and polarization filtering, increases the number and diversity of defect samples, mitigating the impact of class imbalance on model training. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a method for detecting and detecting defects in multi-task insulator components according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the network structure of the insulator detection model in a multi-task insulator component detection and defect detection method according to an embodiment of the present invention; Figure 3 This is a defect and component annotation diagram in a multi-task insulator component inspection and defect detection method according to an embodiment of the present invention. Detailed Implementation

[0023] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0024] According to an embodiment of the present invention, a method for detecting and defectively inspecting multi-task insulator components is provided.

[0025] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figures 1-3 As shown, the method for detecting and detecting defects in multi-task insulator components according to an embodiment of the present invention includes: S1. Obtain close-up images or video frames containing transmission line insulators to obtain a basic image dataset of insulators.

[0026] It should be explained that this involves acquiring close-up images or video frames containing transmission line insulators. Image acquisition is accomplished using a robotic camera or a fixed, high-definition camera. The acquisition equipment is configured with a resolution of approximately 640×480 and a frame rate of 30fps to ensure real-time performance and sufficient clarity. To improve image quality, a telephoto lens can be used to capture insulator details up close, and a polarizing filter can be used to reduce glare caused by direct sunlight. Additional lighting may be necessary to address backlighting or shadows. The acquisition process should cover insulator images from different angles and under varying lighting conditions to provide comprehensive training and testing data for subsequent algorithms.

[0027] S2. Manually annotate the basic image dataset of insulators, and perform data augmentation on the manually annotated basic image dataset of insulators to construct an insulator detection sample dataset; In this optional embodiment, the insulator basic image dataset is manually annotated, and the manually annotated insulator basic image dataset is then augmented to obtain the insulator detection sample dataset. Dividing the insulator detection sample dataset into a training set, a validation set, and a test set includes the following steps: S21. Manually annotate insulator components and defects in the basic image dataset of insulators; S22. Perform data augmentation processing on the insulator basic image dataset after manual annotation; S23. Divide the data augmentation-processed insulator basic image dataset into a training set, a validation set, and a test set to form an insulator detection sample dataset.

[0028] In this optional embodiment, manual annotation of insulator components and defects in the insulator basic image dataset includes: The location boundary boxes and component categories of insulator components are marked, and the component categories include top, insulator body, flange, and base; The location bounding box and defect category of the insulator defect are marked. The defect categories include cracks, chipped edges, flashover marks, missing umbrellas, and foreign matter attachment. For missing umbrella defects, a virtual bounding box or marker symbol is drawn at the location of the missing umbrella skirt for annotation.

[0029] It needs to be explained that after data acquisition, all images were manually annotated for defects and components. Annotators used specialized tools to draw bounding boxes around the locations of various insulator components in each image and labeled their categories, such as the top, insulator body, flange, and base. Simultaneously, they marked the locations of discovered defects and their corresponding categories, such as cracks, chipped edges, flashover marks, missing insulator skirts, and foreign matter attachments. For "missing insulator skirt" defects, such as missing insulator skirts, since the defect itself represents a missing component, the annotation used a virtual bounding box or marker symbol to indicate the missing location (e.g., where the skirt should be). Figure 3 (As shown). Figure 3 The diagram illustrates the marking methods for components and defects. Components such as tops and flanges are marked with solid-line rectangles, while defects such as cracks and missing parts are marked with different colors or symbols.

[0030] All annotation work was completed independently by two professionals and cross-checked to minimize mislabeling and omissions. After annotation, the data was divided into training, validation, and test sets. This division ensured that samples of each defect category were covered in each dataset, preventing any defect category from existing only in the training or test sets, thus ensuring more objective evaluation. Furthermore, a series of data augmentation operations were applied to the annotated datasets, such as random rotation, translation, scaling, brightness / contrast adjustment, and simulating effects like raindrops, water mist, and lens flare, to enhance the model's robustness to complex real-world environments. These augmentations, without altering the defect and component label positions, enriched the diversity of training samples, providing more sufficient and effective data for model learning.

[0031] S3. Based on the cross-task attention guidance mechanism, the detection head of the target detection network is decoupled and improved; and using the multi-task loss function and training set, the improved target detection network is trained in multiple stages to obtain the insulator detection model. In this optional embodiment, the improved target detection network includes a backbone network, a feature pyramid neck layer, and a dual-branch detection head; The dual-branch detection head includes a component detection branch and a defect detection branch, which are connected by a cross-task feature guidance module. The cross-task feature guidance module is used to guide the detection area of ​​the component detection branch to the defect detection branch.

[0032] In this optional embodiment, guiding the detection area of ​​the component detection branch to the defect detection branch includes the following steps: Extract the feature map of the corresponding level from the component detection branch as the guiding feature, and upsample the guiding feature to the same spatial resolution as the feature map of the target level of the defect detection branch; The upsampled guiding features are then subjected to global average pooling and global max pooling in sequence to obtain several single-channel spatial feature maps. Several single-channel spatial feature maps are concatenated along the channel dimension to obtain a concatenated feature map. The concatenated feature map is then input into a convolutional layer for feature fusion and normalized using the Sigmoid activation function to generate a spatial attention map. The spatial attention map is expanded to the same number of channels as the feature map of the target level of the defect detection branch to obtain the expanded spatial weights. The expanded spatial weights are then multiplied element-wise with the feature map of the target level of the defect detection branch to obtain the modulated defect feature map. The defect detection branch performs defect detection based on the modulated defect feature map to guide the detection area.

[0033] In this optional embodiment, the improved target detection network is trained in multiple stages using a multi-task loss function and a training set to obtain an insulator detection model, including the following steps: A multi-task loss function is constructed based on component inspection branch loss, defect inspection branch loss, and structural consistency constraint loss. Initialize the model parameters of the improved object detection network and input the training set into the improved object detection network; The improved target detection network is trained in multiple stages. Once the improved target detection network converges to a preset accuracy or completes a preset total number of training rounds, the training is stopped and the parameters of the improved target detection network are saved to obtain the insulator detection model.

[0034] In this optional embodiment, the multi-task loss function is constructed based on the component detection branch loss, defect detection branch loss, and structural consistency constraint loss, including: Based on the component bounding box regression loss and component classification loss, a component detection branch loss is constructed; Based on the defect bounding box regression loss and the defect classification loss, a defect detection branch loss is constructed; The structural consistency constraint loss is calculated based on the maximum intersection-union ratio between the defect detection box and the component detection box. The component detection branch loss, defect detection branch loss, and structural consistency constraint loss are weighted and summed to obtain the total multi-task loss function.

[0035] Specifically, an improved multi-task detection network was built and trained offline using labeled data. Training employed a batch size of 16, 100 training epochs, an initial learning rate of 0.001, and a gradient descent strategy to adjust the learning rate.

[0036] To balance the performance of component inspection and defect detection tasks, a multi-task loss function is designed: ; in, The component detection branch loss includes bounding box regression loss and component classification loss; For the defect detection branch loss, where the defect classification loss is calculated, recall should be prioritized. Tversky loss is used to enhance robustness against imbalanced positive and negative samples. The Tversky loss function is as follows: ; in, To predict probabilities, This is a real label; and False positive and false negative penalties are controlled separately and are configurable hyperparameters. The bounding box loss uses EIoU loss to achieve more accurate localization. The structural consistency loss term is used to constrain the consistency of the outputs of the two branches. The above losses are weighted and summed in the total loss, where λ is the consistency loss weight.

[0037] During the training phase of the improved object detection network, this invention adds a structural consistency constraint as an auxiliary term in the loss function to ensure that the prediction results of the two branches are structurally consistent. Structural consistency is mainly reflected in the fact that defects should be attached to a detected component, and should not appear at locations where no component exists.

[0038] Based on this, a loss function based on the positional relationship of the detection boxes was designed. Let the set of bounding boxes obtained from the component branch prediction during training be... The defect box set obtained from defect branch prediction is For each defect prediction box Calculate it with all component prediction boxes The IoU (Intersection over Union) is taken as the maximum value. Structural consistency loss is defined as: ; in, The number of defect prediction boxes. This loss measures the extent to which each defect finds a corresponding part; if a defect box does not overlap with any part box ( If the defect box falls entirely within a component box, then the contribution to the total loss is 1, indicating inconsistency; If the value approaches 1, the loss tends to 0, indicating consistency. By minimizing this loss, the improved object detection network is encouraged to match the output defect boxes with component boxes, thus avoiding isolated defect detections. It should be noted that for special cases such as missing umbrellas, the defect box may not have a corresponding component box. In this case, the actual component box does not exist during training. However, if the improved object detection network can correctly predict the missing umbrella defect, it will still generate a consistency loss signal due to the lack of a corresponding component, guiding the improved object detection network to attempt to predict a location frame or missing component in the component branch. In practice, the weight λ of the consistency loss can be set relatively small, mainly to assist optimization without interfering with the normal convergence of the main task. Combining the cross-task guidance module and structural consistency loss, the two branches form a collaborative mechanism in both the training and inference phases: the former is a forward explicit constraint, and the latter is a backward implicit constraint, jointly improving the rationality and consistency of the prediction results of the improved object detection network.

[0039] In this optional embodiment, multi-stage training includes a first stage training, a second stage training, and a third stage training; The training process involves three stages: the first stage requires freezing the defect detection branch and training only the component detection branch; the second stage requires unfreezing the defect detection branch, enabling the cross-task feature guidance module, and jointly training both branches; and the third stage requires combining structural consistency constraint loss to jointly optimize the parameters of the entire improved target detection network.

[0040] Specifically, to avoid gradient conflicts caused by simultaneous updates across multiple task branches, this invention employs a phased training strategy: Phase 1 (first 30 rounds) only trains the component detection branch and freezes the defect detection branch parameters; Phase 2 (mid-term 30 rounds) Unfreeze the defect branch and enable cross-task attention modules (CT, CTAM). Phase 3 (the last 40 rounds) simultaneously optimizes the parameters of the entire network and introduces structural consistency constraints.

[0041] It should be explained that this invention proposes a structure-prior-guided enhancement strategy, which performs "axial translation, random rotation, and occlusion simulation" operations on the input image to enhance sample diversity while maintaining the topology of the components. The enhancement parameters are as follows: Translation range: ±5% of the total length of the insulator; Rotation angle: ±10°; Occlusion ratio: 0.1-0.3 (prioritize the umbrella skirt area).

[0042] like Figure 2As shown in the diagram (C2f represents the feature reorganization structure, a lightweight residual connection module that achieves a balance between information preservation and model lightweighting by grouping and stacking the input feature maps step by step and fusing them; SPPF represents fast spatial pyramid pooling, used to integrate multi-scale spatial context information, enhancing the model's robustness to changes in target size through multi-scale max pooling operations, and is more computationally efficient than the SPP module; Upsample represents upsampling, used to restore low-resolution feature maps to high resolution, often used in feature fusion and image generation tasks, such as bilinear interpolation or transposed convolution; CT represents cross-task attention, used to concatenate two single-channel feature maps along the channel dimension, then input them into a 1×1 convolutional layer to obtain the fused attention map, and then upsample the single-channel heatmap and apply Sigmoid activation to normalize its value to the 0-1 range; CTAM represents cross-task attention modulation module, used to expand the attention map to the same number of channels as the corresponding defect branch feature map, and multiply it element-wise with the feature map of the defect branch as spatial weights to obtain the modulated defect feature map), this invention adopts a multi-task network structure with a shared backbone and decoupled branches. The network is divided into three parts: the backbone network, the feature pyramid neck, and the dual-branch detection head.

[0043] The branch-decoupled dual-head structure: After the FPN neck, the network branches into two detection heads, each undertaking different tasks, making the predictions for the component detection and defect detection subtasks more tailored to their respective characteristics. Specifically: 1) Component Detection Branch: This branch utilizes feature maps at three scales (P3, P4, and P5) for target detection, outputting instance-level localization and classification of each insulator component. For example, for a rod-shaped composite insulator, component categories include the top, insulator, flange, and base. This branch's detection head is more sensitive to larger-scale features, therefore the P2 layer is not used to reduce interference. The component detection head at each scale consists of several convolutional layers and an output layer, with the predicted bounding boxes categorized into the aforementioned four component types.

[0044] 2) Defect Detection Branch: This branch takes feature maps at three scales (P2, P3, and P4) as input and performs detection on five types of defects: cracks, chipped edges, flashover marks, missing umbrella skirts, and foreign matter attachments. Considering that defects are generally small in size and have fine textures, this invention introduces fine-grained features of the P2 layer on the basis of conventional multi-scale structures to improve the detection rate of small defects.

[0045] To further enhance the focusing capability of the defect branch on the insulator region, this invention introduces a cross-task guidance module into this branch. This module upsamples the features of higher layers (P3, P4, P5) in the component detection branch and generates a spatial attention heatmap. This heatmap is then multiplied element-wise with the features of the corresponding layers (P2, P3, P4) in the defect branch, achieving cross-scale feature modulation. In this way, the defect branch can focus on the region identified as an "insulator component" by the component branch, suppressing background region responses and thus reducing false detections.

[0046] The defect detection head has a similar structure to the component branch, but outputs the five defect categories mentioned above. The dual-branch structure decouples the detection head: the parameters of the two convolutional layers are independent. The component branch focuses more on geometric shape and overall structural features, while the defect branch focuses on texture details and local anomalies. This avoids feature conflicts caused by a single detection head learning too many tasks at the same time, thus improving the overall detection accuracy.

[0047] To enable the two branches to collaborate during reasoning, this invention incorporates a cross-task feature guidance module between the component branch and the defect branch. The core idea is to utilize the component branch's determination of the insulator's position to guide the defect branch to focus only on these areas, thereby reducing background interference and improving defect prediction accuracy. The specific implementation is as follows: 1) Feature upsampling: The higher-level feature maps output by the component branches, such as the guiding feature for defect P2 from component branch P3, the guiding feature for defect P3 from component P4, and the guiding feature for defect P4 from component P5, are amplified by bilinear interpolation so that their spatial resolution is adjusted to be the same as the size of the feature maps (P2, P3, P4) at the corresponding scale of the defect branches.

[0048] 2) Channel Pooling: For each upsampled component feature map, global average pooling and global max pooling are performed on the channel dimension to obtain two single-channel spatial feature maps. , Spatial feature map , This represents the overall response of the component features at each spatial location at this scale.

[0049] 3) Generate attention map The two single-channel feature maps above are concatenated along the channel dimension, as shown in the following formula: ; In the formula, Concat represents feature concatenation, which concatenates feature maps from different network layers along the channel dimension for feature fusion or cross-scale information integration. Represents the space of real numbers. This represents a real tensor of size H×W×2 (i.e., a feature map), where H and W represent the height and width of the feature map.

[0050] Another 1×1 convolutional layer is input to obtain the fused attention map. ,as follows: ; In the formula, Conv represents convolution, used to extract local features of an image. It is the most basic operation module in deep learning and is often used in conjunction with activation functions. Used together with a normalization layer.

[0051] Upsample the single-channel heatmap, apply Sigmoid activation, and normalize the value to the range of 0 to 1 to output a cross-task fused attention map. : ; This attention map can be seen as encoding the regions in the current image that "may contain insulator components": a value close to 1 indicates that the location is likely to belong to a component region, and a value close to 0 indicates that the location is more likely to be a background region.

[0052] 4) Feature modulation: The obtained attention map Copy along channel dimension Second-rate( ), so that its size and defect characteristics Figure 1 To, as in the following formula: ; In the formula, The number of channels in the feature map. Used as spatial weights. Feature map of defect branches element-wise multiplication The modulated defect feature map is obtained. Through this element-wise multiplication, the following equation applies: ; The defect branch significantly weakens features in background regions with low attention values ​​(no components), while maintaining or even relatively strengthening features in regions with high attention values ​​(components present). As a result, the defect detection branch essentially only activates near locations where insulator components are predicted to exist, thus focusing on these critical regions for defect detection.

[0053] In its specific implementation, the present invention relates to several key technical parameters and adjustable selection parameters, as shown in Table 1.

[0054] Table 1: Actual Parameter Table Image input resolution Input image size for model processing 640×480 key Detect frame rate Real-time processing speed 30FPS key backbone network CNN backbone model structure YOLOv8n key Feature level The scale used in the FPN feature layer P2, P3, P4, P5 key Component inspection head usage scale Feature layers used in component branches P3, P4, P5 key Defect detection head usage scale Feature layers used in defect branches P2, P3, P4 key Consistency loss weight Weight of structural consistency loss in total loss 0.2 Optional Training batch size Number of samples in each iteration 16 Optional Initial learning rate Optimized initial learning rate 1e-3 Optional Training rounds Total number of training iterations epochs 100 Optional Optimizer Weight update strategy SGD Optional Number of frozen layers Number of backbone network layers frozen in the first 30 epochs 10th floor Optional NMS threshold Non-maximum suppression IoU threshold 0.5 Optional Confidence threshold Output detection minimum confidence level 0.45 Optional In Table 1, "critical" indicates essential core parameters or design choices that are indispensable in the embodiments of the present invention, such as the requirement to use P2 layer features and dual branch heads in this solution; "optional" indicates parameters that can be adjusted or replaced according to actual conditions.

[0055] In this context, CNN stands for Convolutional Neural Network; YOLO represents a single-stage object detection architecture (You Only Look Once); FPN stands for Feature Pyramid Network, used for multi-scale feature fusion; P2 / P3 / P4 / P5 represent different levels of feature maps in FPN, for example, P2 refers to a feature layer downsampled by 4 times; NMS stands for Non-Maximum Suppression, used to filter overlapping redundant detection boxes; and IoU stands for Intersection over Union, which evaluates the degree of overlap between the detection box and the ground truth.

[0056] S4. Input the test set into the insulator detection model, and use the non-maximum suppression algorithm, preset cross-parallel ratio threshold and confidence threshold to post-process the detection results of the insulator detection model to obtain the final detection list.

[0057] In this optional embodiment, the test set is input into the insulator detection model, and the detection results of the insulator detection model are post-processed using a non-maximum suppression algorithm, a preset cross-parallel ratio threshold, and a confidence threshold to obtain the final detection list, including the following steps: S41. Input the test set into the insulator detection model to obtain the component detection results and defect detection results; S42. Perform non-maximum suppression processing on the component detection results and defect detection results respectively to obtain the component detection box and the defect detection box; S43. Based on the preset intersection-union ratio threshold, filter out component detection boxes and defect detection boxes with high overlap; combined with the preset confidence threshold, remove component detection boxes and defect detection boxes with confidence levels below the threshold. S44. Integrate the filtered and rejected component detection boxes and defect detection boxes, associate the defect detection boxes with the corresponding component detection boxes, and generate the final detection list.

[0058] It should be explained that the detection results of each branch are processed using the Non-Maximum Suppression (NMS) algorithm. A preset IoU threshold is used to filter redundant detections with high overlap, and a confidence threshold is used to eliminate detections with low confidence. This filters overlapping and redundant candidate boxes and eliminates low-confidence detections. Specifically, NMS uses intra-category NMS on the defect branch (different categories of defects are allowed to exist in the same area). This ensures that, for example, when both "crack" and "flashover" defects exist in a component area, the two defect boxes will not inhibit each other and can be retained simultaneously. After NMS processing, the final output includes several component location boxes and their categories (e.g., "top," "flange," etc.) and several defect location boxes and their categories (e.g., "crack," "missing umbrella," etc.), each with a confidence score. Due to the aforementioned cross-task guidance and consistency constraints, in most cases, each defect box corresponds to the location of a component box. Therefore, the final detection result can be intuitively understood as: a certain insulator component has been detected, and a certain defect has been found on that component. This result format makes it easy for maintenance personnel to quickly locate problematic components and determine the type of defect.

[0059] According to another embodiment of the present invention, a multi-task insulator component inspection and defect detection system is provided, the system comprising: an insulator data acquisition module, an insulator inspection sample dataset construction module, an insulator inspection model training module, and a post-processing module; The insulator data acquisition module is used to acquire close-up images or video frames containing transmission line insulators to obtain a basic insulator image dataset. The insulator detection sample dataset construction module is used to manually annotate the basic insulator image dataset and perform data augmentation on the manually annotated basic insulator image dataset to obtain the insulator detection sample dataset. The insulator detection sample dataset is divided into training set, validation set and test set. The insulator detection model training module is used to decouple and improve the detection head of the target detection network based on a cross-task attention guidance mechanism; and to perform multi-stage training on the improved target detection network using a multi-task loss function and training set to obtain the insulator detection model. The post-processing module is used to input the test set into the insulator detection model and use the non-maximum suppression algorithm, preset cross-parallel ratio threshold and confidence threshold to post-process the detection results of the insulator detection model to obtain the final detection list.

[0060] In summary, by utilizing the above-mentioned technical solutions of this invention, the present invention optimizes the multi-scale feature fusion strategy in the target detection network, enhancing the model's ability to perceive fine-grained defects. By introducing higher-resolution P2 layer feature fusion and corresponding small-scale detection heads, the recall rate of small target defects such as cracks and edge breakage is improved. This invention employs a dual-branch decoupled detection head, outputting insulator components and defects respectively. A cross-task guidance module is introduced between the two branches, using the attention heatmap generated by the component detection branch to modulate the features of the defect branch, ensuring that defect detection only occurs within the predicted component region. This guides the model to achieve a similar focusing effect to a two-stage model under a single-stage architecture, avoiding inconsistencies between defect detection results and component detection results. Simultaneously, a structural consistency constraint loss is added during training to further ensure that the outputs of the two branches match spatially, reducing the false detection rate. This invention uses Tversky loss instead of ordinary cross-entropy for the defect detection branch, reducing the weight of easily classified samples and focusing on difficult samples, thereby improving the detection accuracy of minority class defects. Furthermore, an uncertainty-weighted multi-task loss balancing strategy is introduced to automatically adjust the weights of the loss terms for component and defect sub-tasks, which is superior to fixed hyperparameter weighting and alleviates the problem of inconsistent gradient scales across tasks during training. At the data level, data augmentation (such as copy-pasting defect regions, simulating rain / fog / glare, and polarization filtering) increases the number and diversity of defect samples, mitigating the impact of class imbalance on model training.

[0061] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting defects in multi-task insulator components, characterized in that, The method includes: S1. Obtain close-up images or video frames containing transmission line insulators to obtain a basic image dataset of insulators; S2. Manually annotate the insulator basic image dataset and perform data augmentation on the manually annotated insulator basic image dataset to obtain the insulator detection sample dataset. Divide the insulator detection sample dataset into training set, validation set and test set. S3. Based on the cross-task attention guidance mechanism, the detection head of the target detection network is decoupled and improved; and using the multi-task loss function and training set, the improved target detection network is trained in multiple stages to obtain the insulator detection model. S4. Input the test set into the insulator detection model, and use the non-maximum suppression algorithm, preset cross-parallel ratio threshold and confidence threshold to post-process the detection results of the insulator detection model to obtain the final detection list.

2. The method for detecting and identifying defects in multi-task insulator components according to claim 1, characterized in that, The process of manually annotating the basic insulator image dataset and then performing data augmentation on the annotated basic insulator image dataset to obtain the insulator detection sample dataset, and dividing the insulator detection sample dataset into training set, validation set, and test set, includes the following steps: S21. Manually annotate insulator components and defects in the basic image dataset of insulators; S22. Perform data augmentation processing on the insulator basic image dataset after manual annotation; S23. Divide the data augmentation-processed insulator basic image dataset into a training set, a validation set, and a test set to form an insulator detection sample dataset.

3. The method for detecting and identifying defects in multi-task insulator components according to claim 2, characterized in that, The manual annotation of insulator components and defects in the insulator basic image dataset includes: The location boundary boxes and component categories of insulator components are marked, and the component categories include top, insulator body, flange, and base; The location bounding box and defect category of the insulator defect are marked. The defect categories include cracks, chipped edges, flashover marks, missing umbrellas, and foreign matter attachment. For missing umbrella defects, a virtual bounding box or marker symbol is drawn at the location of the missing umbrella skirt for annotation.

4. The method for detecting and defectively inspecting multi-task insulator components according to claim 1, characterized in that, The improved target detection network includes a backbone network, a feature pyramid neck layer, and a dual-branch detection head. The dual-branch detection head includes a component detection branch and a defect detection branch, which are connected by a cross-task feature guidance module. The cross-task feature guidance module is used to guide the detection area of ​​the component detection branch to the defect detection branch.

5. The method for detecting and defectively inspecting multi-task insulator components according to claim 4, characterized in that, The process of guiding the component detection branch to the detection area of ​​the defect detection branch includes the following steps: Extract the feature map of the corresponding level from the component detection branch as the guiding feature, and upsample the guiding feature to the same spatial resolution as the feature map of the target level of the defect detection branch; The upsampled guiding features are then subjected to global average pooling and global max pooling in sequence to obtain several single-channel spatial feature maps. Several single-channel spatial feature maps are concatenated along the channel dimension to obtain a concatenated feature map. The concatenated feature map is then input into a convolutional layer for feature fusion and normalized using the Sigmoid activation function to generate a spatial attention map. The spatial attention map is expanded to the same number of channels as the feature map of the target level of the defect detection branch to obtain the expanded spatial weights. The expanded spatial weights are then multiplied element-wise with the feature map of the target level of the defect detection branch to obtain the modulated defect feature map. The defect detection branch performs defect detection based on the modulated defect feature map to guide the detection area.

6. The method for detecting and defectively inspecting multi-task insulator components according to claim 5, characterized in that, The process of using a multi-task loss function and a training set to train the improved target detection network in multiple stages to obtain the insulator detection model includes the following steps: A multi-task loss function is constructed based on component inspection branch loss, defect inspection branch loss, and structural consistency constraint loss. Initialize the model parameters of the improved object detection network and input the training set into the improved object detection network; The improved target detection network is trained in multiple stages. Once the improved target detection network converges to a preset accuracy or completes a preset total number of training rounds, the training is stopped and the parameters of the improved target detection network are saved to obtain the insulator detection model.

7. The method for detecting and identifying defects in multi-task insulator components according to claim 6, characterized in that, The multi-task loss function constructed based on component detection branch loss, defect detection branch loss, and structural consistency constraint loss includes: Based on the component bounding box regression loss and component classification loss, a component detection branch loss is constructed; Based on the defect bounding box regression loss and the defect classification loss, a defect detection branch loss is constructed; The structural consistency constraint loss is calculated based on the maximum intersection-union ratio between the defect detection box and the component detection box. The component detection branch loss, defect detection branch loss, and structural consistency constraint loss are weighted and summed to obtain the total multi-task loss function.

8. The method for detecting and defectively inspecting multi-task insulator components according to claim 7, characterized in that, The multi-stage training includes a first-stage training, a second-stage training, and a third-stage training. The training process involves three stages: the first stage requires freezing the defect detection branch and training only the component detection branch; the second stage requires unfreezing the defect detection branch, enabling the cross-task feature guidance module, and jointly training both branches; and the third stage requires combining structural consistency constraint loss to jointly optimize the parameters of the entire improved target detection network.

9. The method for detecting and defectively inspecting multi-task insulator components according to claim 1, characterized in that, The process of inputting the test set into the insulator detection model and post-processing the detection results using a non-maximum suppression algorithm, a preset cross-parallel ratio threshold, and a confidence threshold to obtain the final detection list includes the following steps: S41. Input the test set into the insulator detection model to obtain the component detection results and defect detection results; S42. Perform non-maximum suppression processing on the component detection results and defect detection results respectively to obtain the component detection box and the defect detection box; S43. Based on the preset intersection-union ratio threshold, filter out component detection boxes and defect detection boxes with high overlap; combined with the preset confidence threshold, remove component detection boxes and defect detection boxes with confidence levels below the threshold. S44. Integrate the filtered and rejected component detection boxes and defect detection boxes, associate the defect detection boxes with the corresponding component detection boxes, and generate the final detection list.

10. A multi-task insulator component inspection and defect detection system, used to implement the multi-task insulator component inspection and defect detection method according to any one of claims 1-9, characterized in that, The system includes: an insulator data acquisition module, an insulator test sample dataset construction module, an insulator test model training module, and a post-processing module; The insulator data acquisition module is used to acquire close-up images or video frames containing transmission line insulators to obtain a basic insulator image dataset. The insulator detection sample dataset construction module is used to manually annotate the basic insulator image dataset and perform data augmentation on the manually annotated basic insulator image dataset to obtain the insulator detection sample dataset. The insulator detection sample dataset is divided into training set, validation set and test set. The insulator detection model training module is used to decouple and improve the detection head of the target detection network based on a cross-task attention guidance mechanism; and to perform multi-stage training on the improved target detection network using a multi-task loss function and training set to obtain the insulator detection model. The post-processing module is used to input the test set into the insulator detection model and use the non-maximum suppression algorithm, preset cross-parallel ratio threshold and confidence threshold to post-process the detection results of the insulator detection model to obtain the final detection list.

Images