Multi-task power transmission line defect detection algorithm based on feature sharing

Abstract

The invention relates to the technical field of intelligent inspection, and provides a multi-task power transmission line defect detection algorithm based on feature sharing. The objective of the invention is to solve the technical problems of low power transmission line defect detection precision, high false alarm rate and weak model generalization ability in the unmanned aerial vehicle inspection process. The main scheme comprises the steps of collecting RGB image data of a power transmission line through an unmanned aerial vehicle; performing device-level target identification by adopting a DINO target detection model, and outputting a position frame and a category array containing normal devices and defects; clustering according to a target center point by using a K-means clustering algorithm, combining the cut areas in combination with a pre-adaptive window size threshold, and outputting cut target image stacks to optimize scale processing; inputting the cut image into a shared feature extraction network, respectively outputting defect detection results of different device categories through a plurality of independent detection heads, and improving the distinguishing capability by using multi-task learning; according to result mapping, the detection result is mapped back to an original image according to cutting frame coordinates, and final defect positioning and classified output are generated.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology for power equipment, and more specifically, to a multi-task transmission line defect detection algorithm based on feature sharing. Background Technology

[0002] Addressing the high costs and low efficiency of traditional manual inspections of power transmission lines and equipment, drone inspection has become a primary method for the maintenance and repair of power transmission equipment. This method integrates intelligence, safety, and efficiency, providing crucial support for improving national economic development and people's living standards. Currently, with the construction of more and more power transmission facilities, the increasing demand for drone inspections, and the increasingly complex environment of power transmission lines, the performance requirements for drone defect detection algorithms are also rising. Utilizing drone inspection data and deep learning technology to efficiently and accurately identify defect locations has become paramount in drone inspection.

[0003] The main methods for detecting defects in drones currently include:

[0004] A voting-based UAV defect detection method: This method uses multiple deep learning models to detect multiple defects. It leverages the different models' abilities to extract important features such as scale, defect shape, and texture details for target identification. In post-processing, non-primary targets are filtered using methods such as non-maximum suppression to obtain the defect targets. For this type of method, the number and selection of models, as well as the design of the post-processing mechanism, are crucial. Therefore, extensive experimental trials are necessary to obtain the optimal number and structure of expert models.

[0005] Attention-based UAV defect detection methods: This approach utilizes various attention mechanisms, including but not limited to Convolutional Block Attention (CBAM), Channel Attention (SE), and Efficient Multi-Scale Attention (EMA), as well as self-attention-based feature extraction algorithms (Transformer, ViT, etc.), to extract features. Then, it uses feature fusion to detect defects at different scales by adjusting the output at different scales. For these methods, the number and position of attention mechanism insertions, and corresponding changes in model structure, still require extensive ablation to achieve optimal results. Self-attention-based methods leverage global attention to acquire long-range contextual information and are currently among the best-performing defect detection algorithms. However, these models have high computational complexity and strong data dependency; for high-resolution image data, computation and memory usage increase dramatically.

[0006] A UAV defect detection method based on YOLO and its improved models: This method uses YOLO and its improved models (with attention enhancements and feature extraction enhancements) as the main body, primarily through lightweight improvements such as pruning and quantization, or by using mosaic and dynamic anchor box design to improve data distribution and enhance defect detection accuracy. While this type of method is relatively fast and can meet the needs of most real-time detection tasks, its accuracy is difficult to match that of Transformer series models for complex tasks.

[0007] In summary, this patent provides a multi-task transmission line defect detection algorithm based on feature sharing, focusing on model performance and efficiency. Summary of the Invention

[0008] The purpose of this invention is to solve the technical problems of low detection accuracy, high false alarm rate and weak model generalization ability of transmission line defects caused by large changes in target size, high similarity between defect classes and scarce defect data during UAV inspection.

[0009] To achieve the above objectives, the present invention employs the following technical means:

[0010] This invention provides a multi-task transmission line defect detection algorithm based on feature sharing, comprising the following steps:

[0011] Step 1: Collect RGB image data of the power transmission line using a drone;

[0012] Step 2: First-stage detection: The DINO target detection model is used to perform device-level target recognition on the input image, and the output is an array containing the location boxes and categories of normal devices and device defects;

[0013] Step 3: Second-stage clustering and cropping: Based on the output of the first stage, the target images are clustered according to the target center point using the K-means clustering algorithm, and the cropping regions are merged by combining the pre-adaptive window size threshold, and the cropped target images are stacked.

[0014] Step 4: Third-stage multi-task detection: The cropped image is input into a shared feature extraction network for feature extraction. The obtained shared features are then output as defect detection results for different device categories through multiple independent detection heads.

[0015] Step 5: Result Mapping: Map the detection results back to the original image based on the cropping box coordinates to generate the final defect location and classification results.

[0016] In the above scheme, step 1 includes:

[0017] Drones are used to inspect power transmission lines along pre-set flight routes, and cameras capture image data from different angles and positions. The raw image data is then processed to transform...

[0018] Images in RGB format.

[0019] In the above scheme, step 2 includes:

[0020] Step 2.1: Data Processing: Online data augmentation of the input image is performed using Large Scale Japonics (LSJ), including:

[0021] Random sampling scaling factor s:

[0022] s~Uniform(s min ,s max )

[0023] Among them, s min Indicates the lower limit of the scaling ratio, s max Indicates the upper limit of the scaling ratio;

[0024] Calculate the longer side L of the scaled image:

[0025] L = max(sH, sW)

[0026] Where H represents the original image height and W represents the original image width;

[0027] If L>L max Then scale it up twice proportionally:

[0028]

[0029] Where L max The preset long side threshold is used;

[0030] Step 2.2: Object Detection: Input the data processing results into the DINO detection model to obtain the output results. The DINO structure includes feature extraction, Transformer encoder, Transformer decoder and object detection head;

[0031] Step 2.3: Result merging: using a score threshold θ score and size threshold θ size The filtered detection results are merged into an array of dimension (N, 5), where each row contains the center coordinates (x, y), width w, height h, and label l.

[0032] In the above scheme, step 2.2 includes the following sub-steps:

[0033] Step 2.2.1: Feature Extraction: The Swin-Transformer-Large model is used to extract features from the input image. This model contains 4 stages, each of which outputs semantic features at different levels. Then, channel mapping is performed through a single-layer convolution with a kernel size of 3 to adjust the number of feature channels to adapt to the encoder input. Finally, the features output from each stage are concatenated and input into the Transformer encoder.

[0034] Step 2.2.2: Encoding: In the Transformer encoder, feature encoding is performed on the concatenated features to convert the features into a global context token sequence;

[0035] Step 2.2.3: Decoding: In the Transformer decoder, a two-branch approach is used to process the input token sequence. Branch 1 is the matching branch, which is used for normal supervised learning. Branch 2 is the denoising branch, which adds random noise to the input labels and bounding boxes to enhance the robustness and anti-interference ability of the model and provide stable supervision signals for positive and negative samples.

[0036] Step 2.2.4: Detection head output: The detection head part classifies and regresses the decoding results, and outputs the target's coordinate position and category score.

[0037] In the above scheme, step 3 includes the following sub-steps:

[0038] Step 3.1: For the first-stage output array, obtain the cropping region range in the image using the prefit window method;

[0039] Step 3.2: Crop the image according to the area to be retained, and obtain the cropped image;

[0040] Step 3.3: Stack the cropped images to obtain the second stage output.

[0041] In the above scheme, step 3.1 includes the following sub-steps:

[0042] Step 3.1.1: Set the pre-fit window size threshold and the maximum number of clipping frames;

[0043] Step 3.1.2: Using the k-means clustering method, divide the output array from the first stage into different clusters according to the center points of the boxes. The objective function for clustering is:

[0044]

[0045] Where C i Let μ represent the i-th cluster. i Let x represent the centroid of the i-th cluster, and let x represent the cluster G. iThe sample points, K represents the set number of clusters; G i Indicates the i-th cluster;

[0046] ||x-μ i || 2 This represents the distance from sample point x to its cluster centroid μ. i The square of the Euclidean distance;

[0047] Step 3.1.3: Calculate the minimum bounding rectangle window that covers all bounding boxes within each cluster for each cluster;

[0048] Step 3.1.4: Perform coordinate merging calculations on all calculated minimum bounding rectangle windows, and use the pre-adaptive window size threshold to judge the merged window: if the size of the merged window meets the pre-adaptive window size threshold, then output the merged window as a clipping region; if it does not meet the threshold, continue to merge with other windows until the maximum number of clipping boxes is met or all windows have been calculated.

[0049] In the above scheme, step 4 includes the following sub-steps:

[0050] Step 4.1: Shared Feature Extraction: The second-stage multi-task object detection head model using shared features employs Cascade-RCNN as the base detection model and Swin-Transformer-Large as the shared feature extraction structure, i.e., STL-Cascade-RCNN. The cropped images output from Step 3 are stacked and input into the shared feature extraction network to obtain shared features.

[0051] Step 4.2: Multi-task detection head application: The shared features extracted in step 4.1 are simultaneously input into multiple independent target detection heads, and each detection head is specifically responsible for outputting the defect detection result for a specific device category;

[0052] Step 4.3: Obtaining weights through three-stage progressive freeze training: The weights of the shared feature extraction structure and multiple independent detection heads are obtained through three-stage progressive freeze training.

[0053] In the above scheme, the three-stage progressive freeze training to obtain weights specifically includes the following steps:

[0054] 4.3.1: First Stage Training - Self-Supervised Pre-training: Using a dataset containing N device categories, the feature extraction network is trained using Masked Image Modeling (MIM) self-supervised learning to obtain self-supervised weights W1. Specifically, this includes: occluding the input image with a randomly generated mask and dividing it into several image patches; the model only receives the unoccluded patches, learns the context representation through the encoder, and predicts the pixel values ​​of the occluded regions; the pixel reconstruction error of the occluded regions is calculated as the loss function, which uses the mean absolute error, as shown in the following formula:

[0055]

[0056] Among them, y i This represents the true value of the i-th occluded pixel. represents the predicted value of the i-th occluded pixel, and n represents the total number of pixels in the occluded area.

[0057] Step 4.3.2: Second Stage Training - Fine-tuning of Object Detection:

[0058] Step 4.3.2.1: Initialize the feature extraction structure of the STL-Cascade-RCNN model constructed in Step 4.1 with the weights W1 obtained in Step 4.3.1, and randomly initialize the remaining structures;

[0059] Step 4.3.2.2: Use the image data obtained by cropping based on the real annotations using the pre-adaptive window method as the training set; train using a stochastic gradient descent (SGD) optimizer and a linear decaying learning rate;

[0060] Step 4.3.2.3: After training is completed, obtain the object detection pre-training weights W2;

[0061] Step 4.3.3: Third-stage training - Freezing the feature extraction structure to train the multi-head detector: Initialize and freeze the corresponding structure of the current model using the feature extraction structure weights in weight W2 obtained in step 4.3.2; for each category i in the N device categories:

[0062] Use a set of defect categories that belong to device category i. Cropped image data was used as the training set;

[0063] Train a detection head specifically designed for device category i;

[0064] After training, the model weights for each device category i are obtained.

[0065] For each Perform structured decomposition, extract and separate its feature extraction weights. and detection head weight

[0066] Weight all detection heads Stack and package the data to obtain the final multiple plug-and-play detection header weights. Because the weight parameters of the feature extraction structure are frozen during training, each Since the parameters are the same, only one shared feature extraction weight is retained as... Feature extraction weights:

[0067]

[0068] The above scheme also includes a testing phase for three progressive freeze training stages: obtaining data from the three progressive freeze training stages. Weights and Weights are assigned and used to initialize the second-stage feature-sharing multi-head object detection model STL-Cascade-RCNN;

[0069] The cropping target of the second stage is input into the model. After passing through the feature extraction network, the extracted features are simultaneously input into multiple target detection heads to obtain target prediction results on different devices.

[0070] All object detection results are collected, and coordinate mapping is performed based on the original image coordinates of the two-stage cropping boxes to map the coordinates of the predicted targets back to the original image, thus obtaining the final output result.

[0071] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:

[0072] 1. By using large-scale scaling data augmentation and clustering image cropping techniques, the problem of missing small targets caused by large changes in target scale was solved, thereby improving detection accuracy and reducing missed detections.

[0073] In drone inspections, the scale of power transmission equipment and defects varies greatly due to changes in shooting angle and distance, easily leading to missed detections of small targets. This invention first employs Large Scale Scaling (LSJ) data augmentation, randomly scaling the input image to different scales during the training phase (scaling ratio s is sampled from a uniform distribution), exposing the model to targets of diverse scales and enhancing its robustness to scale changes. Subsequently, clustering and cropping (based on the k-means algorithm, clustering by target center point and merging cropped regions) focuses on densely populated target areas, reducing background interference. Logically, LSJ forces the model to learn scale-invariant features, while clustering and cropping, by cropping high-density target areas, amplifies the target's proportion in the image, making it easier for the model to capture details of small targets. This solves the problem of missed detections caused by scale changes, improving the recall rate of small targets and overall detection accuracy.

[0074] 2. By adopting a three-stage freeze training strategy and a multi-detector structure, the problem of false alarms caused by high similarity between defect classes is solved, thereby improving detection accuracy and robustness.

[0075] Defects in power transmission equipment (such as insulator cracks and contamination) exhibit high inter-class similarity, making it easy for single detection models to confuse features and increase false alarms. This invention employs a three-stage frozen training approach: the first stage learns general features of defect-free samples through self-supervised pre-training (Masked ImageModeling); the second stage fine-tunes the target detection task and initializes the shared feature extraction network; the third stage freezes the shared network, training only multiple independent detection heads (each head dedicated to defects of a specific device category). Logically, self-supervised pre-training builds a robust feature foundation, while the independent multi-detector head structure decouples the task by device category, avoiding feature overlap between similar defects. This reduces inter-class interference, solves the false alarm problem, and improves the detection accuracy and model robustness for defects in various equipment categories.

[0076] 3. By employing self-supervised pre-training and supervised fine-tuning in the three-stage frozen training strategy, the problem of weak generalization ability caused by scarce defective data was solved, thereby enhancing the model's discriminative power and sensitivity.

[0077] The scarcity of defective samples causes the model to favor the defect-free class, resulting in weak generalization ability. This invention addresses this issue through a three-stage training process. First, self-supervised pre-training (e.g., mask reconstruction task) utilizes a large number of defect-free samples to learn scene context features (with a pixel-level MAE loss function). Then, supervised fine-tuning introduces a small number of defective samples to adjust the discrimination boundary. Logically, the self-supervised stage fully mines the structural and semantic information of the defect-free data to construct a generalized feature space; the fine-tuning stage freezes the backbone network, optimizing only the detection head, allowing the limited number of defective samples to efficiently guide feature optimization. This solves the data imbalance problem and improves the model's sensitivity to defects and discrimination accuracy, even under conditions of scarce samples.

[0078] In summary, this invention systematically solves the three core problems of scale variation, inter-class similarity, and data scarcity, significantly improving the accuracy, robustness, and generalization ability of transmission line defect detection. Attached Figure Description

[0079] Figure 1 The diagram shows a simplified overall structure and process. In the diagram, the multi-task two-stage detection based on feature sharing refers to the second stage using Swin Transformer + Cascade-RCNN, i.e., STL-Cascade-RCNN.

[0080] Figure 2 This is a flowchart of a three-stage progressive freeze training process. Detailed Implementation

[0081] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0082] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0083] A multi-task transmission line defect detection algorithm based on feature sharing includes the following steps:

[0084] Step 1: Data Acquisition: The data comes from image data generated by the drone inspection. Following the drone's pre-set flight path, the drone's camera will capture images of different devices from different angles and positions. Therefore, the captured image data contains defects in target equipment of varying sizes. The raw data will undergo image processing and conversion to obtain RGB format images.

[0085] Step 2: Algorithm Detection: The algorithm consists of two phases. The first phase identifies and locates defects and normal devices. The second phase performs clustering and image cropping based on the location results from the first phase. The third phase uses shared feature extraction and a multi-task detection head for defect detection. Details are as follows:

[0086] Step 2.1: First Stage: For the image data input to the model, the first step is to use a target detection algorithm to identify device-level targets. This stage primarily uses DINO as the baseline algorithm. By inputting an image, it obtains the bounding boxes (bboxes) and classes of normal devices and device defects, storing them as NumPy array vectors with dimensions (N, 5). A detailed description follows:

[0087] Step 2.1.1: Data Processing: To enable the model to adapt to the scale changes of targets during UAV inspections, this invention employs Large Scale Jitter (LSJ) as the primary data augmentation method for online data augmentation of image data. LSJ generates samples at different scales by randomly scaling the input image, typically combining the following operations: the image scaling ratio s is randomly sampled from a continuous interval, as shown in Formula 1; the original image is then scaled using the scaling ratio s to obtain the scaled image I. s To avoid extreme sizes, the longer side of the scaled image is often limited to a certain threshold L. max As shown in Formula 2, where H and W are the original image dimensions.

[0088] s~Uniform(s min s max (1)

[0089]

[0090] Step 2.1.2: Object Detection: Input the data processing results into the Dino detection model to obtain the output results. The Dino structure mainly includes feature extraction, a Transformer encoder, a Transformer decoder, and an object detection head.

[0091] Step 2.1.2.1: Feature Extraction Structure. The Swin-Transformer-Large model is used as the first-stage feature extraction model, comprising four stages, each outputting semantic features at different levels. After the image is processed by this structure, the output of each stage is obtained. Then, channel mapping is performed using a single-layer convolution with a kernel size of 3 to adjust the number of feature channels to fit the encoder input. Finally, the features are concatenated and input into the encoder.

[0092] Step 2.1.2.2: In the encoder, the extracted features are encoded to convert the features into a global context token sequence.

[0093] Step 2.1.2.3: In the decoder, a two-branch approach is used to process the input token sequence. The first branch is the matching branch, which is used for normal supervised learning. The second branch is the denoising branch, which adds a certain amount of random noise to the input label and box to enhance the model's robustness and anti-interference ability to learn from samples and provide stable supervision signals for positive and negative samples.

[0094] Step 2.1.2.4: The detection head will perform classification and coordinate regression on the output, and output the target's coordinate position and category score.

[0095] Step 2.1.3: Prediction result merging: The coordinate positions and class scores of the output targets are filtered by the score filtering threshold and the size filtering threshold, and merged into a (n, 5) NumPy array, where each row represents a target box, including the center x coordinate of the box, the center y coordinate of the box, the width w of the box, the height h of the box, and the label 1 of the box.

[0096] Step 2.2: Second stage: Based on the output results of the first stage, this stage uses a clustering and cropping method to control the number of output images, so as to reduce the amount of computation and save the detection time of a single image.

[0097] Step 2.2.1: For the first-stage output NumPy array, obtain the cropping region range in the image using the prefit window method. The prefit window method is described as follows:

[0098] Step 2.2.1.1: Set the pre-adaptive window size threshold and the maximum number of clipping boxes.

[0099] Step 2.2.1.2: Use the k-means clustering method to divide the NumPy array into different clusters according to the center points of the boxes. The objective function is shown in Equation 3, where C i Let μ represent the i-th cluster. i Let x represent the centroid of the i-th cluster, and let x represent the sample point belonging to that cluster.

[0100]

[0101] Step 2.2.1.3: Calculate the minimum window covering all targets for each cluster.

[0102] Step 2.2.1.4: Perform coordinate merging calculations on all windows, and use the pre-adapted window size threshold to judge the merged windows. If the threshold is met, output the merged box; otherwise, continue merging until the maximum number of clipping boxes is met or all windows have been calculated.

[0103] Step 2.2.2: Crop the image according to the retained cropping area to obtain the cropped image.

[0104] Step 2.2.3: Stack the cropped images to obtain the second-stage output.

[0105] Step 2.3: Third Stage: After obtaining the output results of the second stage, this stage extracts features from the results using a shared feature extraction structure, and then feeds the extracted features to different defect category detection heads for defect detection. For training in this stage, Swin Transformer + Cascade-RCNN is used as the baseline model, and STL-Cascade-RCNN is used subsequently. Customized defect detection heads are trained according to the output result category boxes and category device categories. During training, this patent provides a three-stage progressive freeze training method to achieve high performance of each detection head on the corresponding detection target and to have anti-interference ability in complex environments; at the same time, in order to reduce the number of parameters, a shared feature extraction structure is used and the above-mentioned multiple detection heads are used as a multi-task output structure. During testing, the shared feature extraction weights and multi-head defect detection weights are directly loaded to complete the detection task.

[0106] Step 2.3.1: Three-Stage Progressive Freeze Training: Assume there are a total of M defect categories to be detected, and these defects can be classified into N categories based on the device in which they occur. In the first stage, the dataset containing N device categories is used as the training set, and self-supervised weights W1 are obtained using Masked Image Modeling (MIM) self-supervised learning. In the second stage, the feature extraction structure of STL-Cascade-RCNN is trained without freezing using the first-stage weights W1 to obtain pre-trained weights W2 for object detection. In the third stage, the feature extraction structure of STL-Cascade-RCNN is frozen using the pre-trained weights W2, and only the detection head part is trained, resulting in multiple plug-and-play... Detection head weights and shared Feature extraction weights.

[0107] Step 2.3.1.1: First Stage Training: This stage obtains self-supervised weights W1 based on a self-supervised task. First, the image is randomly occluded using a mask and divided into several patches, constructing an unsupervised training method. Then, during training, the model only receives the unoccluded patches and learns the context representation through the encoder to predict the pixel values ​​of the masked regions. Finally, the reconstruction error of the masked regions is calculated as the loss, used to optimize the model. The loss function adopts MAE loss, expressed as follows, where y i This represents the i-th real pixel. Let represent the i-th predicted pixel, and n represent the total number of pixels in the mask region. By performing occlusion reconstruction on data from N device categories, the model can adapt to the data distribution of power transmission line drones, improving its ability to extract scene context information.

[0108]

[0109] Step 2.3.1.2: Second Stage Training: In this stage, pre-trained weights W2 for object detection are obtained based on the object detection task. The feature extraction structure of the ST-Cascade-R-CNN feature extraction structure used in this stage is initialized using the feature extraction structure of the model weights W1 from the first stage; other parts are initialized randomly. The optimizer used during training is SGD, and the learning rate is linearly decaying. The training data consists of cropped data based on the ground truth (gt) annotations using a pre-adaptive window method, resulting in the pre-trained weights W2 for object detection.

[0110] Step 2.3.1.3: Third Stage Training: This stage acquires multiple plug-and-play detection heads based on the object detection task. Weights and Weights. The feature extraction structure weights in the object detection pre-trained weights W2 are used to initialize the feature extraction structure of the third-stage model. After initialization, the feature extraction structure weights are frozen, and only the detection head part is trained. Specifically, for N device categories, the set of defect categories to be detected for each category is as follows:

[0111] Will contain The defect category set and the cropped image data using the pre-adaptive window method are used as the training set to train the model in this stage with frozen feature extraction structure;

[0112] Obtain the model weights after training.

[0113] right By performing structured decomposition, we obtain and This can be simplified to Formula 5, where "stack" refers to a stacking method where all weights are merged and packaged. Since frozen training is used, all feature extraction structure parameters are identical, therefore, they are retained. The first weight in the matrix is ​​used as the shared feature extraction structure weight, i.e. Feature extraction weights.

[0114]

[0115] Step 2.3.2: Testing Phase: Obtaining data from the three progressive freeze training sessions. Weights and Weights are assigned and used to initialize the second-stage feature-sharing multi-head object detection model, STL-Cascade-RCNN. The cropped target from the second stage is then input into this model, and after passing through a feature extraction network, the extracted features are simultaneously input into multiple object detection heads to obtain object prediction results on different devices.

[0116] Step 2.3.3: Gather all target detection results, perform coordinate mapping based on the original image coordinates of the two-stage cropping box, map the coordinates of the predicted target back to the original image, and obtain the final output result.

[0117] Furthermore, to facilitate a better understanding of the technical concept and non-obviousness of this invention by those skilled in the art, the following more detailed description is provided, highlighting the main difficulties in the field of UAV defect detection:

[0118] 1. Large Target Scale Variation: Due to the influence of drone aerial photography routes, the scale of power transmission equipment and defects on the equipment varies greatly in the images. General detection models that directly identify the original image will result in a large number of small targets being missed, affecting target recall.

[0119] 2. High similarity between defect classes: In power transmission line equipment, the same type of equipment may have different defects in different materials and models, but they have a very high degree of similarity in appearance. Therefore, directly using a single detection model to identify a large number of targets with high structural similarity is prone to false alarms among different defects, reducing the overall recognition accuracy. On the other hand, using multiple models for detection will result in a large amount of repetitive feature calculation, which will seriously affect the recognition efficiency and consume a lot of graphics memory.

[0120] 3. Limited Defect Data: In actual image samples, the number of defective samples is often far less than that of normal samples, leading to a severe imbalance in data distribution. This class imbalance causes the model to tend to learn features of the defect-free class during training, inhibiting effective learning of the defective class. Due to the insufficient number of defective samples, the model struggles to fully capture their diversity and discriminative features, easily leading to overfitting to the limited number of defective samples in the training set, thus affecting the model's generalization ability and robustness in real-world scenarios.

[0121] To address the above challenges, this invention offers the following technical advantages:

[0122] To address challenge 1, the first stage of this invention proposes a strong data augmentation technique using LSJ (Laser-Scaling Jutsu), which randomly scales images over a very wide range, allowing the model to encounter targets of various scales during training. This enables the model to better adapt to changes in target scale during testing, particularly improving the ability to recognize small targets. In the second stage, clustering and cropping of the detected targets from the first stage reduces the proportion of useless information in the image while increasing the proportion of the target to be detected, allowing the model to more accurately identify the target.

[0123] To address challenge 2, the second phase of this invention proposes a three-stage frozen training strategy. For complex defect scenarios at the device level, a multi-detection head structure is designed, and defect types are grouped and modeled according to device category, achieving task decoupling between different detection heads. This method effectively reduces the risk of false detections caused by overlapping defect features between similar devices, thereby significantly improving the accuracy and robustness of defect detection within various device categories while maintaining overall recall.

[0124] To address challenge 3, a three-stage frozen training strategy is proposed, which systematically integrates the advantages of self-supervised pre-training and supervised fine-tuning. First, self-supervised pre-training is performed on a large number of defect-free samples to fully explore their potential information in structural and semantic representations, thereby constructing an initial feature space with generalization capabilities. Subsequently, by freezing the parameters at different levels of the model in stages and introducing a small number of defective samples for supervised fine-tuning, the model achieves transfer optimization from general features to task-specific discriminative features. Since the model has already fully modeled the defect-free distribution during pre-training, its guiding role is significantly amplified in subsequent stages, even with a limited number of defective samples. This allows for effective adjustment of the discrimination boundary based on the pre-trained weights, thereby improving the model's sensitivity and discrimination accuracy in defect recognition tasks.

Claims

1. A multi-task transmission line defect detection algorithm based on feature sharing, characterized in that, Includes the following steps: Step 1: Collect RGB image data of the power transmission line using a drone; Step 2: First-stage detection: The DINO target detection model is used to perform device-level target recognition on the input image, and the output is an array containing the location boxes and categories of normal devices and device defects; Step 3: Second-stage clustering and cropping: Based on the output of the first stage, the target images are clustered according to the target center point using the K-means clustering algorithm, and the cropping regions are merged by combining the pre-adaptive window size threshold, and the cropped target images are stacked. Step 4: Third-stage multi-task detection: The cropped image is input into a shared feature extraction network for feature extraction. The obtained shared features are then output as defect detection results for different device categories through multiple independent detection heads. Step 5: Result Mapping: Map the detection results back to the original image based on the cropping box coordinates to generate the final defect location and classification results.

2. The method according to claim 1, characterized in that, Step 1 includes: The drones inspect the power transmission lines according to a pre-set flight route and use cameras to capture image data from different angles and positions. The raw image data is then processed and converted into RGB format images.

3. The method according to claim 1, characterized in that, Step 2 includes: Step 2.1: Data Processing: Online data augmentation of the input image is performed using Large Scale Japonics (LSJ), including: Random sampling scaling factor s: s~Uniform(s min ,s max ) Among them, s min Indicates the lower limit of the scaling ratio, s max Indicates the upper limit of the scaling ratio; Calculate the longer side L of the scaled image: L = max(sH, sW) Where H represents the original image height and W represents the original image width; If L>L max Then scale it up twice proportionally: Where L max The preset long side threshold is used; Step 2.2: Target Detection: Input the data processing results into the DINO detection model to obtain the output results. The DINO structure includes feature extraction, Transforer encoder, Transforer decoder and target detection head; Step 2.3: Result merging: using a score threshold θ score and size threshold θ size The filtered detection results are merged into an array of dimension (N, 5), where each row contains the center coordinates (x, y), width w, height h, and label l.

4. The method according to claim 3, characterized in that, Step 2.2 includes the following sub-steps: Step 2.2.1: Feature Extraction: The Swin-Transformer-Large model is used to extract features from the input image. This model contains 4 stages, each of which outputs semantic features at different levels. Then, channel mapping is performed through a single-layer convolution with a kernel size of 3 to adjust the number of feature channels to adapt to the encoder input. Finally, the features output from each stage are concatenated and input into the Transformer encoder. Step 2.2.2: Encoding: In the Transformer encoder, feature encoding is performed on the concatenated features to convert the features into a global context token sequence; Step 2.2.3: Decoding: In the Transformer decoder, a two-branch approach is used to process the input token sequence. Branch 1 is the matching branch, which is used for normal supervised learning. Branch 2 is the denoising branch, which adds random noise to the input labels and bounding boxes to enhance the robustness and anti-interference ability of the model and provide stable supervision signals for positive and negative samples. Step 2.2.4: Detection head output: The detection head part classifies and regresses the decoding results, and outputs the target's coordinate position and category score.

5. The method according to claim 1, characterized in that, Step 3 includes the following sub-steps: Step 3.1: For the first-stage output array, obtain the cropping region range in the image using the prefit window method; Step 3.2: Crop the image according to the area to be retained, and obtain the cropped image; Step 3.3: Stack the cropped images to obtain the second stage output.

6. The method according to claim 5, characterized in that, Step 3.1 includes the following sub-steps: Step 3.1.1: Set the pre-fit window size threshold and the maximum number of clipping frames; Step 3.1.2: Using the k-means clustering method, divide the output array from the first stage into different clusters according to the center points of the boxes. The objective function for clustering is: Where C i Let μ represent the i-th cluster. i Let x represent the centroid of the i-th cluster, and let x represent the cluster G. i The sample points, K represents the set number of clusters; G i Indicates the i-th cluster; ||x-μ i || 2 This represents the distance from sample point x to its cluster centroid μ. i The square of the Euclidean distance; Step 3.1.3: Calculate the minimum bounding rectangle window that covers all bounding boxes within each cluster for each cluster; Step 3.1.4: Perform coordinate merging calculations on all calculated minimum bounding rectangle windows, and use the pre-adaptive window size threshold to judge the merged window: if the size of the merged window meets the pre-adaptive window size threshold, then output the merged window as a clipping region; if it does not meet the threshold, continue to merge with other windows until the maximum number of clipping boxes is met or all windows have been calculated.

7. The method according to claim 1, characterized in that, Step 4 includes the following sub-steps: Step 4.1: Shared Feature Extraction: The second-stage multi-task object detection head model with shared features uses Cascade-RCNN as the base detection model and Swin-Transformer-Large as the shared feature extraction structure of the model, namely STL-Cascade-RCNN. The cropped images output from step 3 are stacked and input into the shared feature extraction network to obtain shared features. Step 4.2: Multi-task detection head application: The shared features extracted in step 4.1 are simultaneously input into multiple independent target detection heads, and each detection head is specifically responsible for outputting the defect detection result for a specific device category; Step 4.3: Obtaining weights through three-stage progressive freeze training: The weights of the shared feature extraction structure and multiple independent detection heads are obtained through three-stage progressive freeze training.

8. The method according to claim 1, characterized in that, The three-stage progressive freeze training for obtaining weights specifically includes the following steps: 4.3.1: First Stage Training - Self-Supervised Pre-training: Using a dataset containing N device categories, the feature extraction network is trained using the MaskedImage Modeling self-supervised learning method to obtain self-supervised weights W1. Specifically, this includes: occluding the input image with a randomly generated mask and dividing it into several image patches; the model only receives the unoccluded patches, learns the context representation through the encoder, and predicts the pixel values ​​of the occluded regions; the pixel reconstruction error of the occluded regions is calculated as the loss function, which uses the mean absolute error, as shown in the following formula: Among them, y i This represents the true value of the i-th occluded pixel. represents the predicted value of the i-th occluded pixel, and n represents the total number of pixels in the occluded area. Step 4.3.2: Second Stage Training - Fine-tuning of Object Detection: Step 4.3.2.1: Initialize the feature extraction structure of the STL-Cascade-RCNN model constructed in Step 4.1 with the weights W1 obtained in Step 4.3.1, and randomly initialize the remaining structures; Step 4.3.2.2: Use the image data obtained by cropping based on the real annotations using the pre-adaptive window method as the training set; train using a stochastic gradient descent (SGD) optimizer and a linear decaying learning rate; Step 4.3.2.3: After training is complete, obtain the object detection pre-training weights W2: Step 4.3.3: Third-stage training - Freezing the feature extraction structure to train the multi-head detector: Initialize and freeze the corresponding structure of the current model using the feature extraction structure weights in weight W2 obtained in step 4.3.2; for each category i in the N device categories: Use a set of defect categories that belong to device category i. Cropped image data was used as the training set; Train a detection head specifically designed for device category i; After training, the model weights for each device category i are obtained. For each Perform structured decomposition, extract and separate its feature extraction weights. and detection head weight Weight all detection heads Stack and package the data to obtain the final multiple plug-and-play detection header weights. Because the weight parameters of the feature extraction structure are frozen during training, each Since the parameters are the same, only one shared feature extraction weight is retained as... Feature extraction weights:

9. The method according to claim 8, characterized in that, It also includes a testing phase for three progressive freeze training stages: obtaining data from the three progressive freeze training stages. Weights and Weights are assigned and used to initialize the second-stage feature-sharing multi-head object detection model STL-Cascade-RCNN; The cropping target of the second stage is input into the model. After passing through the feature extraction network, the extracted features are simultaneously input into multiple target detection heads to obtain target prediction results on different devices. All object detection results are collected, and coordinate mapping is performed based on the original image coordinates of the two-stage cropping boxes to map the coordinates of the predicted targets back to the original image, thus obtaining the final output result.

Images