A method for inspecting power transmission line insulators by drone in complex scenarios

By constructing a UAV-based insulator inspection model for power transmission lines in complex scenarios based on YOLOv5, and utilizing the residual feature pyramid RFPN and the coordinate convolution detection head CoordH, the accuracy and speed issues of UAV insulator inspection in complex scenarios were solved, achieving efficient insulator inspection.

CN117612044BActive Publication Date: 2026-11-17XIANGTAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311639096.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-01
Publication Date
2026-11-17
Estimated Expiration
2043-12-01

AI Technical Summary

Technical Problem

Existing UAV aerial image detection methods for insulators struggle to achieve high-precision detection in complex scenarios with low computational resource consumption, facing challenges such as complex backgrounds, multi-scale changes in targets, changing perspectives, and limited airborne resources.

Method used

A backbone feature extraction network based on YOLOv5 and the CSP-Darknet53 structure is adopted, combined with the residual feature pyramid RFPN and the coordinate convolution detection head CoordH, and supervised learning is performed using the DWD loss function to construct a transmission line insulator detection model.

Benefits of technology

It improves the accuracy and speed of insulator testing with low computational resource consumption, meeting the needs of autonomous inspection by UAVs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117612044B_ABST
    Figure CN117612044B_ABST
Patent Text Reader

Abstract

This invention discloses a method for inspecting insulators on power transmission lines using unmanned aerial vehicles (UAVs) in complex scenarios, primarily addressing the problems of low detection accuracy and high computational resource consumption in existing technologies. The method involves acquiring images of the insulators to be inspected, inputting these images into a pre-trained insulator image detection model to obtain the detection results. The insulator image detection model includes a backbone feature extraction network, a residual feature pyramid, and a spatial convolution detection head. This invention improves the accuracy of insulator detection in aerial images of power transmission lines with low computational resource requirements and can be applied to the routine inspection of power transmission line insulators' operational status using UAVs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a method for aerial image detection of insulators in transmission lines, specifically a method for unmanned aerial vehicle (UAV) inspection of transmission line insulators in complex scenarios. Background Technology

[0002] Insulators, as key components of overhead power lines, are responsible for providing electrical isolation and mechanical support. However, insulators are exposed to the elements for extended periods, operating in harsh environments such as high temperatures, sun, rain, and freezing, making them susceptible to damage. This damage affects insulator performance, causing power grid failures, and even power outages and equipment damage. Therefore, routine inspection and maintenance of insulators are crucial for the safe and efficient operation of modern power grids. Because insulators are installed at high altitudes, limited visibility makes manual visual inspection labor-intensive and time-consuming, resulting in low efficiency; it is also highly subjective and difficult to accurately assess insulators with obstructions or overlaps. Therefore, drone-based insulator imagery is an effective means of detecting defects in transmission line insulators. In recent years, many studies have focused on the visual task of automated inspection of transmission line insulators. To achieve automated drone inspection of transmission line insulators, the first step is to quickly and accurately identify and locate the insulator equipment.

[0003] In recent years, with the development of deep learning, convolutional neural networks (CNNs) have become mainstream, enabling them to adaptively learn features in images. CNN-based methods have become mainstream, including network models such as Single-Shot Multi-Box Detector (SSD), YOLO series, R-CNN, Fast R-CNN, and Faster R-CNN. Although these general object detection methods have achieved high detection accuracy for general objects in natural scenes, the detection of insulators and their faults still cannot achieve satisfactory results due to the multi-scale variations, complex backgrounds, and small target characteristics of UAV aerial images. The task of detecting insulators in UAV aerial images still faces the following challenges:

[0004] (1) Complex backgrounds in aerial images. Transmission lines cover a variety of environments such as forests, lakes, urban areas and snowfields. The different backgrounds in the drone aerial images of transmission line insulators increase the difficulty of insulator identification and detection.

[0005] (2) Multi-scale changes in targets. Due to the influence of high and low voltage on power transmission lines, the safe flight altitude of UAVs changes, and the scale of the same type of target in UAV images changes drastically accordingly. Moreover, large and small targets may coexist in the same image. The model needs to be able to detect targets with changing scales.

[0006] (3) Changing drone shooting angle. The aerial shooting angle changes dynamically during drone inspection, and the same target presents multiple perspectives in different images.

[0007] (4) Limited onboard resources. Due to limitations in load-bearing capacity, power consumption, and cost, the computing platform resources onboard to UAVs are extremely limited. Deploying the detection model on such a resource-constrained computing platform requires reducing the model's parameters, computational complexity, and inference time while ensuring detection accuracy.

[0008] Existing methods, under the aforementioned challenges, cannot detect the coherence between insulators from aerial images of transmission line insulators with low computational resource consumption and with speed and accuracy. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a method for inspecting insulators of transmission lines by drones in complex scenarios, which addresses the above-mentioned problems of the prior art. The present invention can solve the problem that the existing methods have low image detection accuracy of drones inspecting insulators of transmission lines while maintaining low computing resource consumption.

[0010] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0011] A method for inspecting transmission line insulators using unmanned aerial vehicles (UAVs) in complex scenarios includes acquiring images of the transmission line insulators to be inspected, and inputting the images into a pre-trained transmission line insulator detection model to obtain the detection results of the insulators. The transmission line insulator image detection model includes:

[0012] The backbone feature extraction network follows the CSP-Darknet53 structure of YOLOv5 and is used to extract different levels of features {C1, C2, C3, C4, C5} from the image of transmission line insulators. Feature map C5 is obtained from feature map C4 through the SPPF module.

[0013] The Residual Feature Pyramid (RFPN) consists of a 1×1 convolutional module, a reparameterized feature fusion module (RCSP), and a concatenation module; it integrates multi-level feature information, enriches the feature space, and enhances the model's representational ability.

[0014] The CoordH coordinate convolution detection head, with cascaded 3×3 coordinate convolution and 1×1 convolution modules, performs detection based on the output features R4, R5 and R6 of the residual pyramid structure to obtain the detection results of the transmission line insulators.

[0015] The DWD loss function is used to supervise the learning of the network.

[0016] Optionally, the reparameterized feature fusion module RCSP has different structures in the training and inference phases, which are denoted as the training phase structure RCSP respectively. train and the reasoning stage structure RCSP inferenceThe training phase structure RCSP is described above. train For a multi-branch structure, the feature map Fc with C input feature channels is divided into two branches for processing. Branch one consists of cascaded 1×1 convolutions and multi-branch parallel processing modules, with outputs F1, F2, F3, and F4 respectively. Branch two processes the feature map with C / 2 channels through a 1×1 convolution to obtain feature map F′. C / 2 The outputs F1, F2, F3, F4 and F′ of the two branches. C / 2 Perform the concatenation operation to obtain the output F. C The multi-branch parallel processing module includes four branches. The first branch of the multi-branch parallel processing module is a 1×1 convolution, which yields a feature map F1 with C / 8 channels. The second branch of the multi-branch parallel processing module is a 3×3 convolution, which yields a feature map F2 with C / 8 channels. The third branch of the multi-branch parallel processing module is a BN layer, which yields a feature map F3 with C / 8 channels. The fourth branch of the multi-branch parallel processing module is a convolutional sequence of 1×1 convolution and 3×3 convolution, which yields a feature map F4 with C / 8 channels.

[0017] The aforementioned inference stage structure RCSP inference For a single-branch structure, the RCSP structure during the training phase... train The reparameterization process involves several steps: first, reparameterizing the BN layer of the multi-branch parallel processing module into a 3×3 convolution; second, reparameterizing the 1×1 convolution into a 3×3 convolution; third, reparameterizing the serial sequence convolutional module constructed from the 1×1 and 3×3 convolutions into a 3×3 convolutional module; fourth, reparameterizing the 3×3 convolutional modules on the four branches with C / 8 channels into 3×3 convolutional modules with C / 2 channels, and then reparameterizing them together with the 1×1 convolutional module in branch one into a 3×3 convolutional module; fifth, reparameterizing the 1×1 convolutional module in branch two into a 3×3 convolutional module; and finally, reparameterizing the 3×3 convolutional modules in branch one and branch two into a single-branch structure of 3×3 convolutional modules to obtain the output feature map F with C channels. C .

[0018] Optionally, the coordinate convolution detection head CoordH includes a 3×3 coordinate convolution module and a 1×1 convolution module. Input features Ri (i = 4, 5, 6) with C channels are processed by the 3×3 coordinate convolution module, introducing spatial information into the feature map. This adds two channels to the feature map, one representing the x-coordinate and the other the y-coordinate, resulting in a feature map F with C+2 channels. C+2 Feature map F C+2 Then, through a 1×1 convolution module, the category score, confidence score, and bounding box position of the target object are obtained, which are the defect detection results of the transmission line insulator image.

[0019] Optionally, the DWD loss function is combined with the DIoU loss function and the NWD loss function for supervised learning of the network, thereby improving the model's detection accuracy for both large and small targets.

[0020] Optionally, before inputting the transmission line insulator image into the pre-trained transmission line insulator image detection model, the method further includes the step of training the transmission line insulator image detection model:

[0021] S1) Collect images of insulators of transmission lines and attach labels to them to construct an image dataset of insulators of transmission lines;

[0022] S2) Construct an image detection model for power transmission lines;

[0023] S3) The transmission line insulator image detection model is trained using preset training parameters, and finally a pre-trained transmission line insulator image detection model is obtained.

[0024] Optionally, the preset training parameters are: an initial learning rate of 0.01, a standard SGD optimizer training model with a momentum of 0.937, and weight decay always set to 5 × 10⁻⁶. -4 A total of 200 batches were trained.

[0025] Furthermore, the present invention also provides a method for inspecting power transmission lines by drone in complex scenarios, including a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the steps of the deep learning-based power transmission line insulator image detection method.

[0026] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program for execution by a computer device to implement the steps of the method for inspecting transmission line insulators by unmanned aerial vehicles in complex scenarios.

[0027] Compared with the prior art, the present invention has the following advantages: the present invention improves the accuracy of insulator detection in drone aerial images of power transmission line insulators with low computational resource consumption, and can be applied to the daily inspection of power transmission line operation status by drone autonomous inspection. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the implementation of the present invention;

[0029] Figure 2 This is a schematic diagram of the network structure of the image detection model for transmission line insulators in an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the RFPN-RCSP structure in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the structure of the CoordH coordinate detection head in an embodiment of the present invention; Detailed Implementation

[0032] This example provides a method for inspecting power transmission line insulators using drones in complex scenarios. The process is as follows: Figure 1 As shown, the process includes acquiring images of transmission line insulators to be detected, constructing a transmission line insulator image detection model, pre-training the transmission line insulator image detection model, and inputting the transmission line insulator images into the pre-trained transmission line insulator image detection model to obtain the detection results of the insulators to be detected. The specific steps are as follows: S1) Acquire transmission line insulator image samples and attach labels to them to construct a transmission line insulator image dataset; In this embodiment, a transmission line insulator image dataset is constructed, which includes four types of insulators: glass insulators (GI), ceramic insulators (CI), broken insulators (BI), and flashing insulators (FI). The dataset is then divided into a training / validation set and a test set at a 7:3 ratio, and further divided into a training set and a validation set at an 8:2 ratio.

[0033] S2) Construct an image detection model for transmission line insulators;

[0034] S3) The transmission line insulator image detection model is trained using preset training parameters, and finally a pre-trained transmission line insulator image detection model is obtained.

[0035] Furthermore, in this embodiment, the preset training parameters in step S3) are: an initial learning rate of 0.01, a momentum of 0.937, and a standard SGD optimizer training model, with the weight decay always set to 5 × 10⁻⁶. -4 A total of 200 batches were trained.

[0036] The network structure of the image detection model for transmission line insulators is as follows: Figure 2 As shown, it includes:

[0037] The backbone feature extraction network is used to extract features at different levels from the input image of transmission line insulators;

[0038] The Residual Feature Pyramid (RFPN) network is used to perform multi-scale, multi-level feature aggregation on the feature maps of the backbone feature extraction network, and the resulting feature maps are then fed into the detection head network.

[0039] The coordinate convolutional detection head network is used to detect feature maps after multi-scale feature aggregation to obtain the detection results of power transmission lines.

[0040] The backbone feature extraction network follows the CSP-Darknet53 structure of YOLOv5 and is used to extract different levels of features {C1, C2, C3, C4, C5} from the image of transmission line insulators. Feature map C5 is obtained from feature map C4 through the SPPF module.

[0041] like Figure 3 As shown, the Residual Feature Pyramid (RFPN) network consists of a 1×1 convolution module, a reparameterized feature fusion module (RCSP), and a concatenation module. The inputs to the residual feature pyramid are feature maps C2, C3, and C5. Feature map C5 is convolved with a 1×1 convolution to obtain feature map C5', which is then used as the input to the first reparameterized feature fusion module (RCSP_1) to obtain feature map R1. Feature map C3 is convolved with a 1×1 convolution to obtain feature map C3', and feature map C5 is obtained by upsampling feature map C5. up Feature maps C3' and C5 up The feature map C2 is concatenated with R1 and used as input to the second reparameterized feature fusion module RCSP_2 to obtain feature map R2; feature map C2 is convolved with 1×1 to obtain feature map C2', and feature map C3 is obtained by upsampling feature map C3. up Feature maps C2 and C3 up The feature map R3 is obtained by concatenating R2 with R2 and using the result as input to the third reparameterized feature fusion module RCSP_3. Feature map R3 is then passed through a 1×1 convolution module to obtain feature map R3', which is then passed as input to the fourth reparameterized feature fusion module RCSP_4 to obtain feature R4. Feature map R2 is passed through a 1×1 convolution to obtain feature map R2', and feature map R3 is downsampled to obtain feature map R3. down Feature maps R2' and R3 down R1 and R4 are concatenated and used as input to the fifth reparameterized feature fusion module RCSP_5 to obtain feature R5; R1 is convolved with a 1×1 convolution to obtain feature map R1', and feature map R2 is downsampled to obtain feature map R2. down Feature maps R1' and R2 down The feature R6 is obtained by concatenating R5 with the input of the sixth reparameterized feature fusion module RCSP_6. The reparameterized feature fusion module RCSP has different structures in the training and inference phases, which are denoted as the training phase structure RCSP respectively. train and training phase structure RCSP inference ;

[0042] Training Phase Structure RCSP trainThe multi-branch parallel structure includes the following steps: The feature map Fc with C input feature channels is divided into two branches for processing. Branch one consists of cascaded 1×1 convolutions and multi-branch parallel processing modules, with outputs F1, F2, F3, and F4 respectively. Branch two processes the feature map with C / 2 channels through a 1×1 convolution to obtain feature map F′. C / 2 The outputs F1, F2, F3, F4 and F′ of the two branches. C / 2 Perform the concatenation operation to obtain the output F. C The multi-branch parallel processing module includes four branches. The first branch of the multi-branch parallel processing module is a 1×1 convolution, which yields a feature map F1 with C / 8 channels. The second branch of the multi-branch parallel processing module is a 3×3 convolution, which yields a feature map F2 with C / 8 channels. The third branch of the multi-branch parallel processing module is a BN layer, which yields a feature map F3 with C / 8 channels. The fourth branch of the multi-branch parallel processing module is a convolutional sequence of 1×1 convolution and 3×3 convolution, which yields a feature map F4 with C / 8 channels.

[0043] Inference Stage Structure RCSP inference For a single-branch structure, the RCSP structure during the training phase... train The reparameterization process involves several steps: reparameterizing the BN layer of the multi-branch parallel module into a 3×3 convolution; reparameterizing the 1×1 convolution into a 3×3 convolution; reparameterizing the serial sequence convolutional module constructed from the 1×1 and 3×3 convolutions into a 3×3 convolutional module; then reparameterizing the 3×3 convolutional modules on the four branches with C / 8 channels into 3×3 convolutional modules with C / 2 channels; and then reparameterizing them together with the 1×1 convolutional module in branch one into a 3×3 convolutional module. Next, reparameterizing the 1×1 convolutional module in branch two into a 3×3 convolutional module; finally, reparameterizing the 3×3 convolutional modules in branch one and branch two into a single-branch structure of 3×3 convolutional modules, resulting in an output feature map F with C channels. CThe structural reparameterization includes: fusion of convolutional and BN layers, conversion of 1×1 convolutional modules to 3×3 convolutional modules, conversion of BN layers to 3×3 convolutional modules, fusion of 1×1 and 3×3 convolutional modules, and multi-branch fusion. The process of fusing convolutional and BN layers is as follows: both convolutional and BN layers are linear operations, and these are fused into a single operator through linear operations. The process of converting a BN layer to a 3×3 convolutional module is as follows: first, a 3×3 convolutional layer is constructed; an identity mapping operation is performed on the constructed 3×3 convolutional layer to obtain a 3×3 convolutional layer that does not change the input and output feature maps; finally, it is fused with the BN layer to obtain the reparameterized 3×3 convolutional layer. The process of converting a 1×1 convolutional module to a 3×3 convolutional module is as follows: the padding of the 1×1 convolutional module is set to 1 to obtain the reparameterized 3×3 convolutional module. The process of fusing 1×1 and 3×3 convolutional modules is as follows: first, in order to preserve the input and output features... Figure 1 To achieve this, the padding of the 1×1 convolutional module is set to 0, and the padding of the 3×3 convolutional module is set to 1. The paths of the 1×1 and 3×3 convolutional modules are consistent when performing convolution operations by sliding left. Finally, the 1×1 convolutional kernel is added to the middle of the 3×3 convolutional kernel to complete the fusion of the 1×1 and 3×3 convolutions, resulting in the reparameterized 3×3 convolutional module. The multi-branch fusion process is as follows: the parameters of the reparameterized 3×3 convolutional modules in each branch are added together to obtain a 3×3 convolutional module that is fused from multiple branches into a single branch.

[0044] like Figure 4 As shown, the coordinate convolutional detection head CoordH contains a 3×3 coordinate convolution module and a 1×1 convolution module. Input features Ri (i = 4, 5, 6) with C channels are processed by the 3×3 coordinate convolution module, introducing spatial information into the feature map. This adds two channels to the feature map, one representing the x-coordinate and one representing the y-coordinate, resulting in a feature map F with C+2 channels. C+2 Feature map F C+2 Then, a 1×1 convolution module is used to obtain the category score, confidence score, and bounding box position of the target object, which is the defect detection result of the transmission line insulator image. Taking the initial input of the entire network model as 460×460 as an example, it is divided into three detection branches. When Ri (i=4), the feature input is the first detection head of CoordH, which detects the 80×80 feature map to obtain the detection result of the insulator. When Ri (i=5), the feature input is the second detection head of CoordH, which detects the 40×40 feature map to obtain the detection result of the substation equipment. When Ri (i=6), the feature input is the third detection head of CoordH, which detects the 20×20 feature map to obtain the detection result of the substation equipment.

[0045] The DWD loss function, combined with the DIoU loss function and the NWD loss function, for supervised learning of the network, also includes at least the following steps:

[0046] S310. Calculate the confidence level of the DIoU loss function, using the predicted bounding box BB as an example. pred And real frame BB gt The center point is used to obtain the prediction box BB. pred And real frame BB gt Euclidean distance d D and containing prediction boxes BB pred And real frame BB gt The minimum closure region diagonal distance c; the intersection-union ratio expression of DIoU is:

[0047]

[0048] Where IoU represents the predicted bounding box BB pred And real frame BB gt The intersection-union ratio; the confidence level of the DIoU loss function is:

[0049] L DIoU =1-DIoU, (2)

[0050] S320. Calculate the prediction box BB pred And real frame BB gt The two-dimensional Gaussian distribution predicts the bounding box BB. pred =(c x ,c y ,w,h), (c x ,c y w and h represent the center coordinates, width, and height, respectively, to construct the prediction bounding box BB. pred The equation of the inscribed ellipse is expressed as follows:

[0051]

[0052] Among them, (u x u y ) represents the coordinates of the center of the ellipse, σ x , σ y Let μ be the length of the semi-axis along the x and y axes; therefore, μ x =c x μ x =c x , Predicted Box (BB) pred The probability density function of the two-dimensional Gaussian distribution is:

[0053]

[0054] Where x, μ, and ∑ represent the coordinates (x, y), mean, and covariance of the Gaussian distribution. When:

[0055] (x-μ) T ∑ -1 (x-μ)=1, (5)

[0056] At that time, the predicted bounding box BB pred The equation of the inscribed ellipse is a density profile of a two-dimensional Gaussian distribution, which can be expressed as:

[0057]

[0058] Similarly, the true frame (BB) can be obtained. gt Two-dimensional Gaussian distribution:

[0059]

[0060] S330. Calculate the confidence level of the NWD loss function and predict the bounding box BB. pred And real frame BB gt The intersection-union ratio is the predicted box BB. pred And real frame BB gt The distance from a two-dimensional Gaussian distribution is used to calculate the predicted bounding box (BB) using Wasserstein distance. pred And real frame BB gt The two-dimensional Gaussian distribution distance is calculated using the following formula:

[0061]

[0062] Simplified to:

[0063]

[0064] Among them, ||·|| F It is the F-norm;

[0065] Introducing prediction boxes (BB) pred = (cx1, cy1, w1, h1) and the true bounding box BB gt = (cx2, cy2, w2, h2), we get:

[0066]

[0067] The obtained After performing exponential normalization, the confidence level of the NWD loss function is obtained, expressed as:

[0068]

[0069] Where q is a constant;

[0070] S340. Calculate the confidence level of the DWD loss function. Based on the distribution ratio of small targets to medium and large targets in the transmission line insulator image dataset, set the proportion coefficient p, and use the L obtained in S310. DIoU L obtained from S330 NWD The confidence level of the DWD loss function is obtained by adding the proportional coefficients together, and the expression is:

[0071] L DWD = (1-p)*L NWD +p*L DIoU (12)

[0072] To verify the high-precision detection performance of the proposed method, we compared its detection accuracy with that of two-stage models (EfficientDet-D1 and Faster-rcnn), one-stage detection models (SSD, YOLOv8, YOLOv7, YOLOX-s, YOLOv5-s and CenterNet), anchorfree detection model (FCOS), and Transformer detection models (DETR and Swin). The evaluation metrics used were detection accuracy mAP@0.5, number of parameters, and detection speed FPS. The experimental results are shown in Table 1.

[0073] Table 1. Experimental results comparing the present invention with other algorithms on this dataset.

[0074]

[0075] In Table 1, mAP@0.5(%) represents the average accuracy of the four types of substation equipment when the IoU threshold is 0.5. According to Table 1, the mAP@0.5 of the method in this example is 93.2%, which is the best performance among all methods, exceeding the second-best performing YOLOv7-s by 3.7%. This method achieves high-precision detection results with only a small increase in the number of parameters and computational load, and the detection speed for a single image can reach 131.58 frames per second, meeting the requirements of real-time detection.

[0076] To better verify the effects of the residual feature pyramid RFPN-RCSP structure, the coordinate convolution detection head CoordH, and the DWD loss function in this example method, these structures were added to YOLOv5 for experiments. The experimental results are shown in Table 2. It can be seen that each part of this method contributes to improving the detection accuracy of transmission line insulators.

[0077] Table 2. Experimental results of ablation of various structures proposed in this invention.

[0078]

[0079] To further validate the role of the nearest neighbor feature pyramid (RFPN) in this example method, RFPN was compared with mainstream feature pyramid structures FPN, PAN, and GFPN. The experimental results are shown in Table 3. It can be seen that RFPN has the highest detection accuracy, while reducing the number of parameters and FLOPs by 33.9% and 14.2%, respectively, compared to GFPN.

[0080] Table 3. Experimental results comparing the RFPNS proposed in this invention with other characteristic pyramid structures.

[0081]

[0082] To verify the generalization ability of the proposed method, tests were conducted on the publicly available insulator dataset (SFID). The SFID dataset includes two defect classes: insulator and bracken piece, with a total of 6860 images, including 3841 training images, 960 validation images, and 2058 test images. The experimental results are shown in Table 4. The detection network model proposed in this invention also exhibits excellent performance on other insulator datasets, verifying the generalization ability of the proposed method.

[0083] Table 4: Generalization experiments with other algorithms on the public insulator dataset SFUD

[0084] Methods Year Backbone mAP@0.5 (%) mAP@0.5:.95(%) FasterRCNN 2016 Resnet50 98.4 80.1 SSD 2016 VGG16 97.5 65.2 MaskRCNN 2017 Resnet50 98.4 86.1 Retinanet 2017 Resnet-101 87.5 71.8 Centernet 2019 Resnet50 98.4 86 Efficientdet 2020 EfficientNet-D0 88.6 62.1 Swin-Transformer 2021 Swin-T 99 86.3 YOLOX 2021 Darknet 99.3 86 YOLOv5 2021 Darknet 99.2 86.2 FINet

[31] 2022 Darknet 99.4 86.4 YOLOv7 2022 Darknet 99.2 85.3 YOLOv8 2022 Darknet 99.1 85.6 Ours 2023 Darknet 99.5 90

[0085] Furthermore, this example also provides a method for inspecting transmission line insulators by drone in complex scenarios, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the steps of the deep learning-based transmission line insulator image detection method.

[0086] Furthermore, this example also provides a computer-readable storage medium storing a computer program for execution by a computer device to implement the steps of the method for inspecting transmission line insulators by unmanned aerial vehicles in complex scenarios.

[0087] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for inspecting power transmission line insulators using unmanned aerial vehicles (UAVs) in complex scenarios, characterized in that, The process includes using drones to collect images of transmission line insulators, inputting these images into a pre-trained transmission line insulator image detection model, and obtaining the detection results for the insulators. The transmission line insulator image detection model includes: The backbone feature extraction network follows the CSP-Darknet53 structure of YOLOv5 and is used to extract different levels of features {C1, C2, C3, C4, C5} from the image of transmission line insulators. Feature map C5 is obtained from feature map C4 through the SPPF module. Residual Feature Pyramid RFPN, consisting of 1 The system consists of a convolutional module, a reparameterized feature fusion module (RCSP), and a concatenation module; the input to the residual feature pyramid is feature maps C2, C3, and C5; feature map C5 is processed through 1...

1. Convolution yields feature map C5', which is then used as input to the first reparameterized feature fusion module RCSP_1 to obtain feature map R1; feature map C3 is then processed by 1...

1. Convolution yields feature map C3', and upsampling of feature map C5 yields feature map C5. up Feature maps C3' and C5 up The feature map C2 is concatenated with R1 and used as input to the second reparameterized feature fusion module RCSP_2 to obtain feature map R2; feature map C2 is then processed by 1...

1. Convolution yields feature map C2', and upsampling of feature map C3 yields feature map C3. up Feature maps C2 and C3 up Concatenating R2 with R3 and using it as input to the third reparameterized feature fusion module RCSP_3 yields feature map R3; feature map R3 is then processed through 1... The first convolutional module produces feature map R3', which is then used as input to the fourth reparameterized feature fusion module RCSP_4 to obtain feature R4; feature map R2 is processed by 1... Convolution 1 yields feature map R2', and downsampling feature map R3 yields feature map R3. down Feature maps R2' and R3 down R1 is concatenated with R4 and used as input to the fifth reparameterized feature fusion module RCSP_5 to obtain feature R5; R1 is processed through 1 Convolution 1 yields feature map R1', and downsampling feature map R2 yields feature map R2. down Feature maps R1' and R2 down The feature R6 is obtained by concatenating R5 with the input of the sixth reparameterized feature fusion module RCSP_6. The reparameterized feature fusion module RCSP has different structures in the training and inference phases, which are denoted as the training phase structure RCSPtrain and the inference phase structure RCSPinference, respectively. The spatial convolutional detection head CoordH, cascaded with 3 3-axis convolution and 1 1. Convolution module: Based on the output features R4, R5 and R6 of the residual pyramid structure, detection is performed to obtain the detection results of the transmission line insulator; When training the transmission line insulator image detection model, the DWD loss function is used to supervise network learning. The DWD loss function combines the DIoU loss function and the NWD loss function. A proportionality coefficient p is set based on the distribution ratio of small targets to medium-to-large targets in the transmission line insulator image dataset. The confidence scores of the DIoU loss function and the NWD loss function are weighted according to this proportionality coefficient to obtain the confidence score of the DWD loss function, which is expressed as follows: ,(1) in, This represents the confidence level of the DIoU loss function. This represents the confidence level of the NWD loss function. This represents the proportional coefficient set based on the distribution ratio of the small targets to the medium and large targets.

2. The method for inspecting power transmission line insulators by unmanned aerial vehicle (UAV) in complex scenarios according to claim 1, characterized in that, The reparameterized feature fusion module RCSP has different structures in the training and inference phases, denoted as RCSPtrain (training phase) and RCSPinference (inference phase), respectively. The training phase structure RCSPtrain is a multi-branch structure, dividing the input feature map Fc with C feature channels into two branches for processing. Branch one consists of cascaded 1×1 convolutions and multi-branch parallel processing modules. The outputs of branch one are... , and ; Branch Road 2 will have the following number of channels: The feature map is obtained by 1×1 convolution. ; Output of the two branches , and Perform the concatenation operation to obtain the output. The multi-branch parallel processing module includes four branches, with the first branch of the multi-branch parallel processing module being 1. One convolution yields a channel number of Feature map The second branch of the multi-branch parallel processing module is 3.

3. Convolution, resulting in the number of channels is Feature map The third branch of the multi-branch parallel processing module is a BN layer, resulting in a channel number of... Feature map The fourth branch of the multi-branch parallel processing module is 1. 1 convolution and 3 3. A convolutional sequence of convolutions yields a sequence with a number of channels. Feature map ; The inference stage structure RCSPinference is a single-branch structure, obtained by reparameterizing the training stage structure RCSPtrain. The specific reparameterization process includes: First, the BN layer of the multi-branch parallel processing module is reparameterized to 3.

3. Convolution, turning 1 1. Convolution reparameterization to 3 3. Convolution, turning 1 1 convolution and 3 The serial sequence convolutional module constructed by 3 convolutions is reparameterized to 3.

3. Convolutional module, then convert the number of channels to... 3 of the four branch roads 3. The convolution module is reparameterized to have a channel count of 1. 3 3 convolutional modules, and then 1 from branch 1. 1 convolutional module reparameterized to 3 3. Convolutional module; Second, convert 1 in branch 2... 1 convolutional module reparameterized to 3 3. Convolutional module; finally, the 3 in branch one and branch two... The convolutional module was reparameterized to 3.

3. A single-branch structure of the convolutional module yields an output feature map with C channels. .

3. The method for inspecting power transmission line insulators by unmanned aerial vehicle (UAV) in complex scenarios according to claim 1, characterized in that, The confidence scores of the DIoU loss function and the NWD loss function in the DWD loss function are determined through the following steps: S310. Calculate the confidence level of the DIoU loss function, using the predicted bounding box as an example. and real frame The center point is used to obtain the prediction box. and real frame European distance and containing prediction boxes and real frame The minimum closure region diagonal distance c; the intersection-union ratio expression of DIoU is: ,(2) Where IoU represents the predicted bounding box and real frame The intersection-union ratio; the confidence expression for the DIoU loss function is: ,(3) S320. Calculate the prediction box and real frame The two-dimensional Gaussian distribution predicts the bounding box. = , w and Let the center coordinates, width, and height of the prediction box be represented, respectively, and the prediction box be constructed. The equation of the inscribed ellipse is expressed as follows: + = 1,(4) in, The coordinates of the center of the ellipse are: This represents the length of the semi-axis along the x and y axes; = , = , = , = Prediction box The probability density function of the two-dimensional Gaussian distribution satisfies: = 1,(5) Where, x and Coordinates representing a Gaussian distribution The mean and covariance of the Gaussian distribution; the aforementioned prediction box The center x-coordinate, center y-coordinate, width, and height are denoted as follows: , Prediction box The equation of the inscribed ellipse is a density profile of a two-dimensional Gaussian distribution, which can be expressed as: , ,(6) Similarly, the true bounding box can be obtained. A two-dimensional Gaussian distribution; S330. Calculate the confidence level of the NWD loss function and predict the bounding box. and real frame The cross-union ratio is the prediction box and real frame The distance from a two-dimensional Gaussian distribution is used to calculate the predicted bounding box using Wasserstein distance. and real frame The two-dimensional Gaussian distribution distance is introduced into the true bounding box. ,in, These represent the x-coordinate and y-coordinate of the center of the true bounding box, respectively. Let these represent the width and height of the actual bounding box, respectively; the calculation formula is: = ,(7) in, It is a 2-norm; The obtained After performing exponential normalization, the confidence level of the NWD loss function is obtained, expressed as: = exp ,(8) Where q is a constant.

4. A method for inspecting power transmission line insulators by unmanned aerial vehicle (UAV) in complex scenarios, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the steps of the method for inspecting power line insulators in complex scenarios as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a computer device to implement the steps of the method for inspecting power line insulators under complex scenarios as described in any one of claims 1 to 3.