A method for unmanned aerial vehicle to inspect power transformation equipment under complex scene

Abstract

The application discloses a kind of complex scene under unmanned aerial vehicle inspection substation equipment method, mainly solve the problem of low detection accuracy and consume more computing resources of prior art.The method of the present application comprises collecting the infrared image of the substation equipment to be detected, inputting the infrared image of the substation equipment into the pre-trained substation equipment infrared image detection model to obtain the detection result of the substation equipment to be detected, and the substation equipment infrared image detection model comprises: a backbone feature extraction network, a multi-scale feature aggregation extraction network and a detection head network.The present application improves the detection accuracy of the substation equipment in aerial infrared image under low computing resource requirement, and can be applied to the daily inspection of unmanned aerial vehicle autonomous inspection of substation equipment operation state in substation and transmission line.

Description

Technical Field

[0001] This invention relates to an infrared image detection method for power equipment, specifically to a method for unmanned aerial vehicle (UAV) inspection of power equipment in complex scenarios. Background Technology

[0002] As a crucial infrastructure component of the power system, substation equipment plays a vital role in voltage and current transformation, as well as the reception and distribution of electrical energy. Substation equipment mainly includes surge arresters, instrument transformers, disconnect switches, and insulated posts. Operating outdoors for extended periods, these devices are characterized by high voltage and high current, making them prone to localized overheating faults, which seriously threaten the safety and stability of the power grid. Therefore, monitoring the thermal status of substation equipment is a fundamental task of routine inspections. Since substation equipment is often installed at high altitudes, using drones equipped with infrared thermal imagers to acquire aerial images of the equipment is an effective means of identifying abnormal surface heating. In recent years, numerous studies have investigated the visual tasks involved in the automated inspection of substation equipment. Different substation equipment exhibits different surface temperature distribution characteristics. Therefore, to achieve automated detection and diagnosis of the thermal status of substation equipment, the first step is to quickly and accurately identify and locate the equipment within infrared images.

[0003] In recent years, with the development of convolutional neural networks, several general-purpose object detection networks, such as Faster R-CNN, SSD, and YOLO, have been proposed and have achieved significant results in multi-scale object detection in visible light images. However, due to the differences between infrared and visible light images, these general-purpose object detection models are difficult to directly apply to the detection of substation equipment in infrared images. Researchers have improved the general-purpose detection models for different application scenarios. Although these methods have improved the object detection performance of UAV infrared aerial images to some extent, the detection of substation equipment in infrared aerial images still faces the following challenges:

[0004] (1) Uneven spatial distribution of targets. From the perspective of UAVs, power equipment is often concentrated in the central area of ​​the image. This results in dense targets in some areas, which may obscure each other.

[0005] (2) Changing UAV perspectives. The appearance of the same type of power equipment differs under different UAV perspectives; conversely, different types of power equipment appear extremely similar under specific UAV perspectives. This problem of large intra-category differences and small inter-category differences makes it difficult for the model to effectively distinguish targets.

[0006] (3) Changing target scale. Due to changes in flight altitude, the scale of the same type of target in UAV images changes drastically. Moreover, large and small targets may coexist in the same image. The model needs to be able to detect targets with changing scales.

[0007] (4) Limited onboard resources. Due to limitations in load-bearing capacity, power consumption, and cost, the computing platform resources onboard to UAVs are extremely limited. Deploying the detection model on such a resource-constrained computing platform requires reducing the model's parameters, computational complexity, and inference time while ensuring detection accuracy.

[0008] Existing methods, under the aforementioned challenges, cannot simultaneously achieve low computational resource consumption and fast, accurate detection of substation equipment from aerial infrared images. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a method for inspecting substation equipment by drone in complex scenarios, which addresses the above-mentioned problems of the prior art. The present invention can solve the problem that the infrared image detection accuracy of drones for inspecting substation equipment is not high when the existing methods are difficult to maintain with low computing resource consumption.

[0010] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0011] A method for inspecting substation equipment using unmanned aerial vehicles (UAVs) in complex scenarios includes acquiring infrared images of the substation equipment to be inspected, inputting the infrared images into a pre-trained infrared image detection model for substation equipment to obtain the detection result of the substation equipment. The infrared image detection model for substation equipment includes:

[0012] A backbone feature extraction network (GSCSPLnet) is used to extract features at different levels from the input infrared image of the substation equipment. The GSCSPLnet backbone feature extraction network has a ten-layer structure: the first layer is a Focus module; the second, fourth, sixth, and eighth layers are GSConv_BN_SiLU modules; the third, fifth, seventh, and tenth layers are GSCSP-L modules; and the ninth layer is a multi-level receptive field feature enhancement module (MRFFEM). The GSCSP-L module consists of cascaded GSCSP and a long-range feature capture attention mechanism (LDFC). The output of GSCSP is connected to the input of LDFC, and the output of LDFC is the output of the GSCSP-L module. The GSConv_BN_SiLU module consists of cascaded lightweight convolutions GSConv, BatchNorm2d, and the activation function SiLU. The GSCSPLnet backbone feature extraction network has a ten-layer structure: the first layer is a Focus module; the second, fourth, sixth, and eighth layers are GSConv_BN_SiLU modules; the third, fifth, seventh, and tenth layers are GSCSP-L modules; and the ninth layer is a multi-level receptive field feature enhancement module (MRFFEM). The GSCSP-L module consists of cascaded lightweight convolutions GSConv, BatchNorm2d, and the activation function SiLU. P adopts the basic structure of CSPNet. Each GSCSP consists of X GSResblocks and multiple GSRovs. GSResblocks are formed by adding the main branch and the residual branch. The main branch consists of two cascaded GSRov_BN_SiLU modules. The residual branch consists of sequentially cascaded depthwise separable convolutions DWSConv, BatchNorm2d, and the activation function SiLU. The feature maps extracted by the third, fifth, sixth, seventh, eighth, and tenth layers of the backbone feature extraction network GSCSPLnet are denoted as C2, C3, C4', C4, C5', and C5, respectively. The input of the multi-level receptive field feature enhancement module MRFFEM is the feature maps C4' and C5' extracted by the backbone feature extraction network. The output of MRFFEM is passed through GSCSP-L to obtain C5.

[0013] The Multi-Scale Feature Aggregation Network (MFEN) is used to perform multi-scale and multi-level feature aggregation on the feature maps C2, C3, C4, and C5 extracted by the backbone feature extraction network to obtain feature maps P3, P4, and P5.

[0014] The detection head network is used to detect features P3, P4, and P5 after multi-scale feature aggregation to obtain the detection results of the substation equipment.

[0015] Optionally, the Long-Range Feature Capture Attention (LDFC) mechanism includes the following steps: First, the input feature map Z is sequentially processed by an average pooling layer, a 1×1 convolution operation, and a linear activation function to obtain the feature map. and Then, the feature map The input is fed into an adaptive pooling layer in the vertical direction, a central feature aggregation layer, and a 1×1 convolution to capture long-range spatial features in the vertical direction. feature map The input is fed into a horizontal adaptive pooling layer, a central feature aggregation layer, and a 1×1 convolution to capture long-range spatial features in the horizontal direction. Finally, the Sigmoid function and upsampling operation are used to... and Convert to globally learnable weights α H (h) and α H (w).

[0016] Optionally, the multi-level receptive field feature enhancement module MRFFEM includes an auxiliary layer localization feature enhancement block ALFEB and a deep semantic pyramid pooling block DSPPB. ALFEB has one input, three data processing branches, and one output. The input of ALFEB is the feature map C4'. The first data processing branch of ALFEB consists of a cascaded 1×3 spatially separable convolution, a 3×1 spatially separable convolution, a dilated convolution with a kernel size of 3×3 and a dilation rate of 3, and a SimAM injection convolution. Branch 2 consists of cascaded 1×3 spatially separable convolutions, 3×1 spatially separable convolutions, 1×3 spatially separable convolutions, 3×1 spatially separable convolutions, a dilated convolution with a kernel size of 3×3 and an inflation rate of 5, and SimAM attention. Branch 3 consists of cascaded 1×3 spatially separable convolutions, 3×1 spatially separable convolutions, 1×3 spatially separable convolutions, 3×1 spatially separable convolutions, 1×3 spatially separable convolutions, 3×1 spatially separable convolutions, and a convolution with a kernel size of 3×3 and an inflation rate of 5, and SimAM attention. The first data processing branch consists of a 3×3 dilated convolution with a dilation rate of 7 and SimAM attention. Finally, the outputs of the three data processing branches are concatenated and then subjected to a 1×1 convolution to obtain the output f1. The DSPPB includes one input, three data processing branches, and one output. The input of the DSPPB is the feature map C5'. The first data processing branch of the DSPPB consists of a cascaded 1×1 GSConv, an adaptive pooling layer, a 3×3 dilated convolution with a dilation rate of 1, and SimAM attention. Branch 2 consists of a cascaded 3×3 GSConv, an adaptive pooling layer, a dilated convolution with a 3×3 kernel and a dilation rate of 3, and SimAM attention. Branch 3 consists of two cascaded 3×3 GSConv, an adaptive pooling layer, a dilated convolution with a 3×3 kernel and a dilation rate of 5, and SimAM attention. Finally, the outputs of the three data processing branches are concatenated and then processed through a 1×1 convolution to obtain output f2. f1 and f2 are concatenated and then processed through a 1×1 convolution to obtain the final output.

[0017] Optionally, the multi-scale feature aggregation network MFEN consists of three stacked fusion and recombination feature aggregation blocks CARE and one quasi-fusion and recombination feature aggregation block SCARE. The input of the first CARE is feature maps C5, C4, and C3. C3 is downsampled to obtain feature map N3 with the same scale as C4, and C5 is upsampled to obtain feature map L5 with the same scale as C4. N3, C4, and L5 are concatenated and then processed by GSCSP-L to obtain output P4_1. The input of the second CARE is P4_1, C3, and C2. C2 is downsampled to obtain feature map N2 with the same scale as C3, and P4_1 is upsampled to obtain feature map L4_1 with the same scale as C3. N2, C3, and L4_1 are concatenated and then processed by GSCSP-L to obtain output P4_1. -L outputs P3; the inputs to the third CARE are C5, P4_1, and P3. P3 is downsampled to obtain a feature map N3_1 with the same scale as P4_1. C5 is upsampled to obtain a feature map L5_1 with the same scale as P4_1. N3_1, P4_1, and L5_1 are concatenated and then passed through GSCSP-L to obtain the output P4; the inputs to SCARE are C5 and P4. C5 is passed through the feature auxiliary module AFSB to obtain a feature map S5 with the same size as C5. P4 is downsampled to obtain a feature map N4 with the same scale as C5. S5, N4, and C5 are concatenated and then passed through GSCSP-L to obtain the output P5; finally, P3, P4, and P5 are output to the first, second, and third detection heads of the detection head network, respectively.

[0018] Optionally, the feature assistance module AFSB consists of a main branch and a residual branch. The main branch contains two branches: the first branch is a cascaded 1×1 convolution and a Softmax function, and the second branch is an adaptive pooling layer. The outputs of the two branches are multiplied and then passed through a 1×1 convolution, LayerNorm, and ReLU activation function to obtain the output of the main branch. Finally, the output of the main branch is concatenated with the output of the residual branch to obtain the output of AFSB.

[0019] Optionally, the detection head module network includes a first detection head YOLO_Head1, a second detection head YOLO_Head2, and a third detection head YOLO_Head3; the first detection head YOLO_Head1 detects an 80×80 feature map to obtain the detection result of the power equipment; the second detection head YOLO_Head2 detects a 40×40 feature map to obtain the detection result of the power equipment; and the third detection head YOLO_Head3 detects a 20×20 feature map to obtain the detection result of the power equipment.

[0020] Optionally, before inputting the infrared image of the power equipment into the pre-trained infrared image detection model of the power equipment, the method further includes the step of training the infrared image detection model of the power equipment:

[0021] S1) Collect infrared image samples and attach labels to construct an infrared image dataset of substation equipment;

[0022] S2) Construct an infrared image detection model for substation equipment;

[0023] S3) The infrared image detection model of the substation equipment is trained using preset training parameters, and finally a pre-trained infrared image detection model of the substation equipment is obtained.

[0024] Optionally, the preset training parameters are: an initial learning rate of 0.01, a standard SGD optimizer training model with a momentum of 0.937, and weight decay always set to 5 × 10⁻⁶. -4 A total of 400 batches were trained.

[0025] In addition, the present invention also provides a method for inspecting substation equipment by drone in complex scenarios, including a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to perform the steps of the deep learning-based infrared image detection method for substation equipment.

[0026] In addition, the present invention provides a computer-readable storage medium storing a computer program for execution by a computer device to implement the steps of the method for inspecting substation equipment by unmanned aerial vehicles in complex scenarios.

[0027] Compared with the prior art, the present invention has the following advantages: the present invention improves the accuracy of power equipment detection in UAV aerial infrared images with low computing resource consumption, and can be applied to the daily inspection of the operating status of power equipment in substations and power lines by UAV autonomous inspection. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the implementation of the present invention;

[0029] Figure 2 This is a schematic diagram of the network structure of the infrared image detection model for power equipment in an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the structure of GSCSP-L in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the LDFC attention structure in an embodiment of the present invention;

[0032] Figure 5This is a schematic diagram of the structure of the multi-level receptive field feature enhancement module MRFFEM in an embodiment of the present invention. Detailed Implementation

[0033] This example provides a method for inspecting substation equipment using drones in complex scenarios. The process is as follows: Figure 1 As shown, the process includes acquiring infrared images of the substation equipment to be detected, constructing an infrared image detection model for the substation equipment, pre-training the infrared image detection model for the substation equipment, and inputting the infrared images of the substation equipment into the pre-trained infrared image detection model to obtain the detection result of the substation equipment to be detected. The specific steps are as follows:

[0034] S1) Collect infrared image samples and attach labels to construct an infrared image dataset for power equipment. In this embodiment, the infrared image dataset for power equipment includes seven types of power equipment: surge arrester 1 (PB1), surge arrester 2 (PB2), current transformer 1 (TA1), current transformer 2 (TA2), voltage transformer (TV), disconnector switch (DS), and support insulator (PI). The dataset is then divided into training and validation sets and a test set at a 9:1 ratio, and the training and validation sets are further divided into training and validation sets at a 9:1 ratio.

[0035] S2) Construct an infrared image detection model for substation equipment;

[0036] S3) The infrared image detection model of the substation equipment is trained using preset training parameters, and finally a pre-trained infrared image detection model of the substation equipment is obtained.

[0037] Furthermore, in this embodiment, the preset training parameters in step S3) are: an initial learning rate of 0.01, a momentum of 0.937 using the standard SGD optimizer, and a weight decay always set to 5 × 10⁻⁶. -4 A total of 400 batches were trained.

[0038] The loss function preset in this example can be selected as a consistent loss function for each category as needed. The preset loss function is: classification loss L. cls Confidence loss L obj and positioning loss L CIoU In this example, the total loss of the detection model is defined as follows:

[0039] LOSS=λ1L cls +λ2L obj +λ3L CIoU

[0040] Where L cls A binary cross-entropy function (BCE Loss) is used to calculate the classification loss between positive samples in the predicted bounding box and the ground truth bounding box. bjWe still use BCE Loss to calculate the confidence loss for all predicted bounding boxes containing the target. CIoU CIoU Loss (Complete Intersection over Union) is used to calculate the localization error between positive samples in the predicted bounding box and the ground truth bounding box.

[0041] The network structure of the infrared image detection model for the substation is as follows: Figure 2 As shown, it includes:

[0042] The backbone feature extraction network is used to extract features at different levels from the input infrared image of the substation equipment;

[0043] The Multi-Scale Feature Aggregation Network (MFEN) is used to perform multi-scale and multi-level feature aggregation on the feature maps of the backbone feature extraction network, and the resulting feature maps are fed into the detection head network.

[0044] The detection head network is used to detect the feature map after multi-scale feature aggregation to obtain the detection results of the substation equipment.

[0045] The backbone feature extraction network is used to achieve feature extraction at multiple levels. As an optional implementation, such as... Figure 2 As shown, in this embodiment, the backbone feature extraction network is GSCSPLnet, which has a ten-layer structure. The first layer is a Focus module, the second, fourth, sixth, and eighth layers are GSConv_BN_SiLU modules, the third, fifth, seventh, and tenth layers are GSCSP-L modules, and the ninth layer is a multi-level receptive field feature enhancement module MRFFEM. The feature maps obtained by the input image after passing through the third, fifth, sixth, seventh, eighth, and tenth layers of GSCSPLnet are denoted as C2, C3, C4', C4, C5', and C5, respectively. Among them, C4' and C5' are used as inputs to the multi-level receptive field feature enhancement module MRFFEM, and the output of MRFFEM is passed through the tenth layer GSCSP-L to obtain C5. Subsequently, C2, C3, C4, and C5 are fed into the multi-scale feature aggregation network MFEN for further feature processing.

[0046] like Figure 3As shown, the GSCSP-L module consists of cascaded GSCSP modules and the long-range feature capture attention mechanism LDFC. GSCSPLnet contains four GSCSP-L modules, located in layers 3, 5, 7, and 10. Each GSCSP-L module splits the input feature map into two parts. The left part, serving as the backbone, is processed by cascaded GSConv_BN_SiLU and X residual modules GSResblock. The values ​​in the four GSCSP-L modules are {3, 9, 9, 3}. The right part, similar to residual edges, is processed by a GSConv_BN_SiLU module and then directly concatenated with the backbone. After being cascaded with the LDFC attention mechanism, the output of GSCSP-L is obtained. The residual edges of GSResblock incorporate depthwise separable convolution DWSConv, and the backbone also uses a lightweight GSConv_BN_SiLU.

[0047] Long-range feature capture attention mechanism LDFC, such as Figure 4 As shown. The Long Distance Feature Capture Attention (LDFC) mechanism in this embodiment includes the following steps: First, input an original input feature. H represents the height of the feature map, W represents the width of the feature map, and C is the number of channels in the feature map. Downsampling average pooling layers F are sequentially applied to the feature map Z. AvgPool 1×1 convolution F Conv By applying the linear activation function s, a feature map of size C′×H / 2×W / 2 is obtained. and

[0048]

[0049] Then feature map Input to the vertical adaptive pooling layer Central Feature Aggregation Layer and 1×1 convolution F Conv This yields a long-range spatial feature map of size C×H / 2×1 that captures the vertical direction. The backbone of the central feature aggregation layer consists of spatially separable convolutions F... SSConv and batch normalization layer F BN The cascaded structure yields feature f1, which is then passed through the activation function F. ReLU We obtain feature f2, and finally multiply f1 and f2 element-wise to obtain the final central feature layer output Γ. CFA .

[0050]

[0051]

[0052] Γ CFA =F BN (F SSConv (x))⊙F ReLU (F BN (F SSconv (x)))

[0053] feature map Input to the horizontal adaptive pooling layer Central Feature Aggregation Layer and 1×1 convolution F Conv This yields a long-range spatial feature map of size C×1×W / 2 that captures the horizontal direction.

[0054]

[0055]

[0056] Finally, the Sigmoid function σ and the upsampling operation F are used. Upsample Will and Convert to globally learnable weights α H (h) and α W (w) is multiplied by the input feature map Z to obtain the final attention feature map Y. C (h, w).

[0057]

[0058]

[0059]

[0060] The multi-level receptive field feature enhancement module MRFFEM in GSCSPLnet, such as Figure 5As shown, in this embodiment, the multi-level receptive field feature enhancement module MRFFEM includes an auxiliary layer localization feature enhancement block ALFEB and a deep semantic pyramid pooling block DSPPB. ALFEB has three data processing branches. C4′ serves as the input to these three branches, with a size of (40, 40, 256). The first branch of ALFEB consists of a cascaded 1×3 spatially separable convolution, a 3×1 spatially separable convolution, a dilated convolution with a kernel size of 3×3 and an inflation rate of 3, and SimAM attention. The second branch consists of a cascaded 1×3 spatially separable convolution, a 3×1 spatially separable convolution, a 1×3 spatially separable convolution, a 3×1 spatially separable convolution, a dilated convolution with a kernel size of 3×3 and an inflation rate of 5, and SimAM attention. Branch 3 consists of a series of cascaded 1×3 spatially separable convolutions, 3×1 spatially separable convolutions, 1×3 spatially separable convolutions, 3×1 spatially separable convolutions, 1×3 spatially separable convolutions, 3×1 spatially separable convolutions, a dilated convolution with a kernel size of 3×3 and a dilation rate of 7, and SimAM attention. Finally, the outputs of the three branches are concatenated, and after a 1×1 convolution, the output f1 has a size of (20, 20, 512). DSPPB has three data processing branches. C5′ serves as the input to these three data processing branches, and its size is (20, 20, 512). DSPPB's first data processing branch consists of a series of cascaded 1×1 GSConv layers, an adaptive pooling layer, a dilated convolution with a kernel size of 3×3 and a dilation rate of 1, and SimAM attention. Branch 2 consists of a cascaded 3×3 GSConv layer, an adaptive pooling layer, a dilated convolution with a 3×3 kernel and a dilation rate of 3, and SimAM attention. Branch 3 consists of a cascaded two 3×3 GSConv layers, an adaptive pooling layer, a dilated convolution with a 3×3 kernel and a dilation rate of 5, and SimAM attention. Finally, the outputs of the three branches are concatenated and then concatenated to obtain an output f2 of size (20, 20, 512) after a 1×1 convolution. Concatenating f1 and f2 and then performing a 1×1 convolution yields the final output (20, 20, 512).

[0061] like Figure 2As shown, the multi-scale feature aggregation network MFEN in this embodiment consists of three stacked fusion and recombination feature aggregation blocks CARE and one quasi-fusion and recombination feature aggregation block SCARE. The inputs to the first CARE are C5, C4, and C3. C3 is downsampled to obtain a feature map N3 of the same scale as C4, and C5 is upsampled to obtain a feature map L5 of the same scale as C4. N3, C4, and L5 are concatenated and then processed by GSCSP-L to obtain the output P4_1. The inputs to the second CARE are P4_1, C3, and C2. C2 is downsampled to obtain a feature map N2 of the same scale as C3, and P4_1 is upsampled to obtain a feature map L4_1 of the same scale as C3. N2, C3, and L4_1 are concatenated and then processed by GSCSP-L to obtain the output P3. The inputs to the third CARE are C5, P4_1, and P3, P... 3. After downsampling, feature map N3_1 with the same scale as P4_1 is obtained. C5 is upsampled to obtain feature map L5_1 with the same scale as P4_1. N3_1, P4_1 and L5_1 are concatenated and then processed by GSCSP-L to obtain output P4. The input of SCARE is C5 and P4. C5 is processed by the feature auxiliary module AFSB to obtain feature map S5 with the same size as C5. P4 is downsampled to obtain feature map N4 with the same scale as C5. S5, N4 and C5 are concatenated and then processed by GSCSP-L to obtain output P5. Finally, P3, P4 and P5 are output to the first detection head, second detection head and third detection head of the detection head network, respectively.

[0062] The Multi-Scale Feature Aggregation Network (MFEN) incorporates a Feature Auxiliary Module (AFSB), such as... Figure 2 As shown, in this embodiment, the Feature Assisted Module (AFSB) consists of a main branch and a residual branch. The main branch contains two branches: the first branch is a cascaded 1×1 convolution and a Softmax function, and the second branch is an adaptive pooling layer. The outputs of the two branches are multiplied together and then passed through a 1×1 convolution, LayerNorm, and ReLU activation function to obtain the output of the main branch. Finally, the output of the main branch is concatenated with the output of the residual branch to obtain the output of AFSB.

[0063] Finally, the three outputs P3, P4, and P5 of the multi-scale feature aggregation network are fed into the detection head module network for final prediction. Figure 1As shown, in this embodiment, the detection head module network includes a first detection head YOLO_Head1, YOLO_Head2, and YOLO_Head3. The first detection head YOLO_Head1 detects an 80×80 feature map to obtain the detection result of the power equipment; the second detection head YOLO_Head2 detects a 40×40 feature map to obtain the detection result of the power equipment; and the third detection head YOLO_Head3 detects a 20×20 feature map to obtain the detection result of the power equipment.

[0064] To validate the high-precision detection performance of the proposed method, its detection accuracy was compared with that of two-stage models (EfficientDet-D1 and Faster-rcnn), one-stage detection models (YOLOv8, YOLOv7, YOLOX-s, YOLOv5-s, ConvNeXt and CenterNet), anchorfree detection model (FCOS), and Transformer detection models (DETR and Swin). The evaluation metric used was detection accuracy (mAP). 50 The experimental results, including parameters, computational cost (FLOPs), and detection speed (FPS), are shown in Table 1.

[0065] Table 1. Experimental results comparing the present invention with other algorithms on this dataset.

[0066]

[0067] In Table 1, mAP 50 (%) represents the average accuracy of the seven types of power equipment when the IoU threshold is 0.5. As shown in Table 1, the mAP of the method in this example is... 50 With a detection accuracy of 99.2%, this method is the best performing among all methods, outperforming the second-best performing YOLOX-s by 2.08%. This method achieves high detection accuracy with only a small increase in the number of parameters and computational cost, and can detect a single image at a speed of 105.3 frames per second, meeting the requirements for real-time detection.

[0068] To verify the effectiveness of the long-range feature capture attention mechanism LDFC in this example method, it was compared with mainstream attention modules: SimAM, SE, ECA and CBAM. The above attention modules were added to YOLOv5-s for experimentation, and the experimental results are shown in Table 2.

[0069] Table 2. Experimental results comparing the LDFC attention proposed in this invention with other attention methods.

[0070]

[0071] As shown in Table 2, for infrared images of power equipment, compared to other attention mechanisms, the Long-Range Feature Capture (LDFC) attention mechanism can guide the model to focus on important regions, improving the model's detection performance. On the other hand, for targets appearing at image edges, LDFC establishes a correlation between image boundary regions and important regions, capturing missing edge information over long distances, which also helps improve detection performance.

[0072] To verify the effectiveness of the multi-level receptive field feature enhancement module MRFFEM in this example method, MRFFEM is compared with the mainstream spatial pyramid pooling modules SPP, including SPPF, SimSPPF, ASPP, RFB, SPPCSPC, and SPPFCSPC. SPPF is the serial structure of SPP in YOLOv5, SimSPPF is the SPP structure in YOLOv6, ASPP adds dilated convolution to SPP, RFB uses multi-branch pooling with convolution kernels of different sizes, SPPCSPC is the SPP structure in YOLOv7, and SPPFCSPC is the serial structure of SPPCSPC. Experiments were conducted by adding these structures to YOLOv5, and the results are shown in Table 3.

[0073] Table 3. Comparative experimental results of the MRFFEM proposed in this invention with other spatial pyramid pooling modules.

[0074]

[0075]

[0076] As shown in Table 3, for infrared images of substation equipment, compared with other spatial pyramid pooling (SPP) modules, the introduction of the multi-level receptive field feature enhancement module (MRFFEM) can utilize convolutional layers of different sizes to aggregate and extract different fine-grained features of the target, enriching shape, texture, and structural features, thereby improving the model's detection accuracy for substation equipment.

[0077] To verify the effectiveness of the multi-scale feature aggregation network MFEN in this example method, CARE was split into only upsampling OLUP and only downsampling OLDW on YOLOv5-s, and feature auxiliary module AFSB was added to each for experimentation. The experimental results are shown in Table 4.

[0078] Table 4 Comparison of experimental results of MFEN modular ablation proposed in this invention

[0079]

[0080] As shown in Table 4, for infrared images of substation equipment, neither OLUP (only upsampling) nor OLDW (only downsampling) can improve the detection accuracy of the model. The CARE and SCARE functions introduced in the multi-scale feature aggregation network MFEN, which perform both upsampling and downsampling operations, can improve the network's detection performance. Introducing the feature auxiliary module AFSB improves the detection effect by supplementing useful semantic information and reducing the optimization difficulty of the model. In summary, the network model proposed in this example for detecting substation equipment under UAV inspection in complex scenarios features real-time performance and high detection accuracy. It can be deployed in UAV inspection engineering applications, enabling automatic identification and positioning of substation equipment infrared images. It can be applied to the daily inspection of substation equipment operation status by UAVs in substations and input lines.

[0081] Furthermore, this example also provides a method for inspecting substation equipment using unmanned aerial vehicles (UAVs) in complex scenarios, including an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the steps of the deep learning-based infrared image detection method for substation equipment.

[0082] In addition, this example also provides a computer-readable storage medium storing a computer program for execution by a computer device to implement the steps of the method for inspecting substation equipment by unmanned aerial vehicles in complex scenarios.

[0083] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for inspecting substation equipment using unmanned aerial vehicles (UAVs) in complex scenarios, characterized in that, The process includes using drones to collect infrared images of the electrical equipment being detected in substations, inputting these images into a pre-trained infrared image detection model for the electrical equipment, and obtaining the detection results. The steps for constructing the infrared image detection model for the electrical equipment include: A backbone feature extraction network, GSCSPLnet, is constructed. GSCSPLnet employs a ten-layer structure: the first layer is a Focus module; layers two through eight are GSConv_BN_SiLU modules; layers three through ten are GSCSP-L modules; and the ninth layer is a multi-level receptive field feature enhancement module, MRFFEM. The GSCSP-L module consists of cascaded GSCSP and a long-range feature capture attention mechanism (LDFC). The output of GSCSP is connected to the input of LDFC, and the output of LDFC is the output of the GSCSP-L module. The GSConv_BN_SiLU module consists of cascaded lightweight convolutions GSConv, BatchNorm2d, and the SiLU activation function. The GSCSP uses CSPN. The basic structure of et is as follows: each GSCSP consists of X GSResblocks and multiple GSRovs. A GSResblock is formed by adding a main branch and a residual branch. The main branch consists of two cascaded GSRov_BN_SiLU modules, and the residual branch consists of a cascaded depthwise separable convolution DWSConv, BatchNorm2d, and activation function SiLU. The feature maps extracted by the third, fifth, sixth, seventh, eighth, and tenth layers of the backbone feature extraction network GSCSPLnet are denoted as C2, C3, C4', C4, C5', and C5, respectively. The input of the multi-level receptive field feature enhancement module MRFFEM is the feature maps C4' and C5' extracted by the backbone feature extraction network, and the output of MRFFEM is obtained by passing through GSCSP-L to obtain C5. The Multi-Scale Feature Aggregation Network (MFEN) is used to perform multi-scale and multi-level feature aggregation on the feature maps C2, C3, C4 and C5 extracted by the backbone feature extraction network to obtain feature maps P3, P4 and P5. The detection head network is used to detect features P3, P4, and P5 after multi-scale feature aggregation to obtain the detection results of the substation equipment.

2. The method for inspecting substation equipment by unmanned aerial vehicle (UAV) in complex scenarios according to claim 1, characterized in that, The Long-Range Feature Capture Attention (LDFC) mechanism comprises the following steps: First, the input feature map Z is sequentially processed by an average pooling layer, a 1×1 convolution operation, and a linear activation function to obtain the feature map. and Then, the feature map The input is fed into an adaptive pooling layer in the vertical direction, a central feature aggregation layer, and a 1×1 convolution to capture long-range spatial features in the vertical direction. feature map The input is fed into a horizontal adaptive pooling layer, a central feature aggregation layer, and a 1×1 convolution to capture long-range spatial features in the horizontal direction. Finally, the Sigmoid function and upsampling operation are used to... and Convert to globally learnable weights α H (h) and α W (w).

3. The method for inspecting substation equipment by unmanned aerial vehicle (UAV) in complex scenarios according to claim 1, characterized in that, The multi-level receptive field feature enhancement module MRFFEM includes an auxiliary layer localization feature enhancement block (ALFEB) and a deep semantic pyramid pooling block (DSPPB). ALFEB has one input, three data processing branches, and one output. The input to ALFEB is the feature map C4'. ALFEB's first data processing branch consists of a cascaded 1×3 spatially separable convolution, a 3×1 spatially separable convolution, a dilated convolution with a kernel size of 3×3 and a dilation rate of 3, and SimAM attention. The second branch consists of a cascaded 1×3... Spatial separable convolution, 3×1 spatial separable convolution, 1×3 spatial separable convolution, 3×1 spatial separable convolution, dilated convolution with a kernel size of 3×3 and an inflation rate of 5, and SimAM attention are sequentially cascaded. Finally, the outputs of the three data processing branches are concatenated and then processed by a 1×1 convolution to obtain the output f1. The DSPPB contains one input, three data processing branches, and one output. The input of the DSPPB is the feature map C5'. The first data processing branch of the DSPPB consists of a cascaded 1×1 GSConv, an adaptive pooling layer, a 3×3 dilated convolution with a dilation rate of 1, and SimAM attention. The second branch consists of a cascaded 3×3 GSConv, an adaptive pooling layer, a 3×3 dilated convolution with a dilation rate of 3, and SimAM attention. The third branch consists of two cascaded 3×3 GSConv, an adaptive pooling layer, a 3×3 dilated convolution with a dilation rate of 5, and SimAM attention. Finally, the outputs of the three data processing branches are concatenated and then convolved with a 1×1 convolution to obtain output f2; f1 and f2 are concatenated and then convolved with a 1×1 convolution to obtain the final output.

4. The method for inspecting substation equipment by unmanned aerial vehicle (UAV) in complex scenarios according to claim 1, characterized in that, The Multi-Scale Feature Aggregation Network (MFEN) consists of three stacked fusion and recombination feature aggregation blocks (CARE) and one quasi-fusion and recombination feature aggregation block (SCARE). The first CARE takes feature maps C5, C4, and C3 as input. C3 is downsampled to obtain feature map N3 with the same scale as C4, and C5 is upsampled to obtain feature map L5 with the same scale as C4. N3, C4, and L5 are concatenated and then processed by GSCSP-L to obtain output P4_1. The second CARE takes P4_1, C3, and C2 as input. C2 is downsampled to obtain feature map N2 with the same scale as C3, and P4_1 is upsampled to obtain feature map L4_1 with the same scale as C3. N2, C3, and L4_1 are concatenated and then processed by GSCSP-L. The output P3 is obtained; the inputs of the third CARE are C5, P4_1 and P3. P3 is downsampled to obtain a feature map N3_1 with the same scale as P4_1. C5 is upsampled to obtain a feature map L5_1 with the same scale as P4_1. N3_1, P4_1 and L5_1 are concatenated and then passed through GSCSP-L to obtain the output P4; the inputs of SCARE are C5 and P4. C5 is passed through the feature auxiliary module AFSB to obtain a feature map S5 with the same size as C5. P4 is downsampled to obtain a feature map N4 with the same scale as C5. S5, N4 and C5 are concatenated and then passed through GSCSP-L to obtain the output P5; finally, P3, P4 and P5 are output to the first detection head, second detection head and third detection head of the detection head network respectively.

5. The method for inspecting substation equipment by unmanned aerial vehicle (UAV) in complex scenarios according to claim 1, characterized in that, The detection head network includes a first detection head YOLO_Head1, a second detection head YOLO_Head2, and a third detection head YOLO_Head3. The first detection head YOLO_Head1 detects an 80×80 feature map to obtain the detection result of the power equipment; the second detection head YOLO_Head2 detects a 40×40 feature map to obtain the detection result of the power equipment; and the third detection head YOLO_Head3 detects a 20×20 feature map to obtain the detection result of the power equipment.

6. The method for inspecting substation equipment by unmanned aerial vehicle (UAV) in complex scenarios according to claim 4, characterized in that, The Feature Assisted Module (AFSB) consists of a main branch and a residual branch. The main branch contains two branches: the first branch is a cascaded 1×1 convolution and a Softmax function, and the second branch is an adaptive pooling layer. The outputs of the two branches are multiplied, and then passed through a 1×1 convolution, LayerNorm, and ReLU activation function to obtain the output of the main branch. Finally, the output of the main branch is concatenated with the output of the residual branch to obtain the output of AFSB.

7. A method for inspecting substation equipment by unmanned aerial vehicle (UAV) in complex scenarios, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the steps of the method for inspecting substation equipment in complex scenarios as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a computer device to implement the steps of the method for inspecting substation equipment under complex scenarios as described in any one of claims 1 to 6.

Images