Improved YOLO11n lightweight insulator defect detection method deployed in embedded equipment

Abstract

An improved YOLO11n lightweight insulator defect detection network deployed in embedded equipment solves the problems of low insulator defect detection precision, high deployment cost and the like based on a convolutional neural network, and comprises the following steps: step 1, constructing an SHFConv to replace a fifth layer and a seventh layer in a backbone network; step 2, constructing an RPConv, and constructing a C3k2 RPConv through the RPConv to replace all C3k2 modules in the RPConv backbone network; 3, constructing an EGAS attention mechanism and adding the EGAS attention mechanism between the neck network and each detection head; and step 4, carrying out embedded deployment on the improved network.

Description

Technical Field

[0001] This invention relates to the field of image processing and target detection technology and embedded deployment technology in the field of power transmission line equipment inspection technology, specifically to an improved YOLO11n lightweight insulator defect detection method deployed in an embedded device. Background Technology

[0002] Insulators are widely used in power transmission lines, and their operating status directly affects the safety and stability of the lines. However, under long-term exposure to complex natural environments, insulators are prone to defects such as damage and flashover. Therefore, regular inspections to promptly detect and address insulator defects are crucial for ensuring the safe and stable operation of power transmission lines. Currently, power line inspections mainly rely on two methods: manual inspection and drone inspection. Traditional manual inspections require personnel to carry equipment in the field, which is greatly affected by geographical environment and weather conditions, posing not only safety risks but also low efficiency and high inspection costs. With the rapid development of drone technology, drone inspections equipped with image acquisition equipment are gradually becoming an important means of new power line inspections due to their advantages such as ease of operation, high mobility, and less susceptibility to terrain limitations.

[0003] With the continuous development of deep learning and computer vision technologies, and combined with existing hardware conditions, automatic detection of insulator defects in aerial images has become feasible. However, due to the limited computing and storage capabilities of drones and the relatively complex structure of detection models, current inspection processes typically require transmitting images collected by drones back to a data center for centralized processing. This back-end processing mode is time-consuming and lacks timely feedback, making it difficult to achieve real-time insulator detection. To meet the needs of real-time inspection, it is necessary to effectively reduce the number of parameters and computational load of the detection model while ensuring recognition accuracy, enabling it to be deployed directly on drone equipment to complete image processing and defect identification, thereby reducing false positives and false negatives and improving the efficiency and accuracy of insulator defect detection. Summary of the Invention

[0004] The purpose of this invention is to address the problem of poor real-time information performance and low efficiency caused by the complex models carried by UAVs in real-time insulator inspection tasks, and to provide an improved YOLO11n lightweight insulator defect detection method deployed on embedded devices.

[0005] To achieve the objectives of this invention, the following technical solution is proposed: an improved YOLO11n lightweight insulator defect detection method deployed in an embedded device includes:

[0006] Step 1: Construct SHFConv (Statistics High-Frequency Convolution) to replace layers 5 and 7 in the YOLO11n backbone network;

[0007] Step 2: Construct RPConv (Rolling Partial Convolution), and use RPConv to build C3k2_RPConv to replace all C3k2 (CSP Bottleneck with 2 Convolutions) modules in the RPConv backbone network;

[0008] Step 3: Construct the EGAS (Energy-gated Axial Sampling) attention mechanism and add it between the neck network and each detection head;

[0009] Step 4: Deploy the improved network onto the embedded device.

[0010] 2. In step 1, the backbone network sequentially includes: Layer 0: Conv layer (16 channels, 3×3 convolution, stride 2), Layer 1: Conv layer (32 channels, 3×3 convolution, stride 2), Layer 2: C3k2_RPConv module (64 channels), Layer 3: Conv layer (64 channels, 3×3 convolution, stride 2), Layer 4: C3k2_RPConv module (64 channels), Layer 5: SHFConv layer (128 channels, 3×3 convolution, stride 2), Layer 6: C3k2_RPConv module (128 channels), Layer 7: SHFConv layer (256 channels, 3×3 convolution, stride 2), Layer 8: C3k2_RPConv module (256 channels), Layer 9: SPPF module (256 channels, 5×5 cores), and Layer 10: C2PSA module (256 channels).

[0011] The SHFConv process convolves the input feature map with high-pass and low-pass convolution kernels respectively. The feature map after high-pass convolution kernel is then subjected to depthwise separable convolution, batch normalization and ReLU activation to obtain a high-pass feature map. The feature map after low-pass convolution kernel is then upsampled to obtain a low-pass feature map. The high-pass and low-pass feature maps are concatenated by channel dimension and then convolved with a 1×1 convolution to make the number of channels in the output feature map the same as the number of channels in the input feature map.

[0012] Furthermore, in step 2, RPConv is a novel lightweight convolutional algorithm. It first splits the input feature map along the channel direction, performs standard convolution on the first part of the channel feature map to obtain a convolutional branch, and applies the SE attention mechanism to the remaining channel feature map to obtain an attention branch. Then, the convolutional branch and the attention branch are swapped and concatenated, and the concatenated feature map is shuffled by channel. The SE attention mechanism performs global compression of the feature map in the channel dimension, learns the weights of each channel, and recalibrates them.

[0013] The C3k2_RPConv module is formed by replacing the Bottleneck of the C3k2 module with RPConv_Block;

[0014] The RPConv_Block is a residual connection between the input feature map and the feature map after RPConv convolution.

[0015] Furthermore, in step 3, the neck network sequentially includes: layer 11: upsampling layer (2x scaling, nearest interpolation); layer 12: concat layer fusing layer 11 and layer 6; layer 13: C3k2 layer (128 channels); layer 14: upsampling layer (2x scaling, nearest interpolation); layer 15: concat layer fusing layer 14 and layer 4; layer 16: C3k2_EGAS layer (… Layer 17: Conv layer (64 channels, 3×3 convolution, stride 2), Layer 18: Concat layer (merging layers 17 and 13), Layer 19: C3k2_EGAS layer (128 channels), Layer 20: Conv layer (64 channels, 3×3 convolution, stride 2), Layer 21: Concat layer (merging layers 20 and 10), Layer 22: C3k2_EGAS layer (256 channels).

[0016] The EGAS attention mechanism includes an energy gating phase, a dynamic sampling phase, and a local enhancement phase.

[0017] In the energy gating stage, the given input feature map is [B, C, H, W], where B, C, H, and W are the batch size, number of channels, height, and width, respectively. First, the input features are uniformly divided into g groups along the channel dimension, and the input features become [Bg, C / g, H, W]. For each feature map group, the standard deviation of height (std_h) and the standard deviation of width (std_w) are calculated. Then, 1×1 convolution is applied for channel compression, 1×3 convolution is used to obtain the vertical gate, and 3×1 convolution is used to obtain the horizontal gate. Finally, the horizontal and vertical gates are fused to obtain a two-dimensional gated map (gmap).

[0018] The dynamic sampling phase combines within-group statistics and geometric priors. Specifically, it first calculates the mean μ and standard deviation σ of the two-dimensional gating chart to characterize the overall intensity and local variations, i.e., within-group statistics; simultaneously, it introduces normalized spatial coordinates C. x and C y As location cues, i.e. geometric priors; the feature maps μ, σ, and C x and C y A composite feature map is formed by concatenating along the channel dimension, and then processed using 3×3DWConv to fuse the four features to obtain the predicted feature map [Δx, Δy, α]; where Δx and Δy are the pixel offsets of the basic sampling grid, and α is the confidence weight used to suppress unreliable regions; finally, EGAS applies PyTorch grid_sample to resample the predicted feature map; PyTorch grid_sample uses an offset grid with bilinear interpolation.

[0019] The local enhancement stage further refines the features of dynamic sampling; first, 3×3 convolution is used to enhance the local background, and then SE attention mechanism is used to capture inter-channel features; subsequently, 1×1 convolution is used to restore the number of channels to be consistent with the number of input channels, and residual connections are used to ensure the stability of training.

[0020] The head network includes three Detect modules that receive feature maps from layers 16, 19, and 22 respectively, extract features at three scales, and output target detection results.

[0021] This invention presents an improved YOLOv11n network optimized for deployment on embedded devices. Addressing the issues of low accuracy and high deployment cost in insulator defect detection using convolutional neural networks, this invention aims to ensure detection accuracy while reducing computational load and model parameters. It replaces layers 5 and 7 of the backbone network with SHFConv and replaces the C3k2 module of the backbone network with a lightweight convolutional RPConv module, C3k2_RPConv. To enhance the network's resistance to background noise in complex environments, an EGAS attention mechanism is designed and added between the neck network and the detection head. The improved network is then deployed on an embedded device. This invention optimizes the trade-off between accuracy, real-time performance, and lightweight design in existing insulator defect detection algorithms. The fewer model parameters in the improved network mean smaller storage requirements and a lower hardware deployment threshold, enabling embedded deployment and possessing significant engineering application value. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention;

[0023] Figure 2 The structure diagram of the improved YOLO11n;

[0024] Figure 3 Here is a diagram of the SHCConv structure;

[0025] Figure 4 Here is a diagram of the C3k2_RPConv structure;

[0026] Figure 5 Here is a structural diagram of RPConv and RPConv Block;

[0027] Figure 6 Here is a diagram of the SE structure;

[0028] Figure 7 Here is a diagram of the EGAS structure; Detailed Implementation

[0029] The present invention will be further described below with reference to the embodiments. It should be noted that the following embodiments are only used to further illustrate the present invention and should not be construed as limiting the scope of protection of the present invention. Some non-essential improvements and adjustments made by those skilled in the art based on the above-described invention are still within the scope of protection of the present invention.

[0030] This invention proposes an improved YOLO11n lightweight insulator defect detection network deployed on embedded devices, aiming to address the problems of low detection rate, high false negative rate, and poor real-time performance in existing insulator defect detection methods. Through systematic optimization of the network architecture and feature fusion mechanism, this network achieves significant improvements in computational efficiency, network detection accuracy, and robustness to complex environments.

[0031] In this embodiment, the dataset consists of publicly available online datasets and insulator images obtained through independent photography. The images include various states of insulators, such as normal insulators, flashover insulators, damaged insulators, and defective insulators. To ensure the model can accurately identify various defects, the dataset is first labeled and preprocessed: The LabelImg image annotation tool is used to label the insulators in the images with rectangular bounding boxes, and the labels for normal, flashover, damaged, and defective insulators are uniformly set as "normal_insulator", "flasher_insulator", "broken_insulator", and "defect_insulator", respectively. The annotation results are saved in a txt file, recording the bounding box coordinates of different target categories. Subsequently, data augmentation is performed on the labeled dataset: the data is divided into training, validation, and test sets in an 8:1:1 ratio. Based on this, the data is augmented using methods such as random flipping, adding noise, and averaging pixels. The added noise includes Gaussian noise and salt-and-pepper noise. Through these data augmentation operations, the robustness of the model in complex scenes and variable environments is improved, and the model's ability to identify insulators with different defect morphologies is enhanced. (Refer to...) Figure 1 An improved YOLOv11n network for deployment on embedded devices includes the following steps:

[0032] Step 1: Construct SHFConv (Statistics High-Frequency Convolution) to replace layers 5 and 7 in the YOLO11n backbone network;

[0033] The improved YOLO11n network model structure is as follows: Figure 2 As shown, it specifically includes: the backbone network, the neck network, and the head network.

[0034] The backbone network consists of the following layers in sequence: Layer 0: Conv layer (16 channels, 3×3 convolution, stride 2); Layer 1: Conv layer (32 channels, 3×3 convolution, stride 2); Layer 2: C3k2 module (64 channels); Layer 3: Conv layer (64 channels, 3×3 convolution, stride 2); Layer 4: C3k2 module (64 channels); Layer 5: SHFConv layer (128 channels, 3×3 convolution, stride 2); Layer 6: C3k2 module (128 channels); Layer 7: SHFConv layer (256 channels, 3×3 convolution, stride 2); Layer 8: C3k2 module (256 channels); Layer 9: SPPF module (256 channels, 5×5 cores); and Layer 10: C2PSA module (256 channels).

[0035] The neck network consists of the following layers in sequence: Layer 11: Upsampling layer (2x scaling, nearest interpolation); Layer 12: Concat layer fusing Layer 11 and Layer 6; Layer 13: C3k2 layer (128 channels); Layer 14: Upsampling layer (2x scaling, nearest interpolation); Layer 15: Concat layer fusing Layer 14 and Layer 4; Layer 16: C3k2 layer (64 channels); Layer 17: Conv layer (64 channels, 3×3 convolution, stride 2); Layer 18: Concat layer fusing Layer 17 and Layer 13; Layer 19: C3k2 layer (128 channels); Layer 20: Conv layer (64 channels, 3×3 convolution, stride 2); Layer 21: Concat layer fusing Layer 20 and Layer 10; and Layer 22: C3k2 layer (256 channels).

[0036] The head network contains three Detect modules that receive feature maps from layers 16, 19, and 22 respectively. These modules extract features at three scales: 80×80, 40×40, and 20×20, and output four types of targets: normal insulators, flashover insulators, damaged insulators, and defective insulators.

[0037] The SHFConv structure diagram is as follows: Figure 3 As shown, the input feature map is convolved with both high-pass and low-pass convolution kernels. The feature map after the high-pass convolution kernel is then processed with depthwise separable convolution, batch normalization, and ReLU activation. Simultaneously, the feature map after the low-pass convolution kernel undergoes upsampling. Finally, the two processed feature maps are concatenated, and the number of channels is adjusted to match the input feature map using a 1×1 convolution. Figure 1 The high-pass convolution is a 3×3 Laplacian high-pass convolution kernel, and all elements of the 3×3 low-pass convolution kernel are 1 / 9 mean convolution kernels.

[0038] Step 2: Construct RPConv, and use RPConv to build C3k2_RPConv to replace all C3k2 modules in the RPConv backbone network;

[0039] The C3k2_RPConv module mentioned above is as follows: Figure 4 As shown, the C3k2 module's Bottleneck is replaced by RPConv_Block; RPConv_Block is a residual connection between the input feature map and the feature map after convolution with RPConv; RPConv first splits the input feature map according to the number of channels, performs regular convolution on the first part of the channel feature map, processes the remaining channel feature map using the SE attention mechanism, then swaps the positions of the two and concatenates them, and finally performs channel shuffling; the structures of RPConv_Block and RPConv are as follows. Figure 5As shown; the SE attention mechanism is as follows Figure 6 As shown, the feature representation capability is improved by globally compressing the feature map along the channel dimension, learning the weight of each channel, and recalibrating it.

[0040] Step 3: Construct the EGAS attention mechanism and add it between the neck network and each detection head;

[0041] The EGAS attention mechanism comprises an energy gating phase, a dynamic sampling phase, and a local enhancement phase, and its structure is as follows: Figure 7 As shown;

[0042] In the energy gating stage, the given input feature map is [B, C, H, W], where B, C, H, and W are the batch size, number of channels, height, and width, respectively. First, the input features are uniformly divided into g groups along the channel dimension, and the input features become [Bg, C / g, H, W]. For each feature map group, the standard deviation of height (std_h) and the standard deviation of width (std_w) are calculated. Then, 1×1 convolution is applied for channel compression, 1×3 convolution is used to obtain the vertical gate, and 3×1 convolution is used to obtain the horizontal gate. Finally, the horizontal and vertical gates are fused to obtain a two-dimensional gated map (gmap).

[0043] The dynamic sampling phase combines within-group statistics and geometric priors. Specifically, it first calculates the mean μ and standard deviation σ of the two-dimensional gating chart to characterize the overall intensity and local variations, i.e., within-group statistics; simultaneously, it introduces normalized spatial coordinates C. x and C y As location cues, i.e. geometric priors; the feature maps μ, σ, and C x and C y A composite feature map is formed by concatenating along the channel dimension, and then processed using 3×3DWConv to fuse the four features to obtain the predicted feature map [Δx, Δy, α]; where Δx and Δy are the pixel offsets of the basic sampling grid, and α is the confidence weight used to suppress unreliable regions; finally, EGAS applies PyTorch grid_sample to resample the predicted feature map; PyTorch grid_sample uses an offset grid with bilinear interpolation.

[0044] The local enhancement stage further refines the features of dynamic sampling; first, 3×3 convolution is used to enhance the local background, and then SE attention mechanism is used to capture inter-channel features; subsequently, 1×1 convolution is used to restore the number of channels to be consistent with the number of input channels, and residual connections are used to ensure the stability of training.

[0045] Step 4: Deploy the improved network on the NVIDIA Jetson Orin Nano SUPER;

[0046] The specific deployment process is as follows: The insulator defect detection model based on the lightweight improved YOLO11n, trained on the server side, is exported as an ONNX format model file; an image acquisition and inference program is built in the embedded system, and a camera is connected via a USB interface to acquire images of transmission line insulators in real time; the inference output results are post-processed, including anchor frame decoding and confidence threshold filtering, to obtain the location and category information of four types of targets: normal insulators, flashover insulators, damaged insulators, and defective insulators; the detection results are superimposed on the original image in the form of bounding boxes and category labels to achieve real-time online detection of insulator defects.

[0047] This invention acquires images of insulator defects from different perspectives, expands these images using data augmentation methods to increase the diversity of the insulator dataset, and constructs an improved YOLO11n network to enhance the network's focus on insulator defect features, reducing false positives and false negatives. This enables rapid and accurate identification and localization of insulator defects in various complex background images. Furthermore, it can be deployed in embedded systems, solving the problems of low detection accuracy and slow real-time performance in insulator inspection.

Claims

1. An improved method for detecting defects in YOLO11n lightweight insulators deployed in embedded devices, characterized in that, Includes the following steps: Step 1: Construct SHFConv (Statistics High-Frequency Convolution) to replace layers 5 and 7 in the YOLO11n backbone network; Step 2: Construct RPConv (Rolling Partial Convolution), and use RPConv to build C3k2_RPConv to replace all C3k2 (CSP Bottleneck with 2 Convolutions) modules in the RPConv backbone network; Step 3: Construct the EGAS (Energy-gated Axial Sampling) attention mechanism and add it between the neck network and each detection head; Step 4: Deploy the improved network onto the embedded device.

2. The improved YOLO11n lightweight insulator defect detection method deployed in an embedded device according to claim 1, characterized in that: In step 1, the backbone network includes: Layer 0: Conv layer (16 channels, 3×3 convolution, stride 2); Layer 1: Conv layer (32 channels, 3×3 convolution, stride 2); Layer 2: C3k2_RPConv module (64 channels); Layer 3: Conv layer (64 channels, 3×3 convolution, stride 2); Layer 4: C3k2_RPConv module (64 channels); Layer 5: SHFConv layer (128 channels, 3×3 convolution, stride 2); Layer 6: C3k2_RPConv module (128 channels); Layer 7: SHFConv layer (256 channels, 3×3 convolution, stride 2); Layer 8: C3k2_RPConv module (256 channels); Layer 9: SPPF module (256 channels, 5×5 cores); and Layer 10: C2PSA module (256 channels). The SHFConv process convolves the input feature map with high-pass and low-pass convolution kernels respectively. The feature map after high-pass convolution kernel is then subjected to depthwise separable convolution, batch normalization and ReLU activation to obtain a high-pass feature map. The feature map after low-pass convolution kernel is then upsampled to obtain a low-pass feature map. The high-pass and low-pass feature maps are concatenated by channel dimension and then convolved with a 1×1 convolution to make the number of channels in the output feature map the same as the number of channels in the input feature map. Furthermore, in step 2, RPConv is a novel lightweight convolutional algorithm. It first splits the input feature map along the channel direction, performs standard convolution on the first part of the channel feature map to obtain a convolutional branch, and applies the SE attention mechanism to the remaining channel feature map to obtain an attention branch. Then, the convolutional branch and the attention branch are swapped and concatenated, and the concatenated feature map is shuffled by channel. The SE attention mechanism performs global compression of the feature map in the channel dimension, learns the weights of each channel, and recalibrates them. The C3k2_RPConv module is formed by replacing the Bottleneck of the C3k2 module with RPConv_Block; The RPConv_Block is a residual connection between the input feature map and the feature map after RPConv convolution.

3. The improved YOLO11n lightweight insulator defect detection network deployed in an embedded device according to claim 1, characterized in that: In step 3, the neck network includes: layer 11: upsampling layer (2x scaling, nearest interpolation); layer 12: concat layer fusing layer 11 and layer 6; layer 13: C3k2 layer (128 channels); layer 14: upsampling layer (2x scaling, nearest interpolation); layer 15: concat layer fusing layer 14 and layer 4; layer 16: C3k2_EGAS layer (64 channels); layer 17: Conv layer (64 channels, 3×3 convolution, stride 2); layer 18: concat layer fusing layer 17 and layer 13; layer 19: C3k2_EGAS layer (128 channels); layer 20: Conv layer (64 channels, 3×3 convolution, stride 2); layer 21: concat layer fusing layer 20 and layer 10; and layer 22: C3k2_EGAS layer (256 channels). The EGAS attention mechanism includes an energy gating phase, a dynamic sampling phase, and a local enhancement phase. In the energy gating stage, the given input feature map is [B, C, H, W], where B, C, H, and W are the batch size, number of channels, height, and width, respectively. First, the input features are uniformly divided into g groups along the channel dimension, and the input features become [Bg, C / g, H, W]. For each feature map group, the standard deviation of height (std_h) and the standard deviation of width (std_w) are calculated. Then, 1×1 convolution is applied for channel compression, 1×3 convolution is used to obtain the vertical gate, and 3×1 convolution is used to obtain the horizontal gate. Finally, the horizontal and vertical gates are fused to obtain a two-dimensional gated map (gmap). The dynamic sampling phase combines within-group statistics and geometric priors. Specifically, it first calculates the mean μ and standard deviation σ of the two-dimensional gating chart to characterize the overall intensity and local variations, i.e., within-group statistics; simultaneously, it introduces normalized spatial coordinates C. x and C y As location cues, i.e. geometric priors; the feature maps μ, σ, and C x and C y A composite feature map is formed by concatenating along the channel dimension, and then processed using 3×3DWConv to fuse the four features to obtain the predicted feature map [Δx, Δy, α]; where Δx and Δy are the pixel offsets of the basic sampling grid, and α is the confidence weight used to suppress unreliable regions; finally, EGAS applies PyTorch grid_sample to resample the predicted feature map; PyTorch grid_sample uses an offset grid with bilinear interpolation. The local enhancement stage further refines the features of dynamic sampling; first, 3×3 convolution is used to enhance the local background, and then SE attention mechanism is used to capture inter-channel features; subsequently, 1×1 convolution is used to restore the number of channels to be consistent with the number of input channels, and residual connections are used to ensure the stability of training. The head network includes three Detect modules that receive feature maps from layers 16, 19, and 22 respectively, extract features at three scales, and output target detection results.

4. The insulator defect detection method based on a lightweight improved YOLO11n network deployed on an embedded device according to claim 1, characterized in that: In step 4, the embedded device is an embedded platform with edge computing capabilities; The specific deployment process is as follows: The insulator defect detection model based on the lightweight improved YOLO11n obtained at the training end is converted into a model file suitable for embedded inference; an image acquisition and inference program is built in the embedded system, and an imaging device is connected via a wired or wireless interface to acquire insulator images or video frames and input into the model for inference; the inference output results are post-processed, including but not limited to candidate box decoding and confidence threshold filtering, to obtain the location and category information of normal insulators and at least one defective insulator target; the detection results are displayed on the original image in the form of bounding boxes and category labels or output to an external terminal in the form of data, and online detection of insulator defects is realized based on the category information.

Images