A power equipment detection method fusing neighborhood attention and deep convolution

Abstract

This invention discloses a power equipment detection method that integrates neighborhood attention and deep convolution, belonging to the fields of machine vision and power line inspection. The method includes: acquiring power line inspection images containing transmission and distribution equipment and labeling power equipment targets and defect targets in the images; introducing neighborhood attention and deep convolution, combining a convolutional neighborhood attention layer, Mask-RCNN, and a convolutional feedforward network to obtain a detection network for target detection in power line inspection images; updating the detection network parameters based on the labeled image dataset to obtain the trained detection network; and filtering the output results based on a confidence threshold to finally obtain the detection results of power equipment targets and defect targets in the power line inspection image. This invention achieves a balance between low computational cost, few parameters, low latency, and high task accuracy by optimizing the fusion mechanism of attention and convolution to achieve efficient interaction between global information and local features.

Description

Technical Field

[0001] This invention belongs to the field of machine vision and power line inspection, specifically relating to a power equipment detection method that integrates neighborhood attention and depth convolution. Technical Background Power line inspection is a core component of ensuring the safe and stable operation of power transmission and distribution networks. Intelligent methods such as drone aerial photography, line inspection robots, and intelligent substation monitoring have become the mainstream approach. These scenarios place stringent demands on machine vision technology: it must not only process high-resolution images of power equipment and accurately identify defects such as corrosion, spontaneous breakage, and broken strands in equipment such as poles, insulators, conductors, and fittings, but also meet the requirements for lightweight deployment at the edge and adapt to the low computational overhead and low latency operating standards of resource-constrained platforms such as drone embedded terminals and robot main control chips.

[0002] The two mainstream neural network architectures currently available have significant shortcomings in power line inspection applications: Convolutional Neural Networks (CNNs) rely on local receptive fields to extract features and possess efficient translational equivariance, but their ability to capture long-distance dependencies is limited. In power line inspection images, equipment components are often scattered, such as the correlation features between long-span transmission lines, towers, and insulators, which CNNs struggle to model effectively, easily leading to missed or false detections. Transformer-based network architectures achieve global information interaction through self-attention mechanisms, improving feature representation capabilities for complex scenes; however, the computational cost of global attention increases quadratically with image resolution, resulting in a large number of parameters and high inference latency. This characteristic makes it unsuitable for the real-time requirements of UAV inspections, hindering rapid detection of high-resolution images on embedded terminals.

[0003] Existing hybrid architectures that combine attention and convolution generally trade performance improvements for complex layer structures, which directly leads to a significant increase in detection latency and cannot meet the real-time requirements of power line inspection. Some simplified models, on the other hand, fail to fully leverage the complementary advantages of attention and convolution, making it difficult to achieve a balance between equipment detection accuracy and lightweight deployment.

[0004] Therefore, there is an urgent need for a backbone network architecture that combines high-resolution adaptability, power equipment detection accuracy, and lightweight characteristics, so as to reduce computational overhead and latency while improving the general performance of power equipment and defect detection and meeting the engineering application needs of power inspection scenarios. Summary of the Invention

[0005] To address the existing technical problems, this invention proposes a power equipment detection method that integrates neighborhood attention and depthwise convolution. It includes: S1. Collect power inspection images containing power transmission and distribution equipment, and label the power equipment targets and defect targets in the images to obtain an image dataset containing the labels; S2 introduces domain attention and depthwise convolution to construct a domain attention layer containing convolutions; S3 combines a domain attention layer with convolutions, Mask-RCNN, and introduces a convolutional feedforward network to construct a detection network for target detection in power line inspection images. S4, the labeled image dataset is preprocessed and then input into the detection network, and the detection network parameters are updated to obtain the trained detection network; S5 inputs the power inspection image to be detected into the trained detection network, and filters the output results based on the confidence threshold to finally obtain the detection results of power equipment targets and defect targets in the power inspection image.

[0006] Furthermore, in S1, the acquired power inspection images include power inspection images acquired by drone aerial photography, power inspection images acquired by line inspection robots, power inspection images acquired by power channel monitoring cameras, and power inspection images acquired by manual inspection.

[0007] Furthermore, in S1, the annotation includes the border and category corresponding to the target.

[0008] Furthermore, in S2, the specific formula for the convolutional domain attention layer is as follows: ; in, These are all factors used to control the weight ratio, where R is the receptive field size. For element-wise multiplication, for In the receptive field index The weights and neighborhood features at each location, where D represents the learnable convolutional weights and NAC represents the domain attention layer containing convolutions. For the first One sample in The location feature, where r is the receptive field index. For the s-th sample at position p, the receptive field index The eigenvector at that location.

[0009] Furthermore, the factor used to control the weight ratio is obtained by projecting the feature map samples and performing an outer product, specifically: ; in, and Both are weight matrices. The compression ratio is... The outer product in the dimensions of receptive field and channel. These are feature map samples after global pooling and compression ratio.

[0010] Further, in S3, the detection network for target detection in power line inspection images includes a backbone network, a neck, a region recommendation detection head, and a two-stage detection head; the backbone network contains one convolutional embedding layer and several network stages, each network stage containing multiple sequentially connected modules, each module consisting of a convolutional feedforward network and a convolutional domain attention layer; the neck is an FPN structure, used to receive the output features of each network stage of the backbone network; the region recommendation detection head and the two-stage detection head adopt the corresponding structure of Mask-RCNN.

[0011] Furthermore, the overall processing flow of the backbone network is as follows: For the input image, the image is first converted into a feature map through a convolutional embedding layer; Secondly, the feature maps are sequentially input into each network stage for processing, and the output of each network stage is used as an intermediate output; in each network stage, the first module also needs to downsample the input feature maps; Finally, the output of the last network stage is processed by a convolutional layer and input into the neck along with all intermediate outputs.

[0012] Furthermore, in the backbone network, the attention mechanism is not enabled in the first network stage, while the attention mechanism is enabled in the remaining network stages, and the size of the corresponding attention mechanism convolution kernel gradually increases according to the connection order of the network stages.

[0013] Furthermore, in S4, the preprocessing of the labeled image dataset includes randomly cropping, inverting, and scaling the labeled images in sequence.

[0014] Furthermore, in S4, the trained detection network needs to be deployed using different schemes based on the resource conditions of the actual application scenario, specifically: For servers with sufficient computing resources, the detection network is deployed directly on GPU servers, and a high-performance power equipment detection system is built by combining it with the TensorRT acceleration engine; For edge computing devices, the detection network is converted to ONNX format and deployed on embedded GPU or FPGA chips to achieve real-time online detection of inspection images.

[0015] This invention has the following significant advantages over the prior art: (1) The optimal balance between performance and lightweight: By innovatively designing a convolutional neighborhood attention (NAC) layer, neighborhood attention (NA) and depthwise separable convolution (DWC) are organically integrated. Neighborhood attention is used to introduce inductive bias and capture global information, while depthwise convolution is used to enhance local feature extraction. This significantly reduces computational overhead while achieving higher task accuracy. (2) Outstanding inference efficiency: The simple network architecture design avoids the latency loss caused by complex layer structure. At the same time, the computational load of the NAC layer increases linearly with the input resolution, which greatly reduces the inference latency compared with the global attention mechanism. (3) Efficient and intelligent fusion mechanism: The fusion attention is automatically generated through global pooling compression and projection transformation. While introducing global information, it reduces the additional computational overhead and gives full play to the complementary advantages of attention and convolution. Compared with the existing hybrid architecture, it achieves a better balance in terms of parameter quantity, multiplication and addition operation quantity and latency. Moreover, the network can be flexibly adapted to different resource conditions through various versions. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the method of the present invention.

[0017] Figure 2 This describes the computation flow for a neighborhood attention (NAC) layer containing convolutions.

[0018] Figure 3 This is a schematic diagram of the overall structure of the lightweight detection network proposed in the method of this invention.

[0019] Figure 4 The results are from power line inspection images. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0021] The overall process of the lightweight power equipment detection method proposed in this invention, which integrates neighborhood attention and depth convolution, is as follows: Figure 1 As shown, the process includes six main steps: data collection and labeling, construction of the NAC layer and detection network, network training, and inference prediction.

[0022] Step 1: Collect power line inspection images. Inspection images can be collected through drone aerial photography, line inspection robots, power line channel monitoring cameras, or manual inspection.

[0023] The core of this step is to acquire a power inspection image dataset covering multiple scenarios and types, providing sufficient and effective data support for subsequent model training.

[0024] The acquisition channels for inspection images need to comprehensively cover four methods: drone aerial photography, line inspection robot acquisition, power channel monitoring camera capture, and manual inspection photography. Drone aerial photography is mainly used to obtain panoramic images of high-altitude power equipment such as transmission lines and towers, covering equipment status in complex geographical environments such as mountains, plains, and hills. Line inspection robots are equipped with high-definition cameras and can move autonomously along transmission lines to collect close-up detailed images of key parts such as line joints and insulator strings. Power channel monitoring cameras enable all-weather real-time monitoring of equipment in fixed areas such as substations and distribution rooms, capturing image data under abnormal equipment conditions. Manual inspection is used for special areas that are difficult for drones and robots to reach, where maintenance personnel use handheld professional shooting equipment to collect accurate equipment images.

[0025] During image acquisition, image quality must be strictly controlled to ensure a resolution of at least 1920×1080, avoiding blurriness, overexposure, or underexposure that could negatively impact subsequent annotation and model training. Simultaneously, it is crucial to ensure the diversity of the dataset, covering images of power equipment under different seasons and weather conditions, such as equipment status in adverse weather conditions like rain, fog, and snow, as well as images of equipment in normal operation and with defects.

[0026] After the data acquisition is completed, all images are assigned a unique number, and an image acquisition archive is created. This archive records the device type, acquisition time, acquisition location, and environmental information corresponding to each image, providing a basis for subsequent data management and model analysis.

[0027] Step Two: Annotate key power equipment and defects in the inspection images. Professional power maintenance personnel and experts will annotate the key power equipment and defects to be inspected in the images using horizontal borders, generating an annotation file.

[0028] This step aims to add accurate annotation information to the collected inspection images, build a labeled training dataset, and guide the detection model to learn the characteristics of power equipment and defects.

[0029] The annotation work must be completed jointly by professionals and industry experts with more than 5 years of experience in power operation and maintenance to ensure the accuracy and authority of the annotation results. The annotation method adopts the horizontal border annotation method, that is, for each key power equipment and equipment defects in the image, the target area is selected with a rectangular border and the corresponding category label is added.

[0030] Before labeling, a unified labeling standard must be established to clarify the classification system for power equipment and defects. Power equipment categories cover common equipment types such as poles and towers, insulators, transformers, circuit breakers, and disconnect switches; equipment defect categories include typical fault types such as insulator contamination, broken conductor strands, equipment corrosion, and loose bolts.

[0031] During the annotation process, annotators need to review each inspection image, identify target objects within them, and use professional annotation tools to draw horizontal borders, ensuring that the borders accurately enclose the target area without any omissions or misalignments. If multiple targets exist in the same image, they must be selected and annotated separately for each target.

[0032] After initial annotation, a three-tiered review mechanism must be established to ensure annotation quality. The first tier involves self-checking by the annotators to verify border positions and label categories. The second tier involves cross-review by other experts in the same group to correct annotation deviations. The third tier involves final review by the project's technical lead, who randomly checks the annotated data to ensure an accuracy rate of over 99%. Once approved, the annotation information is associated with the corresponding image to generate an annotation file containing the image path, border coordinates, and category labels. The annotation file is stored in the mainstream XML or JSON format for easy reading and parsing by subsequent detection models.

[0033] Step 3: Construct a neighborhood attention (NAC) layer containing convolutions. By fusing neighborhood attention (NA) and depthwise convolution (DWC), and combining global pooling compression and projection transformation to generate fused attention, we can achieve the synergy of global information interaction and local feature extraction.

[0034] Suppose that the feature maps corresponding to the input batch of images are ,in This represents the number of samples, i.e., the total number of feature maps. Represents the number of channels. The number of representative positions (obtained from the width × height of the feature map). This invention uses... Indicates the first In the feature map sample, the th Features corresponding to the positions of each coordinate.

[0035] kernel size is Depthwise separable convolution (DWC) can be represented as: in, To feel the size of the wild, For element-wise multiplication, , respectively corresponding and In the field of perception index Weights and neighborhood features at each location For learnable convolution weights, This is the unfolded feature map.

[0036] Neighborhood attention (NA) is a new form of attention proposed in recent years. Compared with mechanisms such as global attention and Swing attention, the core advantage of NA is that it introduces an inductive bias through the neighborhood receptive field. This bias is similar to a convolution operation, which effectively reduces the amount of computation on the one hand, and enhances the locality and translation equivariance of the network layer on the other.

[0037] In practical applications, multi-head attention mechanisms are commonly used. Each attention head calculates the neighborhood attention separately. , can be represented as: in, , for the s-th sample in the receptive field index Attention weights at the location , for the s-th sample at position p, receptive field index The feature vector at the value of the input feature is derived from the input feature vector. Obtained by direct projection.

[0038] Weight of each attention head The calculation method is as follows: in, The query feature vector of the s-th sample at position p; , for the s-th sample at position p, receptive field index The key feature vector at the location; softmax is the key feature vector along the location. The normalization function for dimension, This is a vector dot product. Both Q and K are derived from the input features. Obtained by direct projection.

[0039] As can be seen from the above formula, NA and DWC have the same information aggregation process. Therefore, this invention proposes to integrate NA and DWC into a unified network layer structure to achieve stronger fitting ability. This layer structure is named the Neighborhood Attention (NAC) layer with convolution.

[0040] The specific formula for the NAC layer is as follows: in, These are all factors used to control the weight ratio, and are called fused attention. The whole constitutes a combined weight.

[0041] To avoid manually setting individual and This invention utilizes globally pooled and compressed features Obtain by performing projection transformation and outer product and The specific formula is as follows: in, and Both are weight matrices. The compression ratio is... This is the outer product over the receptive field and channel dimensions. It is obtained in this way. and The required computation is relatively small, and and It can not only control the weight ratio, but also introduce global information.

[0042] Based on the above structure, the input image is first transformed into a feature map through several pre-convolutional network layers and then transmitted to the NAC layer. The NAC layer uses a multi-head attention mechanism and convolutional operations to aggregate the neighborhood information of the feature map to obtain the processed feature map, which serves as the input data for the subsequent neck.

[0043] The overall process of the NAC layer is as follows: Figure 2 As shown, for the sake of simplicity, Figure 2 The standard operations such as linear projection and pooling are omitted. This involves a convolution operation. The specific steps are as follows: First, the feature map The process is divided into h attention heads for neighborhood attention calculation, with a channel count of [number missing]. .

[0044] Then, Perform global pooling to obtain Used to represent global information, The compression ratio can be further utilized. Compress to Through the Perform projection transformation and outer product to obtain fused attention. and In each attention head, the query tensor is obtained according to the neighborhood attention mechanism. Key tensors Value tensor and attention weight tensor .

[0045] Secondly, this paper introduces additional convolution weights. ,pass and and To integrate.

[0046] Finally, the merged weights Derability feature map Execution as Figure 2 The generalized convolution operation shown is used to concatenate the convolution results obtained from all attention heads to obtain the output feature map.

[0047] Step 4: Based on the NAC layer and Mask-RCNN network architecture from Step 3, construct a lightweight object detection network and provide different scale versions to adapt to different resource conditions.

[0048] The target detection network consists of a backbone, a neck region, a region-recommended detection head, and a two-stage detection head. For example... Figure 3 The diagram shown is a schematic representation of the overall structure of the detection network of this invention, with the region recommendation detection head being... Figure 3 In the RPN Head, the Region of Interest Pooling operation is... Figure 3 RoIAlign, the two-stage prediction head Figure 3 The Head in the module name. The parentheses after the module name represent its key parameters, for example... Indicates the kernel size as The number of output channels is Step size is For simplicity, the normalization and activation layers in the detection network have been omitted from the convolutional layers.

[0049] The backbone is built based on the general Transformer framework and NAC layers, and includes one convolutional embedding layer (Convs) and four network stages. It has a lightweight structure and is responsible for transforming the input image into multi-scale features.

[0050] Each network stage contains Each module The number of output channels for each module is The kernel size is Step size is Each module Internally, it contains an NAC layer and a convolutional feedforward network (CFFN), with the feedforward network placed before the attention module. This design allows the depth-separating convolutions in the feedforward network to achieve downsampling with minimal computational overhead.

[0051] The specific calculation process for the backbone of the detection network is as follows: In the main structure, the input image is first converted into a feature map through multiple convolutional layers (Convs); These feature maps are then sequentially fed into multiple network stages for processing. In each network stage, the first module downsamples the input feature maps (step size s = 2), and subsequent modules... The feature map in each module maintains a constant size (step size s = 1).

[0052] The output of the last network stage is processed by a convolutional layer and then input into the neck along with the outputs of all network stages.

[0053] The neck is an FPN structure, used to further enhance multi-scale features.

[0054] The region recommendation detection head and the two-stage detection head are consistent with Mask-RCNN. Based on the enhanced features output by FPN, the region recommendation detection head first predicts the regions where objects may exist. Then, the feature map of the corresponding region is pooled to a specific size through the region of interest pooling operation. Finally, the two-stage prediction head is fed into the output bounding box, category and instance mask of the target.

[0055] The depth and width of the proposed network are determined by the number of network stages within each stage. (Number of modules) (Number of channels) and The size of the convolution kernel in the attention mechanism is determined by this. This invention designs three versions with different sizes to suit different usage needs, and their parameter configurations are shown in Table 2.

[0056] The first stage of the network does not introduce an attention mechanism (corresponding to...). The kernel size gradually increases in subsequent network stages. For ease of design, the number of attention head channels in the proposed network layers is... The number of attention heads, h, is fixed at 16, therefore the number of channels can be determined by the number of attention heads. Sure.

[0057] Table 1. Parameter settings for various versions of the proposed lightweight classification network Step 5: Train the lightweight detector.

[0058] The core of this step is to train the lightweight detection network using the labeled dataset from step two, enabling the model to accurately detect power equipment and defects. Before training, the dataset needs to be preprocessed, including data augmentation operations such as random cropping, flipping, and scaling, to improve the model's generalization ability. Random cropping and flipping generate more diverse training samples by randomly cutting and horizontally / vertically flipping the images; scaling adjusts images of different sizes to the network input size to meet the model training requirements.

[0059] The training process is based on the PyTorch deep learning framework. Appropriate hyperparameters and optimizers are set during training to avoid overfitting. During training, a validation set is used to monitor model performance in real time. Gradient backpropagation is used to optimize the network parameters. Training stops when the validation set loss no longer decreases for 10 consecutive epochs, and the optimal model weights are saved. Simultaneously, the loss and accuracy curves are recorded during training to facilitate subsequent analysis of the model's training status and optimization of training strategies.

[0060] Step Six: Deployment of Lightweight Detectors and Inference Prediction.

[0061] This step focuses on the engineering deployment of lightweight detectors and inference prediction in real-world scenarios to achieve automated detection of power equipment and defects. Different deployment schemes are selected based on the resource conditions of the actual application scenario.

[0062] For servers with ample computing resources, the system can be directly deployed on GPU servers, combined with the TensorRT acceleration engine, to build a high-performance power equipment inspection system that supports rapid processing of batch inspection images. For edge computing devices, such as drones and line inspection robots, the model can be converted to ONNX format and deployed on embedded GPUs or FPGA chips to achieve real-time online inspection of inspection images. During deployment, inference code adapted to different hardware platforms needs to be written to ensure stable model operation.

[0063] In the inference and prediction phase, the power inspection images to be detected are input into the deployed detectors. The model automatically identifies power equipment and defects in the images through forward inference, outputting the target's category label, bounding box coordinates, and confidence score. A confidence threshold of 0.5 is set; when the target confidence score is higher than the threshold, it is considered a valid detection result; when it is lower, it is considered a false detection and discarded. Simultaneously, a visualization interface is constructed to overlay the detection results onto the original image, intuitively displaying the location and category information of equipment and defects. After the detection is completed, a detection report is generated, recording the detected equipment type, defect type, defect quantity, and location information, providing power maintenance personnel with accurate data for equipment repair and achieving an intelligent upgrade of power inspection.

[0064] To verify the effectiveness of the detection network constructed by the method of the present invention, the following experiment was further designed: 1. Dataset This invention collected 27,001 power grid inspection images with a resolution of 1024 to form a dataset to verify the effectiveness of the proposed method. The dataset is labeled with 14 categories of equipment and defects, including glass insulators, spontaneous glass insulator explosions, composite insulators, suspension clamps, and suspension clamp corrosion. The dataset was divided into training and testing subsets. The detector was trained using the training set, and its performance was evaluated on the testing set. The statistical results of the data distribution for each subset are shown in Table 1.

[0065] Table 1. Data distribution statistics for each subset of the inspection dataset. 2. Evaluation Indicators Average precision (AP) is a key metric for measuring the overall performance of a detector across all confidence thresholds. After matching a certain number of labeled bounding boxes and predicted bounding boxes using a specific algorithm, the results can be categorized into the following cases: (1) True Positive (TP): Samples that are actually positive are correctly predicted as positive, and the labeled objects are detected and the labeled boxes match the predicted boxes.

[0066] (2) True Negative (TN): Samples that are actually negative are correctly predicted as negative, and the labeled background area has no prediction box.

[0067] (3) False Positive (FP): Samples that are actually negative are incorrectly predicted as positive, and a prediction box appears in the labeled background area.

[0068] (4) False Negative (FN): Samples that are actually positive are incorrectly predicted as negative. Therefore, precision and recall are defined as follows: When the detector's confidence threshold changes, some low-confidence predicted boxes are discarded, thus affecting the overall precision and recall. A precision-recall curve can be plotted by iterating through all thresholds, using precision and recall as coordinates. The area under this curve represents the detector's overall performance, i.e., the average precision (AP). Since calculating the area under the curve using only integral methods is too time-consuming, current research typically uses fixed sampling points to estimate the area.

[0069] In the above process, the matching of the labeled bounding boxes and the predicted bounding boxes is determined by the Intersection over Union (IoU) threshold. IoU is defined as the area of ​​intersection of two bounding boxes divided by their area of ​​union; only bounding box pairs with an IoU greater than the threshold are considered a match. Different IoU thresholds result in different AP calculations; commonly used thresholds are 0.5 and 0.75, corresponding to... and .

[0070] 3. Training details The experiment was conducted using PyTorch 2.1 and CUDA 12.1 on four NVIDIA RTX 4090 GPUs.

[0071] Following the setup of existing methods, the backbone is initialized using weights pre-trained on ImageNet-1k, and the bulk network is fine-tuned on COCO. The AdamW optimizer is chosen for the fine-tuning training process, with a learning rate set to 10. -4 The weight decay factor was 0.05, the batch size was 16, and the input image's short side was adjusted to 800 pixels while ensuring the long side did not exceed 1333 pixels. Training consisted of 12 epochs, with the learning rate decaying to 0.1 times its previous value in epochs 8 and 11.

[0072] 4. Analysis of Experimental Results To verify the effectiveness of the proposed method, this invention selected several general lightweight backbones and constructed a lightweight detection network based on the Mask-RCNN framework. These networks were also initialized using weights pre-trained on ImageNet-1k, as shown in Table 3. To characterize the inference cost, this invention measured their inference cost at a resolution of 1024 using an RTX 4090 GPU and the official code implementation of each network (the measurement results have been converted to a single image). Some test set detection results are shown below. Figure 4 As shown.

[0073] Table 3 shows that the proposed lightweight detection network exhibits superior performance on the inspection dataset. Thanks to network layers with local receptive fields and a concise network architecture, the proposed network demonstrates a more significant latency advantage on high-resolution images. Compared to EfficientFormer-L1 (average object detection accuracy of 45.5), FastViT-SA12 (48.0), and RepViT-M1.1 (48.4), the proposed network achieves significantly higher latency in T-versions. It is higher (48.6), and has lower multiply-accumulate computation, fewer parameters, and lower latency. Compared to RepViT-1.5, the proposed B version of the network... It improved by 3.8 percentage points (54.1 vs 50.3), while the two were similar in terms of multiply-accumulate operation volume (148.4G vs 153.2G), number of parameters (16.8M vs 16.8M), and latency (13.5ms vs 14.0ms).

[0074] Table 3 Evaluation results and comparisons of the proposed network on the inspection dataset Validated using an inspection dataset, this invention's lightweight detection network (with a convolutional neighborhood attention NAC layer at its core), based on a fusion of neighborhood attention and depthwise convolution, significantly outperforms traditional CNNs, Transformers, and existing hybrid architectures in terms of performance-to-weight ratio, inference efficiency, and multi-task adaptability, fully validating the effectiveness and practicality of the technical solution. This network can be widely applied to resource-constrained edge platforms such as power line inspection, intelligent monitoring, and autonomous driving, providing a complete, efficient, and reliable technical solution for the engineering implementation of edge machine vision (high-resolution image classification, object detection, panoramic segmentation, etc.).

[0075] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for detecting power equipment that integrates neighborhood attention and depth convolution, characterized in that, include: S1. Collect power inspection images containing power transmission and distribution equipment, and label the power equipment targets and defect targets in the images to obtain an image dataset containing the labels; S2 introduces domain attention and depthwise convolution to construct a domain attention layer containing convolutions; S3 combines a domain attention layer with convolutions, Mask-RCNN, and introduces a convolutional feedforward network to construct a detection network for target detection in power line inspection images. S4, the labeled image dataset is preprocessed and then input into the detection network, and the detection network parameters are updated to obtain the trained detection network; S5 inputs the power inspection image to be detected into the trained detection network, and filters the output results based on the confidence threshold to finally obtain the detection results of power equipment targets and defect targets in the power inspection image.

2. The power equipment detection method fusing neighborhood attention and depth convolution according to claim 1, characterized in that, In step S1, the collected power inspection images include power inspection images collected by drone aerial photography, power inspection images collected by line inspection robots, power inspection images collected by power channel monitoring cameras, and power inspection images collected by manual inspection.

3. The power equipment detection method fusing neighborhood attention and depth convolution according to claim 1, characterized in that, In S1, the annotation includes the border and category corresponding to the target.

4. The power equipment detection method fusing neighborhood attention and depth convolution according to claim 1, characterized in that, In S2, the specific formula for the domain attention layer containing convolutions is as follows: ； in, These are all factors used to control the weight ratio, where R is the receptive field size. For element-wise multiplication, for In the receptive field index The weights and neighborhood features at each location, where D represents the learnable convolutional weights and NAC represents the domain attention layer containing convolutions. For the first One sample in The location feature, where r is the receptive field index. For the s-th sample at position p, the receptive field index The eigenvector at that location.

5. The power equipment detection method fusing neighborhood attention and depth convolution according to claim 4, characterized in that, The factor used to control the weight ratio is obtained by projecting the feature map samples and performing an outer product, specifically: ； in, and Both are weight matrices. The compression ratio is... The outer product in the dimensions of receptive field and channel. These are feature map samples after global pooling and compression ratio.

6. The power equipment detection method fusing neighborhood attention and depth convolution according to claim 1, characterized in that, In S3, the detection network for target detection in power line inspection images includes a backbone network, a neck, a region recommendation detection head, and a two-stage detection head. The backbone network contains one convolutional embedding layer and several network stages. Each network stage contains multiple sequentially connected modules. Each module consists of a convolutional feedforward network and a convolutional domain attention layer. The neck is an FPN structure used to receive the output features of each network stage of the backbone network. The region recommendation detection head and the two-stage detection head adopt the corresponding structure of Mask-RCNN.

7. The method for detecting power equipment by fusing neighborhood attention and depth convolution according to claim 6, characterized in that, The overall processing flow of the backbone network is as follows: For the input image, the image is first converted into a feature map through a convolutional embedding layer; Secondly, the feature maps are sequentially input into each network stage for processing, and the output of each network stage is used as an intermediate output; in each network stage, the first module also needs to downsample the input feature maps; Finally, the output of the last network stage is processed by a convolutional layer and input into the neck along with all intermediate outputs.

8. The method for detecting power equipment by fusing neighborhood attention and depth convolution according to claim 6, characterized in that, In the backbone network, the attention mechanism is not enabled in the first network stage, while it is enabled in the remaining network stages. The size of the corresponding attention mechanism convolution kernel increases gradually according to the connection order of the network stages.

9. The power equipment detection method fusing neighborhood attention and depth convolution according to claim 1, characterized in that, In step S4, the preprocessing of the labeled image dataset includes randomly cropping, inverting, and scaling the labeled images in sequence.

10. The method for detecting power equipment by fusing neighborhood attention and depth convolution according to claim 1, characterized in that, In S4, the trained detection network needs to be deployed using different schemes based on the resource conditions of the actual application scenario, specifically: For servers with sufficient computing resources, the detection network is deployed directly on GPU servers, and a high-performance power equipment detection system is built by combining it with the TensorRT acceleration engine. For edge computing devices, the detection network is converted to ONNX format and deployed on embedded GPU or FPGA chips to achieve real-time online detection of inspection images.

Images