Power target detection method based on spatial interaction and segmentation attention under small sample

Abstract

This application relates to the field of smart grids, and more particularly to a method for power target detection based on spatial interaction and segmentation attention under limited sample conditions, including constructing a limited sample target detection dataset; and constructing a novel residual network AKS based on variable kernel convolution. 2 -Net, including multiple AKS 2 The block and MAFF modules extract features through grouping, spatial displacement, and concatenation, and perform feature fusion based on a segmentation attention mechanism; then, the feature maps are further weighted and fused using the MAFF module; using AKS... 2 Using Net as the backbone network, a target detection network is constructed under small sample conditions. Based on the target detection dataset, the target detection network is trained in conjunction with the overall network function to obtain a power target detection model under small sample conditions. This application effectively extracts irregular and multi-scale features based on variable kernel convolution and segmentation attention under small sample conditions, which can significantly improve the accuracy of target detection.

Description

Technical Field

[0001] This application relates to the field of smart grids, and more specifically, to a method for power target detection based on spatial interaction and segmentation attention under small sample conditions. Background Technology

[0002] Power lines operate in complex environments with diverse safety hazards, including insulator explosions, transformer oil leaks, bird nesting, conductor breaks, hanging foreign objects, and interference from heavy machinery such as forklifts and cranes. These hazards seriously threaten the safe and stable operation of the power grid. Power line inspection tasks face numerous challenges, such as complex and varied natural backgrounds, different imaging conditions, multi-scale inspection objects, and differences in sharpness between different objects within the same image due to focal length variations. Deep learning technology offers a new approach to addressing these challenges; however, commonly used deep learning methods heavily rely on training with a large number of samples. Only with sufficient samples can the trained model meet accuracy requirements.

[0003] In actual power line inspection scenarios, sample acquisition is difficult and annotation costs are high. Samples of safety hazards such as abnormal hanging objects, conductor breaks, bird nests, and large-scale mechanical construction are difficult to collect in large quantities. Furthermore, fault image data such as insulator explosions and missing / misaligned / deformed shock absorbers require annotation by professional technicians. This makes building a comprehensive sample library extremely difficult and cannot meet the training requirements of common deep learning network models. Therefore, research on small-sample target detection has become a key issue in the field of power line inspection.

[0004] From a technical perspective, traditional convolutional neural networks (CNNs) have limitations in power line detection. They typically use convolutional kernels of fixed size and shape, and fixed sampling positions, extracting features by sliding the kernels along the spatial dimension. This approach not only limits the network's adaptability to features of different scales and shapes but also incurs high computational costs. Furthermore, different attention mechanisms focus on different aspects of image features, but traditional fusion methods struggle to effectively correlate and fuse features at different levels, easily leading to the loss of details and semantic information in high-level branches, or neglecting branch feature weights, resulting in problems such as gradient vanishing.

[0005] Furthermore, in two-stage object detection networks such as Faster R-CNN and MetaR-CNN, the mechanism of distinguishing positive and negative samples based on the IoU value between candidate boxes and ground truth boxes is prone to misclassifying some positive samples as negative samples and discarding them when the sample size is small, especially for distant, blurry, or occluded targets, thus interfering with the model learning process. Moreover, due to the difficulty in obtaining defective target samples, the dataset often exhibits a significant imbalance between normal and defective target samples, making the traditional cross-entropy loss function ineffective in such imbalanced scenarios. These problems urgently require technological innovation to improve the performance of small-sample target detection on power lines. Summary of the Invention

[0006] In view of this, this application provides a power line target detection method based on spatial interaction and segmentation attention under small sample conditions. Under small sample conditions, it effectively extracts irregular and multi-scale features based on variable kernel convolution and segmentation attention, while promoting spatial information interaction and performing two-stage power line inspection target detection to improve the accuracy of target detection.

[0007] The technical solution provided in this application is as follows:

[0008] A method for detecting electric targets based on spatial interaction and segmentation attention under small sample sizes includes:

[0009] Define a few-sample object detection task and construct an object detection dataset;

[0010] Constructing a residual network AKS with variable kernel convolution 2 -Net, including multiple AKS 2 Block-based, multi-branch attention feature fusion (MAFF) module; for input AKS 2 -Net feature map, the AKS 2 The block extracts features based on variable kernel convolution AKConv after grouping, spatial displacement, and splicing, and performs feature fusion based on segmentation attention mechanism; the MAFF module is used to perform parallel sampling of the fused feature map based on spatial pyramid pooling module, and merge it through smooth convolution and element-wise multiplication, and then perform weighted fusion of the feature map again based on hybrid skip connection module MSC.

[0011] Using MetaR-CNN network as the basic framework, and AKS 2 -Net serves as the backbone network, constructing a target detection network under small sample conditions;

[0012] Based on the target detection dataset, the target detection network is trained by combining the FocalLoss loss function and the bounding box regression loss function to obtain a target detection model under small sample conditions, which is used for power target detection.

[0013] One possible implementation involves constructing a residual network AKS with variable kernel convolutions. 2 -Net performs feature extraction, including:

[0014] The input feature map is divided into k basis arrays, and each basis array is further divided into r blocks. Each block contains an AKS. 2 Blocks are used for feature extraction to obtain receptive fields at different scales and extract feature information at different scales.

[0015] Spatial displacement operation is performed on the feature groups of each group, and the displacement feature maps are stitched together in the channel dimension to obtain a new feature map, which contains feature information of different spatial locations.

[0016] In one possible implementation, AKS 2 Blocks perform feature fusion based on a segmentation attention mechanism, including:

[0017] The weights of each block are calculated using Split-attention to fuse features within the base array;

[0018] The fused features are compressed in spatial dimension using global average pooling.

[0019] The weight W of the i-th block is calculated using softmax. i ;

[0020] The input feature map of each block is multiplied by its corresponding weight, and the calculation results of each block in the basis array are added together to obtain the feature fusion result of the current basis array.

[0021] In one possible implementation, the MAFF module contains n input branches for receiving feature maps from different levels or scales; each input branch is connected to a dilated convolution.

[0022] The feature maps are then weighted and fused again based on the MAFF module, including:

[0023] In the MAFF module, spatial and channel attention mechanisms are applied to extract the local spatial context and global channel correlation information of the input feature maps of each branch;

[0024] The feature maps of each branch are sampled in parallel using the hollow spatial pyramid pooling ASPP module, generating feature maps of n branches respectively.

[0025] Perform smooth convolution on the feature maps of the n branches respectively, and then merge them by element-wise multiplication;

[0026] Perform a convolution once, changing the number of channels in the feature map;

[0027] The feature maps are weighted and fused again based on the Hybrid Skip Connection Module (MSC) to form a fused feature map; the learnable weights corresponding to each feature map are optimized through backpropagation.

[0028] In one possible implementation, after weighted fusion of the feature maps, the method further includes:

[0029] A convolution operation is performed on the weighted and fused feature map to extract refined features at the pixel level;

[0030] The convolutional features are summed to obtain AKS. 2 -Net's final feature map output.

[0031] In one possible implementation, the target detection network under small sample conditions uses AKS. 2 -Net serves as the backbone network, connecting the Region Proposal Network (RPN) and the ROI Align to form a two-stage object detection network.

[0032] In one possible implementation, the target detection dataset is divided into a base class dataset and a new class dataset, and each sample in the dataset consists of an input image, a class label, and the bounding box coordinates of the target.

[0033] Based on the object detection dataset, the object detection network is trained, including:

[0034] Meta-training is performed on the network parameters of the target detection network based on the aforementioned basic dataset.

[0035] The parameters of the meta-trained object detection model are fine-tuned based on a balanced dataset containing the base class dataset and the new class dataset.

[0036] The obtained target detection model is optimized based on the determined overall network loss function; wherein, the overall network loss function includes the FocalLoss loss function and the bounding box regression loss function.

[0037] In one possible implementation, the FocalLoss loss function is used as the classification loss function, expressed as follows:

[0038]

[0039] In the formula, α c γ is the weight parameter corresponding to category c, used to adjust the importance of samples of that category; γ is the decay parameter, used to adjust the weights of easy and hard samples; when p c When it approaches 1, (1-p c ) γ The term tends to 0; K represents the number of categories, and x represents the output vector of the fully connected layer. i This represents the i-th element in the vector.

[0040] One possible implementation involves obtaining a target detection model under small sample conditions for power target detection, including:

[0041] The trained target detection model was used for target detection in UAV aerial power images under small sample conditions in real-world scenarios to obtain the category, confidence score, and corresponding target bounding box coordinates of each detected target. The detected targets included intact insulators, defective insulators, obstructed insulators, intact isolators, damaged isolators, bird nests, and suspended objects.

[0042] Compared with the prior art, the technical solution provided in this application has the following beneficial effects:

[0043] This application presents a novel two-stage object detection network, AKS, which incorporates spatial interaction and segmentation attention improvements. 2 -Net, via AKS 2 The block implements feature extraction combining variable convolution, spatial displacement, and segmentation attention, enabling spatial information interaction and channel relationship learning between image features, thus enhancing the ability to mine irregular and multi-scale features. Leveraging the dilated spatial pyramid pooling and hybrid skip connections of the MAFF module, it fuses multi-channel, multi-level image detail features and spatial information, improving segmentation accuracy and boundary localization capabilities in complex scenes. A feature similarity calculation method based on metric learning corrects the problem of negative sample misclassification in region candidate networks, reducing the false negative rate of small and blurry targets. The FocalLoss function is introduced to mitigate the impact of sample imbalance on detection. AKS is used... 2 -Net is used as the backbone to construct a two-stage fine-tuned target detection network suitable for small and imbalanced samples, providing a new option for small sample detection. Experimental results show that this method has outstanding ability to detect distant / small / blurred and occluded targets in power line target detection. Attached Figure Description

[0044] Figure 1 This is a flowchart of a power target detection method based on spatial interaction and segmentation attention under small sample conditions, provided in Embodiment 1 of this application.

[0045] Figure 2 This is a flowchart of a power target detection method based on spatial interaction and segmentation attention under small sample size, provided in Embodiment 2 of this application.

[0046] Figure 3 AKS provided in Embodiment 2 of this application 2 - A schematic diagram of the Net network structure.

[0047] Figure 4 This is a schematic diagram of the network structure of the MAFF module provided in Embodiment 2 of this application.

[0048] Figure 5 The AKS provided in Embodiment 2 of this application 2 - A schematic diagram of the Meta R-CNN network structure with Net as the backbone.

[0049] Figure 6 This is a flowchart of a method for detecting power targets in a small-sample UAV aerial image of a power field, as shown in Embodiment 3 of this application.

[0050] Figure 7 The AKS-based embodiment provided in Embodiment 3 of this application 2 -Flowchart of the feature extraction and feature fusion operation method of the Net network.

[0051] Figure 8 This is a schematic diagram of the network structure of the MSC module provided in Embodiment 3 of this application.

[0052] Figure 9 This is a flowchart of a method for training an object detection network provided in Embodiment 3 of this application.

[0053] Figure 10 This is a schematic diagram of the detection results on a small sample dataset of power targets provided in Embodiment 3 of this application.

[0054] Figure 11 The heatmap of the power target detection model provided in Embodiment 3 of this application on a small sample dataset of power targets. Detailed Implementation

[0055] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0056] Example 1

[0057] See Figure 1 This is a flowchart of a power target detection method based on spatial interaction and segmentation attention under small sample conditions, provided in Embodiment 1 of this application. Figure 1 As shown, the specific implementation steps of the above method include:

[0058] Step 101: Define a small sample object detection task and construct an object detection dataset.

[0059] The aforementioned object detection datasets include a base class dataset and a new class dataset, which do not overlap. The base class dataset contains sufficient labeled samples for model training, while the new class dataset contains only a small number of samples. The few-shot object detection task aims to achieve accurate detection of new class objects under the condition of having only a small amount of labeled data.

[0060] Step 102: Construct the residual network AKS with variable kernel convolution. 2 -Net, including multiple AKS 2 The block and the Multi-branch Attention Feature Fusion (MAFF) module.

[0061] Among them, for the input AKS 2 -Net feature maps, the above AKS 2 The block extracts features through grouping, spatial displacement, and concatenation, and then performs feature fusion based on the variable kernel convolution AKConv, and performs feature fusion based on the split-attention mechanism.

[0062] The MAFF module described above contains multiple input branches. Each input branch is connected to a dilated convolution, which samples the feature map of each branch in parallel based on a dilated spatial pyramid pooling module, generating new feature maps for multiple branches. Subsequently, a smooth convolution is performed on the new feature maps of each branch, and then they are merged by element-wise multiplication. A hybrid skip connection module (MSC) is used to achieve weighted fusion of the feature maps to form a fused feature map.

[0063] In this embodiment, a residual connection is established between the fused features and the original input features to form the final output features. Based on the above AKS 2 -Net network can significantly enhance the ability of object detection networks to extract irregular, multi-scale and spatially related features, as well as the fusion effect of multi-path features.

[0064] Step 103: Using the Meta R-CNN network as the basic framework, and employing the AKS residual network with variable kernel convolution... 2 -Net serves as the backbone network, constructing a target detection network under small sample conditions.

[0065] Specifically, the aforementioned target detection network uses AKS 2 -Net serves as the backbone network, connecting the Region Proposal Network (RPN) and ROI Align to form a two-stage object detection model.

[0066] Specifically, the input image to be detected first enters AKS 2-Net extracts features from the image to be detected layer by layer through a series of convolution and pooling operations, transforming the image into a feature map containing rich semantic and structural information. After feature extraction, the final classification and bounding box regression are performed based on fully connected layers, thus completing the detection process from input image to output of target category and location.

[0067] Step 104: Based on the above object detection dataset, perform meta-training and model fine-tuning on the above object detection network, and optimize the generated object detection model based on the FocalLoss loss function and the bounding box regression loss function to obtain the object detection model under small sample conditions.

[0068] Compared with the prior art, the technical solution provided in Embodiment 1 of this application has the following beneficial effects:

[0069] This application enhances the extraction of irregular and multi-scale features and the ability to mine feature channel relationships by introducing deformable convolution, spatial displacement, and segmentation attention in the feature extraction stage, dynamically adjusting convolution kernel parameters, and reducing computational cost. In the feature fusion stage, it utilizes multi-path attention and multi-scale feature fusion MAFF modules combined with spatial and channel attention, ASPP modules, and hybrid skip connections to effectively integrate multi-channel and multi-level detailed features, improving segmentation accuracy and boundary localization capabilities in complex scenes. A sample selection method based on feature similarity calculation corrects the problem of misclassification of positive and negative samples caused by insufficient sample size in two-stage object detection networks, reducing the false negative rate of small and blurry targets. The FocalLoss loss function is used to balance the impact of sample imbalance on detection results, enhancing the training contribution of sparse and difficult-to-classify samples. AKS is used as the basis for further analysis. 2 -Net is a novel object detection network built on a backbone that supports two-stage fine-tuning training under small sample conditions. It comprehensively improves the network's ability to extract irregular features and fuse multi-path and multi-level features, providing a better solution for small sample object detection.

[0070] Example 2

[0071] See Figure 2 This is a flowchart of a power target detection method based on spatial interaction and segmentation attention under small sample conditions, provided in Embodiment 2 of this application. Figure 2 As shown, the specific implementation steps of the above method include:

[0072] Step 201: Define the small sample target detection task.

[0073] Among them, the aforementioned small sample target detection task refers to the accurate detection of new types of targets under the condition of having only a small amount of labeled data.

[0074] Step 202: Obtain the object detection dataset. This dataset includes a base class dataset and a new class dataset.

[0075] Specifically, based on the aforementioned small-sample object detection task, an object detection dataset C is constructed, where each sample can be represented as a triple. Where, x i For the input image, y i ∈C represents the category label, b i This indicates the bounding box coordinates of the target.

[0076] Furthermore, the aforementioned object detection dataset C is divided into a basic class dataset C. base and the new category dataset C novel The two do not overlap, that is Among them, C base There are sufficient labeled samples for model training, while C novel It contains only a small number of K-shot samples (K = 1, 2, 3, ...) to simulate the needs of low-data-volume target detection in real-world application scenarios.

[0077] Step 202: Construct the residual network AKS with variable kernel convolution. 2 -Net, including several AKS for feature extraction 2 The block and multi-branch attention feature fusion MAFF module.

[0078] Specifically, to enhance the spatial information interaction and feature fusion capabilities of the model, this application provides a novel residual network AKS with variable kernel convolution based on a segmentation attention mechanism. 2 -Net. AKS 2 block is AKS 2 -Key modules used for feature extraction in each branch of the Net.

[0079] In traditional convolutional neural networks, features are extracted by sliding the convolution kernel across spatial dimensions. However, the computational cost of convolution operations is relatively high. To address this technical issue, this application utilizes AKS... 2 The block introduces a spatial-shift operation to achieve spatial information exchange through simple feature displacement. The feature maps are shifted along the height (H) and width (W) directions respectively, and then the shifted feature maps are stitched together to form a new feature map.

[0080] Specifically, such as Figure 3 As shown in AKS 2In Net, the input feature maps are divided into multiple groups, such as group 1 to group n. Each group is further divided into multiple blocks for independent processing. An AKS is constructed in each block. 2 To perform multi-scale feature extraction. The input is the residual network AKS. 2 -Net features Figure X The channel dimension is divided into 4 groups, with each group containing C / 4 channels. The 4 groups of feature maps are shifted using three branches, with each branch spatially shifting each feature map in a different direction (one branch does not perform any shifting operation). The 4 shifted feature maps are then concatenated along the channel dimension to obtain a new feature map. This new feature map contains feature information from different spatial locations, achieving spatial information interaction.

[0081] Below, we will use input features Figure X ∈R C×H×W For example, regarding AKS 2 The operation steps of a block are explained in detail.

[0082] (1) Input feature map grouping.

[0083] Input features Figure X The channel dimension C is divided into 4 groups, with each group containing C / 4 channels. Assuming C is divisible by 4, the grouped feature map is represented as follows:

[0084] X = [X1, X2, X3, X4]

[0085] Among them, X i ∈R (C / 4)×H×W .

[0086] (2) Perform spatial displacement on each set of feature maps.

[0087] The four sets of feature maps are shifted using three branches, with each branch shifting the feature maps in a different direction. Figure X i Perform spatial displacement (without displacement of one branch), for example:

[0088] X1[1:w,:,:]←X1[0:w-1,:,:],

[0089] X2[0:w-1,:,:]←X2[1:w,:,:],

[0090] X3[:,1:h,:]←X3[:,0:h-1,:],

[0091] X4[:,0:h-1,:]←X4[:,1:h,:]

[0092] Then the four sets of features after displacement Figure X 1, X2, X3, and X4 are concatenated along the channel dimension to obtain new features. Figure X ′ ∈R C×H×W , is represented as:

[0093] X' = ​​[X1, X2, X3, X4]

[0094] Among them, features Figure X It contains feature information from different spatial locations, enabling the interaction of spatial information.

[0095] (3) Feature extraction.

[0096] In each branch, new features Figure X AKConv is used for feature extraction to enhance the model's ability to extract irregular and multi-scale features, thereby forming new features. Figure X ", is represented as:

[0097]

[0098] In the formula, X'[p] is the value of the output feature map at position p. Δk m (p) is a dynamically generated offset used to adjust the position of the m-th sampling point. m (p) represents the dynamically generated weights. M is the preset maximum number of sampling points. Interp() represents the bilinear interpolation operation, used to handle sampling of non-integer coordinates.

[0099] (4) Feature fusion.

[0100] By using the Split-attention mechanism, feature maps extracted from all branches are fused to form AKS. 2 The final feature map Y output by the block is represented as:

[0101]

[0102] This application addresses the fact that commonly used fusion methods do not effectively correlate and fuse features at different levels spatially. Conventional multi-level feature fusion (Semantic and Detail Infusion, SDI) modules effectively fuse high-level semantic features with low-level detail features by integrating multi-level feature maps generated by the encoder, improving the model's segmentation accuracy and boundary localization capabilities in complex scenes. Applying spatial and channel attention mechanisms to process features at each level better integrates local spatial context and global channel correlation information. However, the SDI module also has two shortcomings: first, it extracts feature maps at different levels for different branches and then resamples them using attention mechanisms, which results in the loss of some feature details and semantic information from higher-level branches; second, for high- and low-level feature fusion, the SDI module uses a simple Hadamard product operation, without considering the weights of features from different branches, which may lead to gradient vanishing.

[0103] Therefore, to solve the above-mentioned technical problems, this application introduces hollow spatial pyramid pooling and hybrid skip connection mechanism based on some advanced concepts of SDI module, and then fuses the features extracted from all groups by constructing a new multi-branch attention feature fusion (MAFF) module.

[0104] For example, such as Figure 4 As shown, the MAFF module has three input branches to receive feature maps from different levels or scales. For the input feature map of each branch, local spatial context and global channel correlation information are extracted based on spatial and channel attention mechanisms.

[0105] like Figure 4 In this approach, each input branch is connected to a dilated convolution. The feature map of each branch is sampled in parallel using a dilated spatial pyramid pooling module to capture the context of multi-scale targets, which is particularly effective for detecting targets of different sizes, generating new feature maps for each branch. Subsequently, smooth convolutions are performed on the new feature maps of each branch separately, and then they are merged by element-wise multiplication. Weighted fusion of the feature maps is achieved through a hybrid skip connection module (MSC) to form a fused feature map.

[0106] Specifically, the Hybrid Skip Connection (MSC) module balances weights using an activation function to ensure effectiveness, then multiplies the input features of each branch with their corresponding weights, and finally aggregates the weighted sum of the feature outputs from all branches. A 3×3 convolution is then used to refine and integrate the generated fused feature map; convolution operations help refine features at the pixel level. Finally, the convolutional features are summed to obtain the final feature map output.

[0107] In this embodiment, a residual connection is established between the fused features and the original input features to form the final output features. Based on the above AKS 2 -Net network can significantly enhance the ability of object detection networks to extract irregular, multi-scale and spatially related features, as well as the fusion effect of multi-path features.

[0108] Step 203: Using the Meta R-CNN network as the basic framework, and employing the AKS residual network with variable kernel convolution... 2 -Net serves as the backbone network, constructing a target detection network under small sample conditions.

[0109] See Figure 5 This is a schematic diagram of the target detection network under small sample conditions provided in Embodiment 2 of this application. Figure 5 As shown, the object detection network uses AKS 2 -Net serves as the backbone network, connecting the Region Proposal Network (RPN) and ROI Align to form a two-stage object detection model. In this embodiment, AKS is used. 2 -Net serves as the feature extraction part of the entire object detection network. The input image to be detected first enters AKS. 2 -Net performs layer-by-layer feature extraction on the image to be detected through a series of convolution and pooling operations, transforming the image into a feature map containing rich semantic and structural information. For details, please refer to step 202 regarding AKS. 2 The description of -Net will not be repeated here.

[0110] like Figure 5 As shown, based on the above AKS 2 After the Net network completes feature extraction, it performs final classification and bounding box regression based on fully connected layers, thus completing the detection process from input image to target category and location output.

[0111] Step 204: Using the above target detection dataset as the training set, train the target detection network under the above few-sample conditions to obtain the target detection model, so as to perform few-sample target detection operations in real scenes and obtain the category, confidence score and corresponding target box coordinate information of each detected target.

[0112] In this embodiment of the application, the entire training process includes two stages: base training and fine-tuning.

[0113] In the basic training phase, a basic category dataset is used. Object detection training is performed to learn general features. During the fine-tuning phase, a new category dataset is received. Furthermore, the ability of the object detection model to detect new categories is improved by using a small number of support samples. Therefore, the entire training set can be represented as C. train =C base ∩C novel .

[0114] During training, each sample consists of an input image, a class label, and a target bounding box (b). x ,b y ,b w ,b h The goal is to learn a general detection model F. θ This enables it to predict the class label for an input sample x. and bounding box Right now It should be noted that in small sample object detection, the number of negative samples is usually significantly greater than that of positive samples. These negative samples are usually uniformly labeled as the background class and participate in the training of classification and bounding box regression.

[0115] Example 3

[0116] This application uses the example of target detection in UAV aerial photography of power grid images under small sample conditions in a real-world scenario to further illustrate the methods provided in Embodiments 1 and 2 above. See also Figure 6 This is a flowchart of a small-sample target detection method for power grid images obtained by unmanned aerial vehicle (UAV) according to Embodiment 3 of this application. Figure 6 As shown, the specific implementation steps of the above method include:

[0117] Step 301: Acquire and process drone aerial power image data, and combine it with publicly available datasets to create a basic class dataset and a new class dataset.

[0118] One possible implementation involves acquiring and processing drone-captured power line images by: using a drone to capture images along power lines, using tools such as LabelImg to annotate the images, and saving the dataset labels in XML format.

[0119] In this embodiment of the application, the publicly available dataset Pascal VOC 2012 is used as the base class dataset C. base A new category dataset C is constructed using drone aerial images of power grids. novel Among them, the basic class dataset C base The new category dataset C includes 20 categories such as airplanes, cars, birds, and boats. novelThe data includes seven categories: intact insulators, defective insulators, obstructed insulators, intact isolators, damaged isolators, bird nests, and hanging objects. Ten samples are labeled for each category, and other data are used for verification and testing.

[0120] Step 302: Construct AKS 2 -Net networks, including AKS 2 -AKS used for feature extraction in each branch of Net 2 block.

[0121] Specifically, to enhance the spatial information interaction and feature fusion capabilities of the power target detection model, this application designs a novel residual network AKS with variable kernel convolution based on the segmentation attention mechanism. 2 -Net. AKS 2 The -Net network is used to divide the input feature map into k groups, and each group is further divided into r blocks for independent processing.

[0122] In this embodiment of the application, taking k=2 and r=3 as an example, AKConv typically uses a convolution kernel size of 5 by default. Figure 7 As shown, AKS 2 The operation of a Net network specifically includes the following steps:

[0123] Step 3021: Divide the input feature map into two basis arrays.

[0124] Step 3022: Each base array is further divided into 3 splits, and each split consists of an AS. 2 Feature extraction is performed within each block. Within each block, feature extraction is performed using a regular convolution with a kernel size of 3, and AKConv convolutions with kernel sizes of 5 and 7, respectively, to obtain receptive fields of different scales and extract information at different scales.

[0125] Step 3023: Calculate the weights of each split using Split-attention and perform feature fusion within the basis array.

[0126] Specifically, the two feature maps are fused together using the summation of all elements, as shown below:

[0127]

[0128] In the formula, r represents the number of splits in each base array; in this embodiment, r = 3. j This represents the input of the j-th split module.

[0129] The fused features are then compressed in spatial dimension using global average pooling, and represented as follows:

[0130]

[0131] In the formula, H and W represent the height and width of the feature, respectively.

[0132] The weight W of the i-th split is calculated using softmax. i The Dense layer is implemented using two fully connected layers. The input feature map of each split is multiplied by its corresponding weight, and then the results of the two splits within the basis array are summed to obtain the feature fusion result of the current basis array, represented as:

[0133]

[0134] Step 3024: Based on the MAFF module, perform feature enhancement and fusion on the feature maps output by the two basis arrays.

[0135] Specifically, in the MAFF module, spatial and channel attention mechanisms are applied to the input feature map of each branch to extract local spatial context and global channel correlation information, represented as:

[0136]

[0137] Where i represents the i-th channel, SA() represents spatial attention operation, and CA() represents channel attention operation.

[0138] The feature map of each branch is sampled in parallel using the Spatial Pyramid Pooling (ASPP) module to capture contextual information of multi-scale targets. This is particularly effective for detecting targets of different sizes, generating feature maps for each of the n branches. Represented as:

[0139]

[0140] For feature maps respectively Perform smooth convolutions, then merge them element-wise, as follows:

[0141]

[0142] Among them, I C This represents the concatenated feature map, and SC() represents the smooth convolution operation.

[0143] Step 3025: Perform convolution again, changing the number of channels in the feature map.

[0144] Step 3026: Use skip connections to connect the feature map with AKS 2 - The original input data of -Net is fused.

[0145] See Figure 8 By using a hybrid skip connection module, the above feature maps can be processed. Weighted fusion to form fusion feature map I R The learnable weights w i This can be optimized through backpropagation, as follows:

[0146]

[0147] The weights are balanced using an activation function to ensure their effectiveness, and then... With the corresponding weight w i Multiplying, and finally aggregating the weighted sums of the n feature outputs, is represented as:

[0148]

[0149] The generated fused feature map is refined and integrated using a 3×3 convolution. Convolution operations help refine features at the pixel level. Then, the convolutional features are summed to obtain the final feature map output, as follows:

[0150]

[0151] Step 303, using AKS 2 Using the Net network as the backbone and Meta R-CNN as the detection network framework, a power target detection model is constructed under small sample conditions.

[0152] Step 304: Train the above power target detection model using the base class and new class datasets respectively based on meta-training and parameter fine-tuning.

[0153] As a feasible approach, such as Figure 9 As shown, step 304 above specifically includes:

[0154] Step 3041: Perform meta-training on network parameters using basic categories.

[0155] Specifically, meta-training employs an N-way K-shot model, where N is the number of classes randomly selected in each training iteration, and K is the number of samples provided for each class. By sampling different N-way K-shot tasks multiple times, the model learns general feature representations and the ability to quickly adapt to new tasks.

[0156] Step 3042: Fine-tune the network parameters using a balanced dataset containing the base class and the new class.

[0157] Specifically, the parameters of the meta-trained model are fine-tuned using a balanced dataset containing the base class and the new class. Each new class provides K object instances, with K set to 1, 2, 3, 5, and 10 respectively, to test the model's adaptability under different sample sizes.

[0158] Step 3043: Optimize the above power target detection model based on the determined overall network loss function.

[0159] Specifically, the overall loss function of the network described above is composed of the classification loss L. cls and bounding box regression loss L reg Composition, represented as:

[0160] L = L cls +L reg

[0161] In this embodiment, FocalLoss is introduced as the classification loss L. cls Specifically, due to the difficulty in obtaining defective target samples, datasets often exhibit a significant imbalance between normal and defective target samples. Traditional cross-entropy loss performs poorly in such imbalanced scenarios. Therefore, in calculating the classification loss, this embodiment of the application chooses to use the FocalLoss loss function to balance the contributions of samples of different quality, introducing FocalLoss as the classification loss L. cls This reduces the impact of sample imbalance on test results.

[0162] FocalLoss addresses class imbalance by reducing the contribution of internal weights (easy samples). Therefore, even with a large number of easy samples, their contribution to the total loss is small, allowing training to focus on sparse data from hard samples. In binary classification tasks, FocalLoss is calculated as follows:

[0163]

[0164] In the case of multi-class classification, the softmax function is:

[0165]

[0166] Specifically, c represents the class of the current sample, p c This represents the probability value of class c in the output probability distribution; K represents the number of classes; x represents the output vector of the fully connected layer, etc. i This represents the i-th element in the vector.

[0167] Therefore, the above classification loss function L cls The calculation formula can be expressed as:

[0168]

[0169] Where, α c γ is the weight parameter corresponding to category c, used to adjust the importance of samples in that category; γ is the decay parameter, usually set to 2, used to adjust the weights of easy and hard samples. When p c When it approaches 1, (1-p c ) γ As the term approaches 0, the loss contribution of correctly classified simple samples decreases significantly. On the other hand, for samples with high classification difficulty, when p... c When the value approaches 0, the loss will be significantly amplified, thus focusing on samples that are difficult to classify and balancing the sample size.

[0170] Step 305: Apply the trained model to target detection in UAV aerial photographs of power lines under small sample conditions in real-world scenarios to obtain the category, confidence score, and corresponding bounding box coordinates of each detected target. The detected targets include, but are not limited to: intact insulators, defective insulators, obstructed insulators, intact isolators, damaged isolators, bird nests, and suspended objects.

[0171] Specifically, the parameters that yield the best test results are used as the final power target detection model, such as... Figure 10 As shown in Table 1, to further verify the effectiveness of the proposed model, this application also compared it with current mainstream few-shot object detection models on the same dataset, such as FSRW, MetaR-CNN, MetaDet, TFA, SRR-FSD, QA-FewDet, FSCE, G-FSD, FSOD-UP, and DCNet. The test results, as shown in Table 1, demonstrate that the method in this application outperforms existing mainstream FSOD methods. In K-sample tasks, it outperforms other methods in most cases. Compared to MetaR-CNN, the mAP is improved by nearly 15 percentage points; compared to QA-FewDet, the overall average improvement is 2 percentage points. This indicates that the proposed method can better handle few-shot object detection tasks, exhibiting good robustness and generalization ability.

[0172] Table 1 Test Results

[0173]

[0174] To further visualize the advantages of the proposed model in feature extraction, AKS 2 The heatmaps of -Net and ResNet-101 in the process of electric target detection under small sample conditions were visualized, such as Figure 11 As shown. From Figure 11 It can be seen that, under small sample conditions, the AKS proposed in this invention...2 -Net has more accurate feature focus on power targets, thereby improving the accuracy of power target detection.

[0175] Compared with the prior art, the technical solution provided in Embodiment 3 of this application has the following beneficial effects:

[0176] This application provides a novel two-stage object detection network, AKS, that incorporates spatial interaction and segmentation attention improvements. 2 -Net. This network introduces a multi-path segmentation attention mechanism for feature extraction and fusion, and uses metric learning to perform secondary screening of candidate targets, enhancing the ability to extract and fuse feature information of irregular, distant / small / blurred targets and occluded targets, and reducing the impact of insufficient samples and imbalanced samples on network performance. Specifically, the technical effects achieved by the technical solution provided in this application are as follows:

[0177] (1) AKS, a grouped convolutional feature extraction network based on variable convolution and spatial displacement 2 The block (AKConv, Spatial-shift and Split-attention block) enables spatial information interaction between image features and the learning of relationships between feature channels, thereby enhancing the network's ability to mine irregular and multi-scale feature information.

[0178] (2) Based on a novel multi-branch attention multi-scale feature fusion (MAFF) module, multi-channel and multi-level image detail features and spatial information are fused through hollow spatial pyramid pooling (ASPP) and hybrid skip connection (MSC), thereby improving segmentation accuracy and boundary localization capability in complex scenes.

[0179] (3) Based on a feature similarity calculation method based on metric learning, the similarity calculation is performed between the negative sample features selected by the regional candidate network and all supporting category features. Combined with the threshold, the candidate negative samples are re-screened, and the positive samples that were misjudged as negative samples are corrected. This can reduce the interference to network training and reduce the network's false negative rate for small and fuzzy targets.

[0180] (4) The FocalLoss function is introduced into the classification loss calculation, which can reduce the impact of sample imbalance on the detection results.

[0181] (5) AKS 2 Using -Net as the backbone, a two-stage fine-tuning-based target detection network suitable for small sample and imbalanced sample conditions is constructed, providing a new option for small sample target detection through the fine-tuning mechanism.

[0182] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting electric targets based on spatial interaction and segmentation attention under small sample conditions, characterized in that, include: Define a few-sample object detection task and construct a few-sample object detection dataset; Constructing a residual network AKS with variable kernel convolution 2 -Net, including multiple AKS 2 Block-based, multi-branch attention feature fusion (MAFF) module; for input AKS 2 -Net feature map, the AKS 2 The block extracts features based on variable kernel convolution AKConv after grouping, spatial displacement, and splicing, and performs feature fusion based on segmentation attention mechanism; the MAFF module is used to perform parallel sampling of the fused feature map based on spatial pyramid pooling module, and merge it through smooth convolution and element-wise multiplication, and then perform weighted fusion of the feature map again based on hybrid skip connection module MSC. Using Meta R-CNN network as the basic framework, and AKS 2 -Net serves as the backbone network, constructing a target detection network under small sample conditions; Based on the aforementioned target detection dataset, the target detection network is trained using the FocalLoss loss function and the bounding box regression loss function to obtain a target detection model under small sample conditions, which is then used for power target detection. Among them, the residual network AKS with variable kernel convolution is constructed. 2 -Net performs feature extraction, including: The input feature map is divided into k There are radix arrays, and each radix array is divided into ... r Each block consists of an AKS. 2 Blocks are used for feature extraction to obtain receptive fields at different scales and extract feature information at different scales. Spatial displacement operation is performed on the feature groups of each group, and the displacement feature maps are stitched together in the channel dimension to obtain a new feature map, which contains feature information of different spatial locations; The trained target detection model was used for target detection in UAV aerial power images under small sample conditions in real-world scenarios to obtain the category, confidence score, and corresponding target bounding box coordinates of each detected target. The detected targets included intact insulators, defective insulators, obstructed insulators, intact isolators, damaged isolators, bird nests, and suspended objects.

2. The method for detecting electric targets based on spatial interaction and segmentation attention under small sample conditions according to claim 1, characterized in that, AKS 2 Blocks perform feature fusion based on a segmentation attention mechanism, including: The weights of each block are calculated using Split-attention to fuse features within the base array; The fused features are compressed in spatial dimension using global average pooling. Calculate the first by combining softmax i Weight of each block W i ; The input feature map of each block is multiplied by its corresponding weight, and the calculation results of each block in the basis array are added together to obtain the feature fusion result of the current basis array.

3. The method for detecting electric targets based on spatial interaction and segmentation attention under small sample conditions according to claim 1, characterized in that, The MAFF module includes n Each input branch is used to receive feature maps from different levels or scales; Each input branch is connected to a dilated convolution; The feature maps are then weighted and fused again based on the MAFF module, including: In the MAFF module, spatial and channel attention mechanisms are applied to extract the local spatial context and global channel correlation information of the input feature maps of each branch; The SPPP module, utilizing the spatial pyramid pooling (SPSP) module, performs parallel sampling of the feature maps for each branch, generating... n Feature maps of each branch; To each n The feature maps of each branch are smoothly convolved and then merged through element-wise multiplication; Perform a convolution once, changing the number of channels in the feature map; The feature maps are weighted and fused again based on the Hybrid Skip Connection Module (MSC) to form a fused feature map; the learnable weights corresponding to each feature map are optimized through backpropagation.

4. The method for detecting electric targets based on spatial interaction and segmentation attention under small sample conditions according to claim 1, characterized in that, After weighted fusion of the feature maps, the method further includes: A convolution operation is performed on the weighted and fused feature map to extract refined features at the pixel level; The convolutional features are summed to obtain AKS. 2 -Net's final feature map output.

5. The method for detecting electric targets based on spatial interaction and segmentation attention in small sample sizes according to claim 1, characterized in that, The target detection network under small sample conditions uses AKS 2 -Net serves as the backbone network, connecting the Region Proposal Network (RPN) and the ROI Align to form a two-stage object detection network.

6. The method for detecting electric targets based on spatial interaction and segmentation attention under small sample conditions according to claim 1, characterized in that, The target detection dataset is divided into a basic class dataset and a new class dataset. Each sample in the dataset consists of an input image, a class label, and the bounding box coordinates of the target. Based on the object detection dataset, the object detection network is trained, including: Meta-training is performed on the network parameters of the target detection network based on the aforementioned basic dataset. The parameters of the meta-trained object detection model are fine-tuned based on a balanced dataset containing the base class dataset and the new class dataset. The obtained target detection model is optimized based on the determined overall network loss function; wherein, the overall network loss function includes the FocalLoss loss function and the bounding box regression loss function.

7. The method for detecting electric targets based on spatial interaction and segmentation attention under small sample conditions according to claim 1, characterized in that, Using the FocalLoss loss function as the classification loss function, it is expressed as: ， In the formula, It corresponds to the category c The weight parameters are used to adjust the importance of samples of the corresponding category; It is the decay parameter, used to adjust the weights of easy and difficult samples; when p c When it approaches 1, The term tends to 0. p c Indicates the category in the output probability distribution c The probability value; K Indicates the number of categories. x This represents the output vector of the fully connected layer. x i Represents the first in the vector i Each element.

Images