A method for detecting small defects in power scenarios based on Gaussian mask supervision and cross-layer attention guidance

Abstract

This invention discloses a method for detecting small defect targets in power scenes based on Gaussian mask supervision and cross-layer attention guidance. The method involves inputting a power scene image into a detection model, then extracting an initial feature map using a backbone network. This initial feature map is then processed through multi-stage feature extraction via convolutional paths. The multi-stage features are further processed by a path aggregation network, outputting multiple fused feature maps with different feature levels from shallow to deep. Based on cross-scale window attention, the deep fused feature map with high semantic value between every two adjacent fused feature maps guides the shallow fused feature map in semantic information modeling. After obtaining multiple output feature maps, a multi-branch detection head processes them separately and outputs the prediction results. This invention integrates shallow feature activation prediction and cross-scale window attention guidance, effectively improving the robustness and localization accuracy of detecting small fault targets in UAV inspection images.

Description

Technical Field

[0001] This invention relates to the field of fault detection, and in particular to a method for detecting small defects in power scenarios based on Gaussian mask supervision and cross-layer attention guidance. Background Technology

[0002] In the daily operation of power systems, fault detection is a crucial technical aspect of maintaining grid stability. However, for high-voltage transmission lines, minute defects often appear on the surface of high-altitude cables, making them difficult to detect by human visual inspection. Furthermore, the image quality of cables captured in adverse weather conditions significantly degrades, leading to a substantial decrease in the accuracy and efficiency of drone inspections.

[0003] Current research has proposed a cable fault detection model, YOLOv9-SED, based on an improved version of YOLOv9. First, an UnfogNet dehazing network is added to the original YOLOv9 model, effectively enhancing image clarity in complex and harsh high-altitude environments. Simultaneously, a SEAM attention mechanism and a Shape-IoU loss function are introduced to improve the model's feature extraction capability for small targets. Finally, DualConv convolutions replace the original Conv convolutional layers, improving model performance while reducing model complexity. However, YOLOv9-SED primarily relies on the feature extraction capabilities of deep networks. While shallow feature maps (such as low-level convolutional outputs) contain rich details (such as small target edges and textures), they lack target activation map prediction via an auxiliary head. This limits the detection accuracy of small defects in complex mountainous scenes. Furthermore, supervision is only provided through conventional bounding boxes and classification losses, without introducing a ground truth-based feature map supervision mechanism, resulting in insufficient sensitivity to target location, especially prone to missed detections in complex backgrounds or low-contrast scenes.

[0004] Meanwhile, other studies have improved the Neck part of YOLOv8 by introducing the C2f-SCConv module (combining cross-stage partial links and spatial channel reconstruction convolutions) and the BiFPN structure to enhance multi-scale feature fusion capabilities. Simultaneously, a global attention mechanism (GAM) is embedded at the end of the backbone network to suppress noise, and the Wise-IoU loss function is used to optimize bounding box regression accuracy. However, this approach suffers from low utilization of shallow features: although the improved model integrates multi-scale features through BiFPN, shallow features (such as low-level convolutional outputs) are not specifically enhanced, resulting in underutilization of high-resolution detail information and affecting the localization accuracy of small targets. Dynamic context modeling is also lacking: although the GAM mechanism enhances global feature weight allocation, it does not explicitly establish local spatial relationships between cross-layer features. For example, deep features cannot directly guide the local region attention adjustment of shallow features, potentially leading to a higher false detection rate in complex backgrounds. Summary of the Invention

[0005] To address the issues of low accuracy in small target detection and severe interference from complex backgrounds in current UAV inspection scenarios for power lines in mountainous areas, which often result in missed or false detections, this invention provides a method for detecting small defects in power scenes based on Gaussian mask supervision and cross-layer attention guidance. This method integrates shallow feature activation prediction and cross-scale window attention guidance, effectively improving the robustness and localization accuracy of detecting minute fault targets in UAV inspection images.

[0006] To achieve the above-mentioned technical objectives, the technical solution of the present invention is as follows:

[0007] A method for detecting small defect targets in power scenarios based on Gaussian mask supervision and cross-layer attention guidance includes:

[0008] Step 1: Input the power scene image into the detection model. Then, the backbone network of the detection model first extracts the initial feature map. Then, multi-stage feature extraction is performed on the initial feature map according to the convolution path, and the feature map obtained in each stage is output respectively, so as to obtain multiple feature maps at different scales. The feature maps formed based on the order of the stages are shallow feature maps and deep feature maps from shallow to deep.

[0009] Step 2: Based on the path aggregation network, a top-down path is first used to upsample the deep feature maps in every two adjacent feature maps obtained in Step 1 and then concatenate them with the shallow feature maps to obtain multiple intermediate feature maps with different feature levels from deep to shallow. Then, a bottom-up path is used to downsample the shallow intermediate feature maps in every two adjacent intermediate feature maps and then concatenate them with the deep intermediate feature maps to finally output multiple fused feature maps with different feature levels from shallow to deep.

[0010] Step 3: Based on cross-scale window attention, the deep fusion feature map with high semantics between every two adjacent fusion feature maps is used to guide the shallow fusion feature map in semantic information modeling. That is, a key vector Key and a value vector Value are generated based on the deep fusion feature map, and a query vector Query is generated based on the shallow fusion feature map. Then, the Key, Value, and Query are divided into windows and split into multiple heads using a projection matrix. Then, a relative position bias is introduced to calculate the multi-head attention between Query and Key. The multi-head attention result is multiplied with the multi-head attention of Value, and then concatenated and mapped by the output projection matrix to obtain the output. The output is then processed by channel gating and added to the shallow fusion feature map to obtain the output feature map. Finally, multiple output feature maps are obtained.

[0011] Step 4: Process the multiple output feature maps obtained in Step 3 using a multi-branch detection head and output the prediction results.

[0012] Furthermore, in the method, step 1 includes:

[0013] The model uses CSPDarkNet as the backbone network, and compresses the size of the input power grid image and increases the dimensionality of the channels through the initial layer composed of convolutional modules in the backbone network to obtain the initial feature map.

[0014] The initial feature map is then input into the convolutional path for feature extraction. The convolutional path includes multiple CSP network modules stacked in series, and each CSP network module is used as a single stage for performing feature extraction. The feature map extracted in each stage is retained, thus obtaining multiple feature maps with different resolutions, i.e. different scales. The higher resolution is the shallow feature map with more texture details, and the lower resolution is the deep feature map with more semantic information.

[0015] Furthermore, in the method described above, step 2, the process of obtaining multiple intermediate feature maps with different feature levels from deep to shallow includes:

[0016] The resolution of the deep feature map is adjusted by upsampling to match that of the shallow feature map extracted in the adjacent stage. Then, the two are concatenated along the channel dimension, and the number of channels is adjusted by a CSP module formed by a single CSP structure to obtain the corresponding intermediate feature map, expressed as:

[0017] ;

[0018] in This represents the intermediate feature map of the i-th layer formed in the top-down fusion path. Indicates nearest neighbor upsampling. This indicates splicing along the channel dimension. Indicates CSP module, The first output of the backbone network Layer feature map, The first output of the backbone network Layer feature map, The number of channels is adjusted by the CSP module, and Maintain consistency;

[0019] Step 2, the process of outputting multiple fused feature maps with different feature levels from shallow to deep, includes:

[0020] The shallow intermediate feature map between two adjacent intermediate feature maps is downsampled using a convolution with a stride of 2, then batch normalized and activated by an activation function, and finally concatenated with the deep intermediate feature map along the channel dimension. The number of channels is adjusted using the CSP module to achieve fusion and obtain a fused feature map. The expression is as follows:

[0021] ;

[0022] in This represents the fused feature map of the i-th layer. This represents the activation function. This indicates Batch Normalization. This represents a 3×3 convolution.

[0023] Furthermore, in the method described above, step 3 includes:

[0024] The following formula is used to generate a key vector (Key) and a value vector (Value) based on a deep fusion feature map with high semantics from every two adjacent fusion feature maps:

[0025] ;

[0026] in Indicates Key, Represents Value, Represents a real matrix. express and The number of channels, express and of high, express and width, Represents the deep fusion feature map. , This indicates that the device is divided along the channel dimension into two parts with an equal number of channels. Includes upsampling and 1×1 convolutional projection. Upsampling for nearest neighbor interpolation;

[0027] The following formula is used to generate the query vector Query based on the shallow fusion feature map of every two adjacent fusion feature maps:

[0028] ;

[0029] in Indicates Query, This represents a 1×1 convolutional projection; This represents a shallow fusion feature map. ;

[0030] Then , , Divided into Non-overlapping windows:

[0031] ;

[0032] in For window size, and These represent the deep fusion feature maps respectively. Height and width;

[0033] Then, based on the query matrix corresponding to each window Key matrix Sum matrix To calculate multi-head attention, for the m-th head:

[0034] ;

[0035] in , and These are the query matrix, key matrix, and value matrix of the m-th header after processing with the projection matrix; , and These are the query projection matrix, key projection matrix, and value projection matrix, respectively, all of which are parameters learned in the model. , For the dimension of attention head, , Represents deep fusion feature maps The number of channels, The number of long positions. , and These represent the query matrix, key matrix, and value matrix of the m-th header within the window, respectively.

[0036] Then, the relative position offset obtained through learning is introduced. To calculate and Attention matrix ,in Used to compensate for the position insensitivity of attention mechanisms :

[0037] ;

[0038] in This represents the softmax normalization function. The superscript T in the matrix indicates matrix transpose. This indicates the offset of the position of the m-th head;

[0039] Then With the corresponding Multiply by each product to obtain the result for the m-th head. Then, the settlement results of all heads are merged and processed through the output projection. After mapping, we obtain fused attention-enhanced features. :

[0040] ;

[0041] ;

[0042] in To output the projection matrix, Finally, attention-enhanced features are fused through gated residual connections. Feature map fused with the original shallow layer This yields the output feature map obtained from every two adjacent fused feature maps:

[0043] ;

[0044] in To output the feature map, This represents the Sigmoid non-linear activation function. For channel attention gating, This indicates element-wise multiplication.

[0045] Furthermore, in the method described above, step 4 includes:

[0046] Based on the enhanced multi-scale features, a multi-branch detection head is used for the final prediction output of the model. For the output features at N scales, N detection heads are used to process and output the prediction results respectively. All detection heads adopt a decoupled structure, including two branches: a classification branch and a regression branch. Both branches consist of a standard 3×3 convolutional module and a 1×1 element-wise convolutional module, and output the class score and the prediction box respectively.

[0047] Furthermore, in the method described above, in step 1, the feature map output by the last stage of the backbone network is directly used as the last deep intermediate feature map of the bottom-up path of the path aggregation network in step 2; in step 2, the last intermediate feature map generated by the top-down path of the path aggregation network is directly used as the first shallow fusion feature map used to generate the query in step 3; and in step 3, the last fusion feature map generated by the bottom-up path of the path aggregation network is directly used as the output feature map processed by the detection head in step 4.

[0048] Furthermore, in the method described above, during the training process of the detection model, when performing step 2, the method further includes generating a predicted target activation mask based on the shallowest fusion feature map output by the path aggregation network, generating a mask for a single target region that follows a Gaussian distribution based on the ground truth, supervising the predicted target activation mask, calculating the loss between the two, and using this step to enhance the activation degree of small target regions in the shallow fusion feature map:

[0049] The shallowest layer of fused feature maps After two upsampling blocks, a reconstructed image is generated through standard 1×1 convolution; each upsampling block includes four steps: standard 3×3 convolution, batch normalization, SiLU activation function, and bilinear interpolation upsampling.

[0050] ;

[0051] in To activate the target, For each upsampling block, the feature information is processed and upsampled by a factor of 2 once. Represents a 1×1 convolution;

[0052] Then, a Gaussian supervised mask is generated based on the ground truth: for the set of ground truth bounding boxes of the input image... Where N is the total number of target samples in the image, and the th... target bounding box It is represented by the coordinates of the top left and bottom right corners of the frame in the image, that is... , express The coordinates of the top left corner of the image, express The coordinates of the bottom right corner of the image; first, set the target filtering criteria for the ground truth, and only retain targets with an absolute pixel area smaller than 32×32 in the mask:

[0053] ;

[0054] in Represents a subset of small targets. Indicates the first Area of ​​the region covered by each target bounding box:

[0055] ;

[0056] in Indicates the first The width of the area covered by the target bounding box Indicates the first The height of the area covered by the target bounding box;

[0057] Subsequently based on sets Constructing an anisotropic Gaussian kernel: For each valid target, first establish a two-dimensional Gaussian probability density function. :

[0058] ;

[0059] Where exp is an exponential function. The coordinates of the valid target; For the first The center coordinates of each target are represented as follows:

[0060] ;

[0061] Two-dimensional Gaussian probability function The standard deviations in the x and y directions are calculated independently to achieve an anisotropic Gaussian distribution in two dimensions, matching the aspect ratio characteristics of the target. The covariance matrix of a single effective target is modeled using Gaussian modeling. for:

[0062] ;

[0063] The standard deviation, proportional to the target size, is calculated using the following formula:

[0064] ;

[0065] in , This refers to the proportional coefficient hyperparameter.

[0066] Finally, Gaussian mixture overlay and normalization are performed to generate the Gaussian mixture response map. :

[0067] ;

[0068] Then, the Gaussian mixture response map is normalized by pixel-by-pixel numerical truncation to calculate the final response mask. :

[0069] ;

[0070] Finally obtained ,in and To detect the length and width of the input image of the model, the response mask is used as a Gaussian supervision mask to supervise the prediction mask based on shallow feature maps;

[0071] Then, based on the predicted target activation mask and the Gaussian supervision mask, calculate their L1 loss functions. That is, the mask prediction loss, which is used as part of the total loss in gradient backpropagation and parameter updates:

[0072] ;

[0073] in and All represent numbers, i.e., the first number. The and the first indivual.

[0074] Furthermore, the method described above, during model training, results in a total loss of... It consists of three parts: classification loss, bounding box regression loss, and mask prediction loss.

[0075] ;

[0076] in Predict loss using a mask; For classification loss, cross-entropy loss is used; For the bounding box regression loss, a combination of DFL loss and CIOU loss is used; These are the weight hyperparameters of the loss function;

[0077] Calculate the total loss of the prediction results Then, the Adam optimizer is used to iteratively optimize the detection model and update the model parameters to reduce the total prediction loss of the model.

[0078] This invention also provides an electronic device, comprising:

[0079] One or more processors;

[0080] Storage device for storing one or more programs;

[0081] The one or more processors execute the one or more programs to implement the methods described above.

[0082] The present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method as described above.

[0083] The technical advantages of this invention lie in its targeted research on two major technical problems in UAV inspection scenarios of power lines in mountainous areas: low accuracy in detecting small targets and severe interference from complex backgrounds. Specifically, to address issues such as weak representation of small targets in shallow feature maps and insufficient interaction between deep semantic features and shallow detail features, this invention proposes a cross-level window attention mechanism. This mechanism divides the deep feature map into non-overlapping local windows, generates a spatial weight matrix through multi-head attention calculation, and dynamically modulates the response intensity of the corresponding local regions in the shallow feature map. Specifically, the shallow feature window acts as the query to retrieve relevant semantics, while the higher-level feature region acts as the key and value to provide contextual information for higher-level semantics. Attention weights are generated after Softmax normalization, enhancing the feature response of the target region and suppressing background noise. Compared to global attention, window partitioning significantly reduces computational complexity, and its local modeling characteristics are more suitable for the sparse distribution of targets in mountainous scenes. This module can significantly improve the detection accuracy of small targets in complex backgrounds and significantly reduce the false detection rate.

[0084] This invention addresses the problem of missed detection of small targets caused by traditional methods neglecting high-resolution details of shallow features. During training, after the model's feature fusion layer, an auxiliary prediction head is added to the shallow feature branch, converting it into a target activation mask. This is then supervised by a Gaussian mask generated based on ground truth. This design forces the model to learn the spatial distribution features of small targets through the L1 loss function, rather than relying solely on bounding box regression. Compared to conventional supervision mechanisms, the Gaussian mask covers the target's central region and appropriately includes the context of small targets, effectively mitigating the localization sensitivity problem caused by slight offsets in the bounding boxes. This improves the robustness of small target detection and, to some extent, avoids the problem of insufficient optimization of shallow features due to gradient vanishing.

[0085] The invention will now be further described with reference to the accompanying drawings. Attached Figure Description

[0086] Figure 1 This is a schematic diagram of the overall framework of the present invention;

[0087] Figure 2 This is a schematic diagram of the Gaussian supervision mechanism based on ground values ​​and shallow feature maps in this invention;

[0088] Figure 3 This is a schematic diagram of the multi-head attention mechanism for cross-layer windows based on channel gating in this invention;

[0089] Figure 4 This is a schematic diagram of the process of the present invention.

[0090] Figure 5This is a schematic diagram showing the results of small target detection in the same power scenario based on DINO, YOLOv8, and the present invention. Detailed Implementation

[0091] See Figure 1 The small target defect detection method for power scene based on Gaussian mask supervision and cross-layer attention guidance disclosed in this embodiment involves inputting a dataset collected by UAV and finely annotated by professionals into the detection model during training.

[0092] The detection model first extracts raw features through a backbone network, which then outputs feature maps at four scales. Specifically, this embodiment uses CSPDarkNet as the backbone network to extract features from the input power grid image. The backbone network first compresses the size of the input image and increases the dimensionality of the channels through a starting layer composed of convolutional modules, thus obtaining an initial feature map. In this embodiment, the initial feature map has a height of 160, a width of 160, and 128 channels, i.e., 160×160×128. The backbone network then performs feature extraction through multiple cascaded CSP network modules based on a stacked CSP structure, retaining the feature map obtained from each CSP network module, i.e., each feature extraction stage, and finally outputs four feature maps at different resolutions. , , and The shallow feature map contains richer texture details than the deep feature maps in subsequent stages, but its semantic information is insufficient and it is also noisy; while the deep feature map has undergone sufficient feature extraction and contains rich semantic information, but its resolution is low and it ignores useful high-frequency details in the image.

[0093] Next, the detection model in this embodiment fully fuses the enhanced multi-scale feature information based on the path aggregation network, thereby propagating the strong semantic information of deep features to the shallow layer, while simultaneously propagating the high-resolution detail features of the shallow layer to the deep layer. To achieve interaction between feature maps of different scales, this embodiment fuses feature maps of different scales based on the path aggregation network, enriching the feature representation at each level. The processing methods mainly include top-down and bottom-up paths. The path aggregation network PANet first uses a top-down path, upsampling the deep feature maps from every two adjacent stages of the feature maps output by the backbone network and concatenating them with the shallow feature maps to obtain multiple intermediate feature maps with different feature levels from deep to shallow. Then, a bottom-up path is used, downsampling the shallow intermediate feature maps from every two adjacent intermediate feature maps and concatenating them with the deep intermediate feature maps, ultimately outputting multiple fused feature maps with different feature levels from shallow to deep.

[0094] Specifically, the path aggregation network first processes the feature maps of the four stages output by the backbone network based on a top-down path. The deeper feature maps in every two adjacent stages are first upsampled to adjust the resolution, so that the resolution is similar to that of the shallow features in adjacent stages. Figure 1 Then, the two are concatenated along the channel dimension, and the number of channels is adjusted through a CSP module formed by a single CSP structure, the expression of which is:

[0095] ;

[0096] in This represents the intermediate feature map of the i-th layer formed in the top-down fusion path. Indicates nearest neighbor upsampling. This indicates splicing along the channel dimension. Indicates CSP module, The first output of the backbone network Layer feature map, The first output of the backbone network Layer feature map, The number of channels is adjusted by the CSP module, and Maintain consistency.

[0097] In the bottom-up path of the path aggregation network, a convolution with a stride of 2 is first used to downsample the shallow intermediate feature maps between two adjacent intermediate feature maps. Then, batch normalization and activation are performed, followed by concatenation with the deep intermediate feature maps along the channel dimension. The CSP module is used to adjust the number of channels, thereby achieving fusion to obtain the fused feature map. The specific expression is as follows:

[0098] ;

[0099] in Indicates the first Layer fusion feature map, This represents the activation function. This indicates Batch Normalization. This represents a 3×3 convolution.

[0100] See Figure 2 In the subsequent training process, a target activation mask is predicted based on the shallowest feature map, and a mask for a single target region following a Gaussian distribution is generated based on the ground truth. The predicted mask is supervised, and the loss between the two is calculated to enhance the activation of small target regions in the shallow feature map. This training process is not repeated after training is completed.

[0101] Since small target information is mainly located in the shallow, high-resolution fused feature map, the prediction performance for small targets depends on the quality of the shallow fused feature map. Therefore, to ensure that the small target regions in the shallow fused feature map are fully activated while the background regions maintain low response, this embodiment designs a shallow feature target reconstruction module based on the shallowest fused feature map output by the path aggregation network. The information of small targets is reconstructed, and a Gaussian mask for small targets is established based on the ground truth and combined with anisotropic Gaussian kernels. The prediction mask is supervised to guide the model to focus on the central features of the target.

[0102] Specifically, the target mask is first predicted based on the shallow feature map. (Shallowest layer feature information) Input to the detection model In comparison, after a 4x downsampling, the original feature information distribution in the feature map should be preserved while restoring the resolution during feature reconstruction. Specifically, this embodiment designs a reconstruction module that generates a reconstructed map by passing the feature map through two upsampling blocks and then through a standard 1×1 convolution. Each upsampling block (layer) includes four steps: standard 3×3 convolution, batch normalization, SiLU activation function, and bilinear interpolation upsampling.

[0103] ;

[0104] in To activate the target, For each upsampling block, the feature information is processed and upsampled by a factor of 2 once. This represents a 1×1 convolution.

[0105] Then, a Gaussian supervised mask is generated based on the ground truth: for the set of ground truth bounding boxes of the input image... Where N is the total number of target samples in the image, and each target bounding box Represented by the coordinates of the top-left and bottom-right corners of the box in the image: , This indicates the coordinates of the top-left corner of the box in the image. This represents the coordinates of the bottom right corner of the bounding box in the image. First, target filtering conditions are set. Since the main function of shallow feature maps is to identify very small targets, such as small targets (small, pixel area < 32×32) and tiny targets (tiny, pixel area < 16×16), the mask only retains targets with an absolute pixel area smaller than 32×32.

[0106]

[0107] in Represents a subset of small targets. This represents the area covered by the bounding box of the i-th target:

[0108] ;

[0109] in Indicates the first The width of the area covered by the target bounding box Indicates the first The height of the area covered by the target bounding box.

[0110] Subsequently based on sets Constructing an anisotropic Gaussian kernel: For each effective target, establish a two-dimensional Gaussian probability density function. :

[0111] ;

[0112] Where exp is an exponential function. The coordinates of the valid target; Let the center coordinates of the i-th valid target be:

[0113] ;

[0114] Two-dimensional Gaussian probability function The standard deviations in the x and y directions are calculated independently to achieve an anisotropic Gaussian distribution in two dimensions, matching the aspect ratio characteristics of the target. The covariance matrix of a single effective target is modeled using Gaussian modeling. for:

[0115] ;

[0116] in The standard deviation is proportional to the target size:

[0117]

[0118] , As a scaling factor hyperparameter, in this embodiment, Set it to 0.5.

[0119] Finally, Gaussian mixture overlay and normalization are performed to generate the Gaussian mixture response map. :

[0120] ;

[0121] To avoid numerical overflow that could lead to training instability, the Gaussian response map is further normalized. However, directly using standard normalization might compress the value range of isolated small target regions to near zero. Therefore, this embodiment uses pixel-by-pixel numerical truncation instead of explicit normalization to calculate the final response map, i.e., the response mask. :

[0122] ;

[0123] Finally obtained ,in and In this embodiment, to detect the length and width of the input image to the model, , This response mask This is used to supervise the prediction of masks based on shallow feature maps. This normalization is not a strict probability normalization, but by restricting the value range to [0,1], the mask can directly provide a stable supervision signal, ensuring that isolated target regions also have high signal strength.

[0124] After obtaining the prediction mask and the Gaussian supervised mask, calculate their L1 loss functions. That is, the mask prediction loss, which will be used as part of the total loss in gradient backpropagation and parameter updates:

[0125] ;

[0126] in and All represent numbers, i.e., the first number. The and the first indivual.

[0127] The above steps involve predicting the target activation mask and calculating the L1 loss function. These are all steps executed during training, and will not be executed again after training is completed.

[0128] See Figure 3Next, the detection model further processes the fused feature maps. Based on the cross-scale window attention module, it uses deep, high-semantic feature maps to guide shallow feature maps in semantic information modeling, and uses channel attention gating to strengthen important features. Specifically, based on cross-scale window attention, the deep fused feature map with high semantics between every two adjacent fused feature maps guides the shallow fused feature map in semantic information modeling. This involves generating key and value vectors based on the deep fused feature maps and a query vector based on the shallow fused feature maps. Then, the key, value, and query are divided into windows, and further split into multiple heads using a projection matrix. A relative positional bias is then introduced to calculate the multi-head attention between the query and key. The resulting multi-head attention is multiplied by the multi-head attention of the value, concatenated, and mapped through the output projection matrix to obtain the output. The output is then processed by channel gating and added to the shallow fused feature map to obtain the final output feature map. Finally, multiple output feature maps are obtained.

[0129] Specifically, with As a shallow fusion feature map among two adjacent fusion feature maps This is a deep feature map. Therefore, a query is generated based on the shallow feature map.

[0130] ;

[0131] in Indicates Query, This represents a 1×1 convolutional projection; This represents a shallow fusion feature map. .

[0132] Generating K and V based on deep feature maps:

[0133] ;

[0134] in Indicates Key, Represents Value, Represents a real matrix. Indicates the number of channels. Indicates high, Indicates width. Represents the deep fusion feature map. , Indicates batch size. This indicates that the device is divided along the channel dimension into two parts with an equal number of channels. Includes upsampling and 1×1 convolutional projection. Upsampling is performed for nearest neighbor interpolation.

[0135] Then divide the above Q, K, and V into Non-overlapping windows:

[0136] ;

[0137] in For window size, and These represent the deep fusion feature maps respectively. The height and width. The corresponding query matrix within each window. Key matrix Sum matrix Based on these, multi-head attention is calculated, then for the m-th head:

[0138] ;

[0139] in , and These are the query matrix, key matrix, and value matrix of the m-th header after processing with the projection matrix; , and These are the query projection matrix, key projection matrix, and value projection matrix, respectively, all of which are parameters learned in the model. , For the dimension of attention head, , Represents deep fusion feature maps The number of channels, The number of long positions. , and These represent the query matrix, key matrix, and value matrix of the m-th header within the window, respectively.

[0140] Then, the relative position offset obtained through learning is introduced. To calculate and Attention matrix ,in Used to compensate for the position insensitivity of attention mechanisms :

[0141] ;

[0142] in This represents the softmax normalization function. The superscript T in the matrix indicates matrix transpose. This indicates the offset of the position of the m-th head;

[0143] The attention is then multiplied by the V of the corresponding attention head, and the multi-head results are combined. After output projection mapping, the final output is obtained:

[0144] ;

[0145] ;

[0146] in To output the projection matrix, Finally, attention-enhanced features are fused through gated residual connections. Feature map fused with the original shallow layer This yields the output feature map obtained from every two adjacent fused feature maps:

[0147] ;

[0148] in To output the feature map, This represents the Sigmoid non-linear activation function. For channel attention gating, This indicates element-wise multiplication.

[0149] Based on the enhanced multi-scale features, a multi-branch detection head is used to perform the final prediction output of the model. For the output features at N scales in the above steps (N is 4 in this embodiment), N detection heads are used to process and output the prediction results respectively. All detection heads adopt a decoupled structure, mainly consisting of two branches: a classification branch and a regression branch. The two branches have similar structures, each consisting of a standard 3×3 convolutional module and a 1×1 element-wise convolutional module.

[0150] It's worth mentioning that in this embodiment, the feature map output by the last stage of the backbone network is directly used as the last deep intermediate feature map of the bottom-up path of the path aggregation network. The last intermediate feature map generated by the top-down path of the path aggregation network is directly used as the first shallow fusion feature map for generating the query. The last fusion feature map generated by the bottom-up path of the path aggregation network is directly used as the output feature map processed by the detection head.

[0151] During the model training phase, a weighted loss function is designed to calculate the error between the model output and the true value, and the mask prediction loss from step 4 is added to the loss. Finally, the model parameters are updated based on the backpropagation algorithm.

[0152] As shown in the following formula, the total loss consists of three parts: classification loss, bounding box regression loss, and mask prediction loss.

[0153] ;

[0154] This is the mask prediction loss; For classification loss, cross-entropy loss is used; For the bounding box regression loss, a combination of DFL loss and CIOU loss is used. In the formula... These are the weight hyperparameters of the loss function.

[0155] After calculating the loss of the prediction results, the Adam optimizer is used to iteratively optimize the model, update the model parameters, reduce the total prediction loss of the model, and improve the prediction accuracy.

[0156] This invention considers that defects in power grid scene images are mostly small targets, and the information of small targets is mainly in shallow, high-resolution feature maps. To minimize the interference of background noise while retaining the effective information of small target regions, this invention introduces a lightweight auxiliary head after the feature fusion layer of the model. This auxiliary head takes the shallowest feature map as input and predicts and outputs a target region activation map. This image is used to represent the activation level of small target regions in shallow feature maps, and a class-independent ground truth heatmap is generated based on the ground truth labels. As a supervisory signal for this auxiliary head, it helps the model learn the ability to preserve foreground features and suppress background-irrelevant information.

[0157] This invention also considers the presence of very small defects in power grid scenarios. Due to their extremely small size and lack of discriminative features, the contextual clues of their surrounding environment are crucial. To address this, instead of using a simple 0-1 mask, this invention establishes a small target distribution mask using an anisotropic Gaussian kernel. This mask, predicted based on shallow feature maps, is then used to supervise the model, guiding it to focus on the central features of the small target. Furthermore, thanks to the properties of the Gaussian distribution, the key contextual regions surrounding the target are also incorporated into the supervision signal, ensuring a sufficiently strong supervision signal. The loss between the ground truth Gaussian mask and the predicted activation mask is used as part of the model's final loss, participating in backpropagation and model parameter iteration. Moreover, this detection head module can be discarded during the inference phase; this mechanism does not increase the number of parameters or computational cost during model inference.

[0158] Because the activation level of small targets with defects is not high in shallow feature maps, while the small target regions in deep feature maps still maintain high activation levels, but the boundaries are relatively blurred, this invention designs a guidance module based on cross-scale window attention, that is, generating queries based on shallow layers and generating keys and values ​​based on deep layers. Unlike existing methods that rely on self-attention or feature enhancement based on themselves, this embodiment aims to use the higher semantic information in deep feature maps to guide shallow feature maps in context modeling, thereby enhancing the feature representation and feature strength of small targets in shallow feature maps, thus effectively improving the model's detection performance for small targets. After multi-head attention operations, important channel features are strengthened and irrelevant channel features are suppressed through a channel attention gating mechanism. In addition, to ensure that the module does not introduce a large amount of additional computation, this embodiment uses a windowed attention mechanism to guide the shallow layers.

[0159] After the detection model is trained, corresponding power grid scene photos can be collected and input into the detection model for processing, thereby outputting defect detection results based on the detection model.

[0160] Meanwhile, using a typical power scenario defect dataset, defect detection was performed based on existing methods such as CascadeRCNN, DetectoRS, VFNet, DINO, YOLOv8, and the method provided in this invention. The detection results are shown in Table 1 below, where mAP represents the overall average accuracy. 50 This indicates a more lenient standard for correctly detecting a bounding box when the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box is ≥ 0.5. (mAP) 75 This indicates a relatively loose criterion for correct detection: an IoU ≥ 0.75 between the predicted bounding box and the ground truth bounding box. mAP t mAP vt mAP s and mAP m These represent the precision of minute targets, extremely minute targets, small targets, and medium-sized targets, respectively.

[0161] .

[0162] As shown in Table 1, experimental results on a typical power scenario defect dataset demonstrate that the detection accuracy of the proposed method is significantly higher than that of existing mature algorithms, especially in mAP. vt (Extremely small target), mAP t (Small target), mAP s The algorithm significantly outperforms other algorithms in the (small target) performance indicator, demonstrating the feasibility and effectiveness of this invention in small target defect detection in power scenarios.

[0163] See Figure 5 , Figure 5 The first column in the table represents the true values, i.e., the actual labels. The labels in the boxes represent the actual defects, which include cable joint insulation covers falling off, circuit breaker insulation covers being missing, pins being missing, and tower top damage. Figure 5 The second and third columns show the detection results of typical advanced target detection algorithms, including DINO and YOLOv8, while the fourth column shows the detection results of the method proposed in this invention. It is evident that DINO and YOLOv8 both exhibit varying degrees of missed detections and identification errors, while the method proposed in this invention can more accurately locate small-sized defect targets in power scenarios.

[0164] According to embodiments of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0165] Electronic devices include:

[0166] One or more processors;

[0167] Storage device for storing one or more programs.

[0168] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0169] In practical use, users can interact with servers, which are also electronic devices, via a network to receive or send messages. Terminal devices are generally various electronic devices equipped with a display and used through a human-computer interface, including but not limited to smartphones, tablets, laptops, and desktop computers. Various specific application software can be installed on these terminal devices as needed, including but not limited to web browsers, instant messaging software, social media platforms, and shopping apps.

[0170] A server is a network service that provides various services. The small target defect detection method for power scenarios provided in this embodiment is generally executed by the server. In practical applications, under necessary conditions, terminal devices can also directly execute the small target defect detection method for power scenarios.

[0171] Similarly, the computer-readable medium of the present invention stores a computer program thereon, which, when executed by a processor, implements a method for detecting small targets in power scene defects according to an embodiment of the present invention.

Claims

1. A method for detecting small defect targets in power scenarios based on Gaussian mask supervision and cross-layer attention guidance, characterized in that, include: Step 1: Input the power scene image into the detection model. Then, the backbone network of the detection model first extracts the initial feature map. Then, multi-stage feature extraction is performed on the initial feature map according to the convolution path, and the feature map obtained in each stage is output respectively, so as to obtain multiple feature maps at different scales. The feature maps formed based on the order of the stages are shallow feature maps and deep feature maps from shallow to deep. Step 2: Based on the path aggregation network, a top-down path is first used to upsample the deep feature maps in every two adjacent feature maps obtained in Step 1 and then concatenate them with the shallow feature maps to obtain multiple intermediate feature maps with different feature levels from deep to shallow. Then, a bottom-up path is used to downsample the shallow intermediate feature maps in every two adjacent intermediate feature maps and then concatenate them with the deep intermediate feature maps to finally output multiple fused feature maps with different feature levels from shallow to deep. Step 3: Based on cross-scale window attention, the deep fusion feature map with high semantics between every two adjacent fusion feature maps is used to guide the shallow fusion feature map to perform semantic information modeling. That is, a key vector Key and a value vector Value are generated based on the deep fusion feature map, and a query vector Query is generated based on the shallow fusion feature map. Then, the Key, Value and Query are divided into windows, and then split into multiple heads through the projection matrix. Then, the relative position bias is introduced to calculate the multi-head attention of Query and Key. The multi-head attention result is multiplied with the multi-head attention of Value, and then concatenated and mapped by the output projection matrix to obtain the output. The output is processed by channel gating and then added to the shallow fusion feature map to obtain the output feature map. Ultimately, multiple output feature maps are obtained; Step 4: Process the multiple output feature maps obtained in Step 3 using a multi-branch detection head and output the prediction results respectively; Step 3 includes: The following formula is used to generate a key vector (Key) and a value vector (Value) based on a deep fusion feature map with high semantics from every two adjacent fusion feature maps: Where K represents Key and V represents Value. Let K represent a real matrix, C represent the number of channels in K and V, H represent the height of K and V, and W represent the width of K and V. Represents the deep fusion feature map. `split(·)` means dividing the channel along the channel dimension into two parts with an equal number of channels, φ k,v (·) includes upsampling and 1×1 convolution projection, ↑(·) is nearest neighbor interpolation upsampling; The following formula is used to generate the query vector Query based on the shallow fusion feature map of every two adjacent fusion feature maps: Where Q represents Query, φ q (·) represents a 1×1 convolution projection; This represents a shallow fusion feature map. Then Q, K, and V are divided into N w Non-overlapping windows: Where S is the window size, and H2 and W2 represent the deep fusion feature maps, respectively. Height and width; Then, based on the query matrix Q corresponding to each window... w Key matrix K w Sum matrix V w To calculate multi-head attention, for the m-th head: where Q m K m and V m These are the query matrix, key matrix, and value matrix of the m-th header after processing with the projection matrix; and These are the query projection matrix, key projection matrix, and value projection matrix, respectively, all of which are parameters learned in the model. d h d represents the dimension of the attention head. h =C2 / M, where C2 represents the number of channels in the deep fusion feature map F2, and M is the number of heads in the multi-head model. and These represent the query matrix, key matrix, and value matrix of the m-th header within the window, respectively. Then, the relative position bias B, obtained through learning, is introduced to calculate the attention matrix Attn of Q and K. m B is used to compensate for the position insensitivity of the attention mechanism. Where Softmax represents the softmax normalization function. The superscript T denotes matrix transpose, B m This indicates the offset of the position of the m-th head; Then Attn m With the corresponding V m Multiply by each other to obtain the settlement result Z for the m-th head. m Then, the settlement results of all heads are merged and processed through the output projection W. O After mapping, the fused attention-enhanced feature Z is obtained: Z m =Attn m V m ; Z=concat(Z1,…,Z m )IN O ; Among them W O To output the projection matrix, Finally, the attention-enhanced feature Z is fused with the original shallow fused feature map through gated residual connections. The output feature map is obtained from every two adjacent fused feature maps: in For the output feature map, Sigmoid represents the Sigmoid non-linear activation function, g(·) is the channel attention gating, and ⊙ represents element-wise multiplication.

2. The method according to claim 1, characterized in that, Step 1 includes: The model uses CSPDarkNet as the backbone network, and compresses the size of the input power grid image and increases the dimensionality of the channels through the initial layer composed of convolutional modules in the backbone network to obtain the initial feature map. The initial feature map is then input into the convolutional path for feature extraction. The convolutional path includes multiple CSP network modules stacked in series, and each CSP network module is used as a single stage for performing feature extraction. The feature map extracted in each stage is retained, thus obtaining multiple feature maps with different resolutions, i.e. different scales. The higher resolution is the shallow feature map with more texture details, and the lower resolution is the deep feature map with more semantic information.

3. The method according to claim 1, characterized in that, Step 2, the process of obtaining multiple intermediate feature maps with different feature levels from deep to shallow, includes: The resolution of the deep feature map is adjusted by upsampling to match that of the shallow feature map extracted in the adjacent stage. Then, the two are concatenated along the channel dimension, and the number of channels is adjusted by a CSP module formed by a single CSP structure to obtain the corresponding intermediate feature map, expressed as: in This represents the intermediate feature map of the i-th layer formed in the top-down fusion path, where ↑(·) indicates nearest neighbor upsampling, Cat(·) indicates concatenation along the channel dimension, CSP(·) indicates the CSP module, and F i F represents the feature map of the i-th layer output by the backbone network. i+1 This represents the feature map of the (i+1)th layer output by the backbone network. The number of channels is adjusted by the CSP module, and is consistent with F i Maintain consistency; Step 2, the process of outputting multiple fused feature maps with different feature levels from shallow to deep, includes: The shallow intermediate feature map between two adjacent intermediate feature maps is downsampled using a convolution with a stride of 2, then batch normalized and activated by an activation function, and finally concatenated with the deep intermediate feature map along the channel dimension. The number of channels is adjusted using the CSP module to achieve fusion and obtain a fused feature map. The expression is as follows: in Let represent the fused feature map of the i-th layer, σ represent the activation function, BN represent Batch Normalization, and Conv3(·) represent 3×3 convolution.

4. The method according to claim 1, characterized in that, Step 4 includes: Based on the enhanced multi-scale features, a multi-branch detection head is used for the final prediction output of the model. For the output features at N scales, N detection heads are used to process and output the prediction results respectively. All detection heads adopt a decoupled structure, including two branches: a classification branch and a regression branch. Both branches consist of a standard 3×3 convolutional module and a 1×1 element-wise convolutional module, and output the class score and the prediction box respectively.

5. The method according to any one of claims 3-4, characterized in that, In step 1, the feature map output by the last stage of the backbone network is directly used as the last deep intermediate feature map of the bottom-up path of the path aggregation network in step 2; in step 2, the last intermediate feature map generated by the top-down path of the path aggregation network is directly used as the first shallow fusion feature map used to generate the query in step 3; in step 3, the last fusion feature map generated by the bottom-up path of the path aggregation network is directly used as the output feature map processed by the detection head in step 4.

6. The method according to claim 1, characterized in that, During the training process, when performing step 2, the detection model further includes generating a predicted target activation mask based on the shallowest fused feature map output by the path aggregation network, and generating a mask for a single target region that follows a Gaussian distribution based on the ground truth. The predicted target activation mask is then supervised, and the loss between the two is calculated to enhance the activation of small target regions in the shallow fused feature map. The shallowest fused feature map F1 is passed through two upsampling blocks, and then a reconstructed map is generated through a standard 1×1 convolution. Each upsampling block includes four steps: standard 3×3 convolution, batch normalization, SiLU activation function, and bilinear interpolation upsampling. M pre =Conv1(layer(layer(F1))); Where M pre To predict the target activation mask, layer(·) is an upsampling block that processes the feature information and performs a 2x upsampling once. Conv1 represents a 1×1 convolution. Then, a Gaussian supervised mask is generated based on the ground truth: for the set of ground truth bounding boxes of the input image... Where N is the total number of target samples in the image, and b is the bounding box of the i-th target. i It is represented by the coordinates of the top left and bottom right corners of the frame in the image, that is... b i The coordinates of the top left corner of the image, b i The coordinates of the bottom right corner of the image; first, set the target filtering criteria for the ground truth, and only retain targets with an absolute pixel area smaller than 32×32 in the mask: Among them B valid Let A represent a subset of small targets. i This represents the area covered by the bounding box of the i-th target: Where w i h represents the width of the area covered by the bounding box of the i-th target. i Indicates the height of the region covered by the i-th target bounding box; Subsequently based on sets Constructing an anisotropic Gaussian kernel: For each effective target, first establish a two-dimensional Gaussian probability density function G. i (x,y): Where exp is an exponential function, and (x,y) represents the coordinates of the effective target; Let the center coordinates of the i-th target be: Two-dimensional Gaussian probability function G i The standard deviations of (x,y) in the x and y directions are calculated independently, thus achieving an anisotropic Gaussian distribution in two dimensions to match the aspect ratio characteristics of the target. The Gaussian modeling covariance matrix Σ of a single effective target is: The standard deviation, proportional to the target size, is calculated using the following formula: Where α∈(0,1), α is the proportionality coefficient hyperparameter; Finally, Gaussian mixture overlay and normalization are performed to generate the Gaussian mixture response map M. raw (x,y): Then, the Gaussian mixture response map is normalized by pixel-by-pixel numerical truncation, and the final response mask M is calculated. gt (x,y): M gt (x,y)←max(0,min(1,M raw (x,y))); Finally obtained H3 and W3 are the length and width of the input image of the detection model. The response mask is used as a Gaussian supervision mask to supervise the prediction mask based on shallow feature maps. Then, based on the predicted target activation mask and the Gaussian supervision mask, calculate their L1 loss functions. That is, the mask prediction loss, which is used as part of the total loss in gradient backpropagation and parameter updates: Where i and j both represent numbers, i.e. the i-th and j-th elements.

7. The method according to claim 6, characterized in that, In model training, the total loss It consists of three parts: classification loss, bounding box regression loss, and mask prediction loss. in Predict loss using a mask; For classification loss, cross-entropy loss is used; For bounding box regression loss, a combination of DFL loss and CIOU loss is used; α, β, and γ are the weight hyperparameters of the loss function; Calculate the total loss of the prediction results Then, the Adam optimizer is used to iteratively optimize the detection model and update the model parameters to reduce the total prediction loss of the model.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs; The feature is that the one or more processors execute the one or more programs to implement the method as described in any one of claims 1-7.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images